Apps that see

Don’t know if you already tried it, but the app (or actually, the services in azure that are behind the app) will also recognize famous people. Try downloading and analyzing a photo of Michael Jackson or Bill Gates for example; it will recognize and tell who they are.

thank you. that was a good video, structurally, the tie-in with the kurzweil, the crappy intel version, and 10 years later this: wow. also, more generally, love you doing accessibility devices. on this app specifically, keep in mind some images will be sent to microsoft servers. as to your question in the video, i don't think the intel device needs another one, but i hope to see you doing accessibility devices again soon. cheers!

I love these insights into assistive technology of the past. This is the type of stuff you never really see or hear about but I'd imagine it's essential for those who use them.

To me I am interested in this subject because I have got an Aunt who over the years has gone blind. She was 85 when the full darkness struck and just recently turned 99. She is still getting along in her home pretty well (she doesn't need much assistance at all, except with things that where made for the seeing like letters!) I have been trying to get her to use a smartphone but you have to unterstand that it is almost impossible for someone who lived through WWII to fathom the intricacies of a smartphone even with accessibility features enabled and thorough explanation (at least that was the result when I tried a few years ago). Also at least for her the swiping and touching doesn't come natural because of tactile limitation due to age . To those who design such apps I encourage You to make them free to try and offer different accessories to the same issue matched to the special but categorize-able needs of the respective users. And please include the most mundane functions you think may be covered by other apps but with a little ears only twist. For example a recorder where you can also record a title for that is played when you navigate over the entry. @Nosh +1

My curiosity has gotten the better of me - I must know how old this app thinks my fat face is. May I ask anyone for any indulgence? (Edited to remove superfluousness)

I made a comment on YT but thought it might get lost in the crowd. I'm an a11y (accessibility) engineer and I love that more attention in being given to this area of technology. I think some people assume that the right people just know these products exist but the truth is many who could benefit from them don't. For example Google has it's own free app "Lookout By google" that does much of the same things the MS app does and it is also free.

I wonder if there's a reason it is iOS only right now? Is it using the cloud for processing or is it running the AI locally?

Wow, that app is a lot of fun. My eyes aren't that bad, but could see this being really useful for many.

I downloaded the app. Holy cow. If this is the free-here-you-go version what is the "real" version like? Pretty scary. BUT ... if I had vision problems, wow - what an improvement in quality of life. I really want to know how it knows my dog is a terrier, though.

This was a wonderful video as usual, Mat. My gran, who began to lose her sight in 1990, would have loved this technology; she absolutely devoured books-on-tape but would have so appreciated a more immersive experience.

If it weren't for my great aunt who was legally blind in her 70s, I probably wouldn't appreciate assistive technologies like this as I do now. She would put a finger over the edge of a glass, for example, when she filled it with water, so she'd know when it was full. Even I do this, when getting a drink at night, and I am fully sighted.

Thanks for another entertaining and informative video, Mat. I down loaded Seeing AI before I finished watching it. I found the app. to be really fun to use as a sighted person. It said I was 69. Not bad. I'll be 75 in December, so I love the app. I see a little of how autonomous vehicles work. The must use a similar algorithm to recognize things it sees. Also, it's really nice that Microsoft offers it free of charge so people who really need such an app. can afford to have one.

Thank you. I'll probably go for the rode. There is a reason it's expensive

Really interesting video. And good to normalise use of helper apps like this. BTW, I for one am pleased to see the return of the old intro lead in video, as I liked its old tech/new tech aesthetic. Ewen

Do you like that wireless mic? I like the rode one but seems to expensive

Did you switch back to the older, ripped from a VHD, intro animation? Did you do that before and I'm only noticing it now? If that's the case, ignore me. :)

Nice one Mat! Partially sighted fella here. Although I have a fairly mild form of my condition (X-linked Retinoschisis), I do still struggle with reading/recognising some things whilst out & about. As I'd grown up with the condition and it worsens gradually over time, I didn't always necessarily noticed that things were becoming more difficult to see, I was just naturally adapting to things, or subconsciously choosing to avoid them. It's only been over the last few years that I've started seeking out apps and modern tech that can help and it's always boss to learn of some new ones. I'm stuck on Android at the moment, but I keep hearing that iOS is better for accessibility and for assistive apps. I'm going to give the other one you mentioned a go. If you ever learn of an app that can read the numbers on buses from a distance, give us a shout! There's a YouTube channel called The Blind Life that covers a lot of new assistive tech. Worth a go. Thanks again for the video man! Assistive tech and 90's hip hop is a boss combo!

There's an app called Be My Eyes which sends video to volunteers who describe the scene for the user. My eyesight is surprisingly good so I signed up as a volunteer when the app first came out. Nobody ever rang me up so I deleted it after a couple of weeks, but do any of you have experience with it, from either side?

Enjoyed this very much! Sad it's not on Android (yet?), but my grandma in her 80s lives alone and has an ipad, so I will ask her if this will help her. Being visually impaired myself, I grew up thinking we'd get this far. Assistive tech was always bulky, specialized, and expensive when I was a kid in the 80s and 90s. I think you were totally respectful of the tech and disabled folk. Two things to remember for the naysayers you may run into - even fully sighted people sometimes lose their vision to the point where they would need something like this app to function (e.g. macular degeneration). The idea of a sighted person experiencing this isn't unheard of. Also, going back to my childhood, my fully blind peers and I were always trying to get the various speech synths to say swear words & other unkind things. Some people just assume disabled people are innocent as the pure driven snow. When they meet people like me, they learn real quick :D

Thanks for highlighting these things we don't usually see (excuse the pun) in our regular life. The blood sugar monitor tech was also interesting from a while ago. More!

I've been using the Seeing AI one for some time. Though fully sighted, I just like playing around with the thing and trying to test it out. Quite impressed it recognised a photo of Hilda Ogden the other day!

Wow. My father had a stroke a few years ago and his vision will never be the same. He can no longer read books or use a computer. He can sort-of see the TV and faces. I do think this app would be really helpful to him in a bunch of situations. I’ve got a tech wiz near him that can get this on his iPhone. Sadly, I live pretty far away. Thanks, Mat. You always make my day with a Techmoan video, but this is going to help my Dad and also make him laugh too. Also you have some great hip-hop cassettes!

The camera app on IOS with voice over will describe what is seen by the camera as well. It is incredible how well it discerns what is in the photo.

Phew, done with work and now I can look at this properly, read comments and description box, etc. Is this fascinating? Absolutely! I am continually amazed at AI technologies, and it is fantastic to see this being put to a beneficial use! I am probably daft and missed it (most probably daft) - or my brain is scrambled from work madness - but Android isn't supporting the exact same app that you demonstrated? I tried and failed with the Envision AI but I am hopeless sometimes. It's probably for the best, because I would obsessively use it for minutes on end, seeing what it can recognize, that I could be using on more productive ventures. Thank you very much for what you do, by the way. I don't think I sufficiently thanked you on the prior video - these always brighten my day. Have a good evening.

The Microsoft app added a year and a month to my age. Now I feel old(er).

You are right that there is a big crossover between this and Autonomous driving. I have read that the German version of my 2013 car can recognise road signs and adjust the Adaptive Cruise control accordingly.

Really interesting video. Do you know if it is totally reliant on the cloud to work or will it work offline. Thinking if a blind person could still use the app if in a room with no network or WiFi or if out and about abroad with no affordable roaming.

Really interesting !

"55 year old man. With beard looking neutral" Even AI has an opinion on your beard Mat!

I SEE what you did there in the thumbnail, BTW. 🤓

Way more than just fun. Pretty amazing, actually. I think you're spot on with the idea of peeking over the horizon of special applications to discover what's next.

Excellent, great video and very interesting. Couldn't agree more on being positive, and trying to put yourself in other people's shoes.

Great video. Nice to see some improvements in the world of AI vision recognition, still got some ways to go but things are improving. On a more personal note, being colour blind I use a couple of colour blind helper apps to help me work out the colours of things.

Very interesting, Mat. My other half (the musician) used to teach at New College for the blind and partially sighted in Worcester. Most of the kids had separate devices to read colours. As part of their education, they were taught that purple doesn't really go with yellow or green etc. and they used these devices to choose suitable matching clothes each day. Regarding the speed reading, they all used email and Social Media of course and had Windows, Android or iOS read text to them directly from the PC or device. Without exception, they all had the reading speed set to at least 300 WPM. Sheila couldn't make it out at that speed but they could. It goes to show the human brain is capable of taking information in far more quickly than the eyes can scan and interpret it.

Apps like this give me good hope as my already bad eyesight grows steadily worse with age.

A couple of times (7:42, 8:23) when you were showing the Microsoft webpages you were seeing my old friend Saqib Shaikh https://twitter.com/saqibs who is the lead on the SeeingAI project. Small world! Let me know if you want an "introduction", he's a very nice guy :)

Just to be difficult, I tried the app on a Clydesdale Bank £5 note, which it ignored. It had no problem with the Bank of England equivalent.

I'm 53 but was delighted that the app underestimated my age by two years. I'm sure the results would have been different if I'd had a beard.

I like your camera videos and I fully understand why someone would want to use something more professional than a smartphone. I agree with you that using a camera at chest height with an angled screen is a better way to film than going around holding a smartphone up in front of your eyes. Have you heard of an app called Microsoft Soundscape? A map delivered in 3D sound. It was recently promoted by Guide Dogs for the Blind for guiding people around streets, although I had to ask what the app was called. https://youtu.be/rHriKpTxff8 I’d love to see a full video on the Intel Reader, even if it's Patreon exclusive.

There is a case to be made for if the app is unable to recognise a coffee maker as a coffee maker, the designer of said coffee maker has failed in their job ;)

Interesting that the Seeing AI app's logo not only looks like an eye with the letter S in it, but also looks like an old-style light sensor.

I just explained the Microsoft App to my 100% blind friend Louise. She said an improvement to hit the buttons on the app would be a short press says the button and a long press executes the command. I don't have an Apple phone but use the app Andre uses on my huge tablet.

Once again the *professionally offended* get all the attention, and here's me getting paid sod all to find things to go off about and being completely overlooked as usual. It's just not on, etc. Seriously though, I find these videos fascinating as I'm neither visually impaired nor acquainted with anyone that is, so it's interesting to see how advances in technology can be put to use in that area _and_ remind me just how fortunate I am to not have to rely on anything like this myself. Mind you, my left eye is so bad that even as a kid the lens on my glasses was so thick it would weigh that side down and make my glasses sit wonky on my face, so I might end up eating those words yet... That Intel Reader looked like a bit of a pain in the arse mind, and it looks like you've already covered the main things it can (just about) do so I'd not be clamouring for a follow-up video on it!

Fascinating view into where the technology is. That old xkcd comic about needing 6 months and a team of PhDs to work out whether a photo contains a bird is no longer true. I'm having a hard time understanding how a non-sighted or partially sighted person would use the app, but then again I am amazed at how good the accessibility features on phones are.

Mat, I am visually impaired. I am NOT offended you had some fun like I do. I watch you nose to a 50 inch (1.25 meter) screen. Apps like this really help me. I have a huge android tablet and use a similar app. It also is my magnifier in low light. I can read and type Braille too. To any piss takers - try wearing a blindfold for a week.

I'd be interested to see how the currency recognition worked with coins and with a few notes/coins at once. will it add up what it sees?

Damn this is fun!

I have just tried the other app you mentioned for android. It is similar to the Microsoft but with monthly charges.

The idea that assistive technology can be useful far beyond its initial target audience is often called the curb cut effect - based on the idea that curb cuts were originally for wheelchair users, but are helpful to people pushing baby strollers, delivery people with dollys, cyclists, etc. Videos like this work to "normalize" assistive technology, and help people understand how to design products and services that are usable by all. The more the public sees this stuff - even in slightly non-serious presentations - the better. https://mosaicofminds.medium.com/the-curb-cut-effect-how-making-public-spaces-accessible-to-people-with-disabilities-helps-everyone-d69f24c58785 And as a non-disabled but *very* colour blind person, the colour reader thing looks so good I will probably download the app. Imagine trying to troubleshoot a wireless router that only communicates via a single red/green colour switching LED where the two colours look identical to you - there are lots of situations where knowing colours is super useful.

The audio seems to have a hiss to it, is that just me?

Another fascinating video. 👍

Thanks for showing, very impressive AI, waiting for the Android version !

I'm the kind of person that can get a little frustrated with smart phones because they seem to be the one defining tech of life. And all we seem to do with them is use them for social media, where everyone argues, tons of bullshit etc. But its videos like this that just remind me how wonderful smart phones can be. I can't imagine what it must be like to have a sight impairment and just be able to use your phone to recognize a 10 pound note from a 5, but its really pretty great and that to me is beautiful.

It's always a good video when you've found the keys to the dressing up cupboard.

I love the way the tech for the blind has improved, my stepfathers first wife who was a good friend of the family who was blind in the 80s just had a talking clock and a detector that would let her know if it was raining, the Microsoft app would have been perfect for her. OCR had come a long way, there is an old tomorrow's world episode on this I saw a while ago which was a huge machine (kinda like the scanner in a previous video) and the fact now it's a free app it just great. I have a blind friend on Facebook I know from Comic Cons, will ask him if he has this app and what his impression is. Great video and loved some of the bad detections... It gets there in the end though and as you said these thing only get better over time (and it's free)

Oooh old intro again. Watching live now... Probably should comment after lol