Site icon Assistive Technology Blog

Seeing AI: Microsoft Project For Blind People That Let’s Them “See”

Venkat Rao
Saqib Shaikh wearing glasses and using Seeing AI to get a description of someone around him
How do blind people know what’s going on around them? Are people interested in their conversations? Are they giving them full attention? What sort of activities are people taking part in around them while they are, let’s say, sitting on a park bench?
 
Wouldn’t it be nice if there could be a device that could describe the people, their emotions, and behavior around them?
 
Saqib Shaikh is a Microsoft developer who lost his vision when he was 7. Being a developer and
knowing what technology, especially newer Microsoft technology, can do, is working on a project called Seeing AI that can provide descriptions of surrounding objects and people to a blind person. The project uses Microsoft Cognitive Services, a set of APIs (tools for building software applications) that can allow devices and apps to see, hear, speak, understand emotions and needs via various devices, apps and platforms. 
 
Seeing AI providing age, gender and emotions of two peopleSeeing AI runs as an app on the phone or on Pivothead smart glasses (with an inbuilt camera). The glasses can take photos of people around a blind person and describing exactly what they are doing (“a young man performing a skateboarding trick”, ” a young girl throwing an orange frisbee”), can tell the age, gender and emotions of people around them (“I see two faces – one is a 40 year old male looking surprised and the other is a 20 year old woman looking happy”), and can also read text from objects. In the video below, Saqib is seen using the phone app to read a menu at a restaurant. Not only does the app guide him on how to angle the camera so the entire menu is in the frame, but also recognizes all the text and starts reading the menu from top to bottom, starting with the appetizers.
 
Saqib is seen saying that artificial intelligence is improving at a very fast rate, and what we see in Seeing AI is just the beginning. It’s exciting to see what’s the future has in store for us!
 
Watch the video to see how Seeing AI works and what all it can do.
 
[Thank you for sharing, Prady!]

Source: Arstechnica, Microsoft Cognitive Services Website

Exit mobile version