Did you see the interview Mark Zuckerberg of facebook did with one of his accessibility engineers a couple of weeks ago where they talked about “Auto alt text”?
Matt King, an accessibility engineer with Facebook who also happens to be blind (and an accomplished athlete and musician), has been working on a project that takes “Alt text” for blind people to a whole new level. Using machine learning and artificial intelligence, his project can automatically describe what’s in a photo. Earlier, if someone was using a screen reader, the alt text part would just say “photo” and move on to the next item. However, with automatic alt text, Facebook’s new technology that has been trained using millions of examples, it is now easy for it understand what’s in a picture and describe the contents of it fairly accurately.
“Image may contain: Tree, sky, outdoor”
“Image may contain: Pizza, food”
“Image may contain: Two people, smiling”
In the interview shown below, Matt explains what went into the artificial intelligence behind this technology, and also goes on to say that this is just the beginning. Automatic alt text can be applied in several other ways in the future. Imagine it on a blind person’s phone. When they start the phone camera to take the photo, automatic alt text recognizes what the lens is capturing and describes the surroundings to that person.
Matt also shows an upcoming project where touching the photo describes what’s right under the user’s finger. (“chair 1”; “chair 6”; “umbrella 1”)
Automatic alt text is already available on the iOS app and will be rolled out to the Android app soon.
Accessibility improvements have always made life easier in increments. Every decade or so, there is a game changing improvement, and automatic alt text may be that for this decade.
Read more here.