Envision smart glasses help blind users read documents
Envision, a leading developer of smart glasses that can help blind people read and identify objects, has now incorporated a game-changing voice-activated sight assistant into its glasses powered by the most advanced artificial intelligence ever built.
In November 2022, OpenAI, the company behind ChatGPT unveiled its disruptive generative AI natural language processing tool that is capable of maintaining human-like conversations with a chatbot.
With a recent update released in March, the technology has been heralded by industry commentators as a pivotal leap forward on the journey towards seeing more advanced types of AI proliferating consumer technology and is already being integrated into products by tech giant Microsoft.
Netherlands-based Envision, who uses Google Glass eyewear on the hardware side, has, since launching in 2020, been deploying the glasses camera to capture text in documents or on packaging to produce an audio text-to-speech read-out for the wearer amongst other features.
However, this latest plugin to ChatGPT 4.0 using a dedicated API announced earlier this month, takes assistive technology for the visually impaired to previously unseen levels by enabling facets of document interpretation similar to what can be achieved by fully sighted people.
A true game changer
Previous to this brand-new feature known as Ask Envision, users wishing to extract key textual information such as the balance due on a utility bill or the cooking instructions on the back of food packaging, had to use their glasses to scan and read out all the available text. This meant wasting time sifting through extraneous information until the key details were announced.
Now, with the Ask Envision feature, users can simply capture the text through the glasses camera and then use their voice to ask ChatGPT direct questions about the document.
This may range from questions like “What is the balance of this bill” to “What are the vegetarian options on this menu?”
Within seconds, Ask Envision will provide detailed and precise answers to the wearer through speech output.
As ChatGPT 4 has been trained on unimaginably vast data sets based on both trawling the web and human interactions, the software also possesses a high level of contextual awareness — meaning that it can accomplish tasks as complex as summarising long documents or suggesting a route to the user based on looking at a public transit map.
Other important tasks that the ground-breaking virtual assistant is capable of mastering include pinpointing essential data from graphs and charts at work, providing a near-instant translation of foreign language documents and packaging, interpreting assembly instructions and guiding the user to a particular item in a travel itinerary or schedule.
It even processes and understands natural language to the point that it can, if requested, instantaneously compose a rap in the style of Eminem, or whatever artist is chosen, using the wording of a restaurant menu or corporate flyer!
Ultimately, however, the true beauty of Ask Envision lies in the sheer simplicity of its user interface.
Despite deploying the most mind-bendingly complex technology conceivable under the hood, the user interface is as straightforward as speaking aloud an everyday question or maintaining a basic conversation.
Indeed, even though the current iteration of the Envision glasses allows users to make remote calls for sighted assistance using the device’s camera feed, Envision CEO Karthik Mahadevan wants the integration with ChatGPT to render the necessity of remote sighted guidance to become more and more obsolete over time:
“The ultimate goal is for the glasses to be as easy to interact with as they would be if there was a sighted person to whom you could ask questions sitting next to you,” says Mahadevan.
“The technology should be able to give just as accurate an answer as that human could whilst making the user feel a lot more independent because they are not having to rely on anybody else.”
Earlier this month, Envision ran a webinar to explain more about Ask Envision to its users. The video feature contributions from a couple of the technology’s beta testers who waxed lyrical about its game-changing potential:
“This is awesome. My mind is totally blown by this,” said Chela from California.
She continued, “It’s going to get better and better and I can see this opening many opportunities for receiving information in various ways. I’m excited to see more of what is coming down the pipeline.”
Wesley from Calgary who works in call center customer support and technical support said, “The Envision glasses have impressed upon me a greater sense of independence, mobility and more of a positive can-do attitude. I’m not really stuck asking for help from others as much and I only look forward to seeing greater advancements in the future.”
New horizons
Speaking of the future, the next stage for Ask Envision, some aspects of which are currently in beta testing, will see a greater focus on being able to leverage the power of ChatGPT to recognize images, objects and live scenes – rather than just written information.
Here, the opportunities are endless from recognizing facial expressions and identifying different objects right through to navigational directions.
In the medium term, it is the consolidation of multiple assistive technology solutions in one place thanks to AI that Mahadevan finds most enticing.
“ChatGPT is helping us leapfrog a whole other bunch of technologies because suddenly you don’t need a separate translation app, you don’t need another app to help read a menu.”
He continues, “We see this as the one feature that will overtake a lot of assistive technologies. Users will have at their disposal this ultra-intelligent AI assistant who will understand everything under the sun for them.”
Aligning this vision with the pace at which artificial intelligence is currently evolving – this will very likely include efficiencies and interactions that could only be dreamt of by individuals living with severe sight impairment just a short few years ago.
