Be My Eyes announced the new ChatGPT-powered Virtual Volunteer functionality.
When I sat down with Microsoft’s Jenny Lay-Flurrie earlier this month, a hot topic of our conversation was the potential of AI-enhanced chatbots as an assistive technology. She shared a personal anecdote of her teenage daughter, who has autism and ADHD, using the new ChatGPT-powered Bing search engine to help her in recently writing an essay on why her school should offer sign language classes. These chatbots, Flurrie said, have the ability to alleviate points of friction associated with motor and cognitive load—especially for a neurodivergent person, where classic methods of research could prove unwieldy and overwhelming.
“[AI chatbots] collate so much information for you very, very quickly. It can save a lot of time,” Flurrie told me of chatbots’ potential to transform accessibility for disabled people. “If you think about someone from a mobility perspective, you can get the right level of information at your fingertips with a couple of clicks as opposed to having to conduct 10 to 20 different searches and go to multiple websites; it can be right there for you. It’s going to be very impactful for particularly neurodiversity… I think about dyslexia [and] dyspraxia. There’s a learning process to it. We’re definitely learning as we go [and learning] how to get the best out of the tools. I think there are some pretty profound implications.”
As you’d expect, Flurrie isn’t alone in her bullishness. Be My Eyes is excited too.
In a blog post published last week, the company was “thrilled” to introduce a new feature called Virtual Volunteer. For the uninitiated, Be My Eyes is a service (available on iOS and Android) that connects Blind and low vision people with sighted volunteers, the latter of which assists in deciphering physical labels such as the expiration date on a carton of milk. The app won an Apple Design Award at WWDC 2021 in the Social Impact category. Be My Eyes boasts more than 6.3 million volunteers spanning 150 countries and 180 languages across the globe.
True to the Be My Eyes ethos, Virtual Volunteer is highly conceptually similar to its human-based volunteer; the difference lies in the breadth and depth of information the artificial intelligence can surface in real time versus a human, such as recipes and more. Virtual Volunteer works via a so-called “dynamic” image-to-text generator, where a user can feed the software a picture and instantly get results on it. In the announcement, Be My Eyes stated it believes the tool will be “transformative” in helping people “powerful new resources to better navigate physical environments, address everyday needs, and gain more independence.”
In an interview conducted late last week over videoconference, Be My Eyes chairman and chief executive Mike Buckley explained the impetus for the feature came in two parts. First, the company ran a survey in which they sought to learn why a user on their platform would rather not make a phone call. The top answer, Buckley told me, was deferential: people don’t want to take a volunteer away from someone who may “need it more,” he said. Other reasons included not being comfortable conversing with a stranger by telephone and, playing into Virtual Volunteer’s raison d’être, a phone call would inhibit feelings of autonomy and independence. The second part of Virtual Volunteer’s origin story, according to Buckley, stemmed from conversations with OpenAI on its ChatGPT-4 technology. Talks between the two companies began only recently, in January. The discussions were general in nature at first, akin to the sentiments Flurrie shared with me about Microsoft’s view on AI chatbots. Then in February, Buckley said OpenAI came to Be My Eyes with an I-have-a-secret-to-tell-you about the then-unannounced new version of ChatGPT. There was no non-disclosure agreement involved; it was OpenAI trusting Be My Eyes with a secret about a profound new technology that could have profound possibilities in an accessibility context.
“They [OpenAI] showed us a demo and they said, ‘Would you want to work on this together and put together a group of beta testers for the product?’ Buckley said of how Be My Eyes partnered with OpenAI. “And we said, ‘This sounds really exciting, absolutely. Talk to us about safety. Talk to us about the efficacy of the product: Can you ensure us we can include members of our community involved in it… developing [it] and [giving] feedback and improving this?’ They said yes.”
He continued: “We believe they’ve [OpenAI] been really thoughtful about these things and approach things with a safety first and society-first model, and that made it very easy to work with them. Literally, this was this was about a five-and-a- half week sprint on this product [Virtual Volunteer]. From the starting point, I think we started working first week of February. Honestly, they’ve been fantastic to work with. It really happened.”
Like many apps, Be My Eyes decided to launch Virtual Volunteer first on iOS. The company’s chief technology officer, Jesper Henriksen, told me the choice was very much intentional. In an interview concurrent to Buckley’s, Henriksen said a big reason for this is the powerful chips inside iPhones, as is the iPhone’s ubiquity. Moreover, Be My Eyes is a small organization, and as such, it’s imperative for the team to focus its resources out of the gate. Broadly, Henriksen said Apple is to be commended for their steadfast commitment “from a software perspective” in making iOS and macOS accessible as can be. Android is getting better, he added, but said rather unequivocally Apple has historically “been ahead of the rest of the industry for quite a while” in terms of best-of-breed assistive technologies.
Buckley thinks the advent of Virtual Volunteer is a seminal moment.
“I don’t think we I’ve seen this level of excitement from from our community,” he said. “Almost at any time in the past, maybe perhaps when iPhone launched. I think the community was very, very excited that this [new kind of accessibility feature] feels like it, or at least rivals that. It’s excitement, it’s optimism, it’s hope.”
In terms of feedback, Virtual Assistant has been a hit thus far in these earliest days. Buckley said the company saw “over 4000” Blind and low vision people sign up for the service in the first 48 hours since Virtual Volunteer was announced. Henriksen shared a story about a family who used Virtual Volunteer to help them locate their pet canary in the house so they could get it back into its cage. An exceptional example of the feature’s usefulness, to be sure, but nonetheless serves as a good predictor of “where this [technology] will go,” Henriksen said.
As for the future of tech’s future, Buckley said it’s about the community.
“We want people who are Blind and low vision involved in every stage of our technical development and building the product,” he said. “The way we talk internally, and we say it just about every day, we want to empower Blind and low people to bend this technology to meet their needs. If we do that, if we empower our community, we’re going to figure out the long-term business models.”
In terms of an actual product roadmap, Buckley told me there is a lot the Be My Eyes team is thinking about and that the possibilities are tantalizing. They believe Virtual Volunteer can make things like online shopping and traveling more accessible and inclusive. In addition, he teased the announcement of the initial set of corporate beta testers in the coming weeks. All told, Buckley was quick to curb his enthusiasm for AI chatbots as a category by readily admitting he’s “very wary” of the incessant hype cycle and deluge of dystopian think pieces. Still, he’s a firm believer in ChatGPT’s (through Virtual Volunteer) ability to change the game in a positive way for people with disabilities. “We believe this could be transformative for accessibility on a global level,” he said of Virtual Volunteer’s potential impact.
Such a sentiment perfectly aligns with Be My Eyes’ North Star.
“[Our] number one thing is making the world more accessible for a quarter billion people and empowering them,” Buckley said.
