(Illustration by iStock/calvindexter)
While visiting family during the winter holidays in 2021, IT professional Dan Scarfe was upset that his 96-year-old grandfather was unable to fully engage in conversation because of his growing deafness. According to the World Health Organization (WHO), more than 1.5 billion people worldwide—nearly 20 percent of the global population—live with some form of hearing loss, and 430 million of those people experience disability from it. And while many deaf people can communicate through sign language or hear with devices such as hearing aids or cochlear implants, the social isolation like that experienced by Scarfe’s grandfather is common.
Scarfe wanted to help. “We’ve got real-time subtitles in [Microsoft] Teams and Zoom, and we were aware of those brand-new augmented-reality (AR) glasses,” he recalls, “and I thought, why can’t we just combine the two together and create live subtitles?”
Scarfe partnered with six friends and former colleagues with backgrounds in machine learning, marketing, and charity work to found XRAI Glass in July 2022. Its mission, he says, is to “subtitle the world.”
XRAI Glass is an app that enables AR glasses to connect to cloud-based transcription services from Amazon, Microsoft, and Deepgram to create real-time captions on the glasses’ screen.
“You just load the software on your phone, and then you can see it connect to the cloud services,” Scarfe explains. “It’s literally like projecting digital content into the real world right in front of you.”
The software has recording and playback features and can even identify a speaker within a group of people.
In just a year, the company has grown to 5,000 users worldwide. Education-management professional Carol Cover, whose hearing deteriorated rapidly after a failed cochlear implant, praises the technology for helping her to participate in conversation in busy restaurants and make confession in church privately, like everyone else. “It’s like a miracle to me,” she says.
The company is funded by several private investors and has raised what Scarfe calls “a small number of millions of dollars.” Charities such as the UK Royal National Institute for Deaf People and DeafKidz International also have given support.
One early concern for Scarfe and his cofounders is user privacy, and XRAI’s policy has been to abstain from collecting data from users’ devices. Instead, the user is the official data controller, legally responsible under local privacy laws for ensuring that their interlocutors consent to the transcription of their words.
The startup phase has also included a series of software upgrades, including the app’s translation services and its virtual assistant. The app can now process 76 different languages and 140 dialects and translate and transcribe them in real time. And the new virtual assistant runs on ChatGPT, which enables users to ask questions like “What will the weather be like today?” and receive answers displayed as subtitles inside their glasses’ screens. The assistant can also replay previous conversations and create a content summary.
The XRAI founders are developing reverse-translation-mode software, which will enable devices that use the app to speak as well as transcribe. This may be an important feature for some, such as those who are deaf from birth and unable to speak themselves—around 70 million people worldwide, according to the WHO.
With many deaf people preferring to use sign language to communicate, Scarfe is keen to develop a version of the software that could translate American Sign Language (ASL), projecting the spoken word into an AR image inside the glasses. He has been talking to a firm that could realize this idea in the coming months.
Scarfe says he regularly hears from users delighted to be able to listen to podcasts, chat over dinner, and go to confession without having to pass notes back and forth. “It’s incredible the feedback we’ve had from around the world,” he says, “for a teeny little company like ours.”
Read more stories by Emma Woollacott.