New glasses for the deaf and hard of hearing display subtitles while people speak

New technology in the form of glasses has been developed to provide real-time transcriptions that allow the deaf and hard of hearing to easily follow a spoken conversation.

For his entire life, Josh Feldman has relied on lip reading in order to understand what someone is saying – but this week, a new product that provides real-time transcriptions allowed him to follow a conversation without even seeing the other person’s face.

Having been born hard of hearing, Feldman has gotten used to lip reading while socializing. But it can be a struggle to keep up in a world that often doesn’t accommodate those who are deaf or hard of hearing, particularly during a pandemic in which face masks have been a necessity.

Enter the first pair of glasses to come with its own subtitles: Nreal smart glasses.

Feldman is one of the first to preview the glasses. Seated on a sofa, wearing what looked like a pair of thick black sunglasses, he carried on a conversation with a man seated to his right.

"I guess that you're more often than not asking people, what did someone say, you know?" the other man asked.

"Hundred percent," Feldman replied, without turning to look at him. "It's about the changes that come as part of having hearing loss."

How was this possible? As the other man spoke, his words appeared in white text on the inside of Feldman’s glasses, allowing him to read and respond in real time.

"The people who are hard of hearing would understand the feeling that I had when I had that conversation, which lasted about five minutes," Feldman said. "And I didn't look at the person to my right once, and solely looked ahead – it's a life-changing moment."

A British startup called XRAI Glass is behind the technology that could allow millions of people who are deaf or hard of hearing to see in-person conversations they cannot hear.

The glasses use augmented reality (AR) technology, and a phone app provides the live transcription. Currently, the app is only supported on Android, but the company is seeking to get approval for iPhone usage as well.

"To have a conversation while you’re not looking at the person you're speaking to," Feldman said. "Wow. Wow."

He can also read conversations he’s having over the phone if the phone is on speaker so the glasses can pick up the audio, and according to the company’s website, it can translate any language.

The concept is especially useful during the COVID-19 pandemic: those who wear face masks cover up their mouths, making lip reading impossible.

"To have something like those glasses, where it takes away the barrier of the mask, that they can actually see what's being said … that is really effective and that would really make a difference to someone," said Teri Devine, Associate Director of Inclusion at the Royal National Institute for Deaf People.

The technology isn’t perfect – it’s less reliable in a group setting when people talk over each other.

"I think the real breakthrough will be when we get smart contact lenses," Dan Scarfe, founder and CEO of XRAI Glass, said. "That will be the thing that actually revolutionizes the space. Their very first prototypes are just coming out and they will go mainstream within the next two to three years."

The glasses will be publicly available starting September.