A smartwatch app alerts users with hearing loss to nearby sounds

A new smartwatch app alerts users who are deaf or hard of hearing of nearby sounds, such as microwave beeps or car horns.

“The main motivation [for the app] came from my own experience, and conversations that my colleagues and I have had with deaf and hard of hearing people over several years,” says Dhruv Jain, who presented the system, called SoundWatch, at the virtual ASSETS conference on October 28.

Jain, who is hard of hearing, uses SoundWatch at home to avoid sleeping through a smoke alarm. “On a nature walk, it’ll tell me that there’s birds chirping, or there might be a waterfall nearby,” he says. “Those sounds make me feel more present and connected to the world.”

Sound awareness apps for smartphones exist. But Jain prefers the immediacy of sound notifications on his wrist, rather than in his pocket — and surveys of people who are deaf or hard of hearing show he’s not alone. 

The SoundWatch app pairs an Android smartwatch and phone. The watch records ambient noises and sends that data to the phone for processing. When the phone detects a sound of interest, the smartwatch vibrates and displays a notification.

Jain, a computer scientist at the University of Washington in Seattle, and colleagues designed the app to identify 20 noises. In experiments, SoundWatch correctly identified those 20 sounds 81.2 percent of the time. When set to listen only for urgent noises — a fire alarm, door knock or alarm clock — the app was 97.6 percent accurate. Eight deaf and hard of hearing people who used SoundWatch around a university campus gave the app broadly favorable reviews, but noted that the app misclassified some sounds in noisy outdoor settings.

Jain and colleagues are now working on a version of SoundWatch that users can train to recognize new sounds, such as their own house alarm, using just a few recordings.