Researchers have developed ‘SoundWatch’, a smartwatch app for people who are deaf and hard-of-hearing who want to be aware of nearby sounds.
The app picks up and identifies a sound the user is interested in – examples include a siren, a microwave beeping, or a bird chirping – sends a buzz on the watch along with information about the sound, developed by a team at the University of Washington.
“This technology provides people with a way to experience sounds that require an action – such as getting food from the microwave when it beeps. But these devices can also enhance people’s experiences and help them feel more connected to the world,” said Dhruv Jain, a doctoral student in the School of Computer Science & Engineering.
“I use the watch prototype to notice birds chirping, and waterfall sounds when I am hiking. It makes me feel present in nature. My hope is that other d/Deaf and hard-of-hearing people who are interested in sounds will also find SoundWatch helpful.”
The team started this project by designing a system for people who are deaf and hard-of-hearing who wanted to know what was going on around their homes.
“I used to sleep through the fire alarm,” said Jain, who was born hard of hearing.
“People mentioned being able to train their pets when they noticed dog barking sounds from another room or realizing they didn’t have to wait by the door when they were expecting someone to come over,” Jain said. “HomeSound enabled all these new types of interactions people could have in their homes. But many people wanted information throughout the day, when they were out in their cars or going for walks.”
The researchers then pivoted to a smartwatch system, which allows users to get sound alerts wherever they are, even in places they might not have their phones, such as at the gym.
The team has the watch send the sound to a device with more processing power — the user’s phone — for classification to speed up the system. Having a phone classify sounds and send the results back to the watch saves time and maintains the user’s privacy because sounds are only transferred between the user’s own devices.
The researchers tested the SoundWatch app in March 2020 — before Washington’s stay-at-home order — with eight participants Deaf and hard-of-hearing in the Seattle area. Users tested the app at three different locations on or around the UW campus: in a grad student office, in a building lounge, and at a bus stop.
The SoundWatch app is available for free as an Android download.
