Speech-In-Noise Comprehension is Improved When Viewing a Deep-Neural-Network-Generated Talking Face.
Tong Shan, Casper E Wenner, Chenliang Xu and 2 others
PMID 36384325WHAT IT FOUND
A synthetic talking face improved the percent of keywords reported correctly versus audio-only by 20.2%, 21.9%, and 16.3% in worse noise with ten normal-hearing adults.
It was not as good as a real face.
Key findings
01At −9, −6, and −3 dB SNR, the synthesized face improved the percent of keywords reported correctly versus audio-only by 20.2%, 21.9%, and 16.3%.
02Natural face performance was better than synthesized face performance overall, but the two did not differ significantly at −3 and 0 dB SNR.
03Every subject showed some benefit from the synthesized face compared with audio-only.
STILL TO COME
How it was doneWhat they foundWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
Only ten subjects were tested, and all had normal hearing thresholds of 20 dB HL or better from 500 Hz to 8000 Hz, so it does not show benefit for people with hearing loss. The participants were English speakers tested with English sentences, so the result may not apply to other languages. The synthesized face was not as good as the natural face overall, so it does not replace viewing a real talker. The video showed only the face, not head movement, blinking, gestures, or emotional expression, which affect communication.
Declared interests
The authors declared no potential conflicts of interest. Funding was from a University of Rochester Augmented / Virtual Reality pilot grant and the National Institute on Deafness and Other Communication Disorders grant R00DC014288.
The easy way to misread this
Do not conclude that a synthetic talking face will help people with hearing loss understand speech. The study tested only ten adults with normal hearing, and the authors say benefit for people with hearing loss is not known.