ISCA Archive ICSLP 1992
ISCA Archive ICSLP 1992

Investigations into the auditory F0 speechreading enhancement effect using a sinusoidal replica of the F0 contour

Arthur Boothroyd, Robin S. Waldstein, Eddy Yeung

Constant-amplitude sinusoidal replicas of the voice fundamental frequency (Fo) contours of video-recorded sentences were prepared off-line by a combination of automatic and manual Fo estimation. The replicas were then re-synchronized with the video portions of the original recording. In 12 normally hearing adults, the replicas were found to be as effective a supplement to speechreading as the low-pass filtered output of an electroglottograph. Both signals increased the number of words recognized in sentences of known topic by almost 40 percentage points. Using a constant-frequency version of the replicas, it was found that about 1/3 of the Fo speechreading enhancement effect could be accounted for by voicing detection alone.