ISCA Archive AVSP 1998
ISCA Archive AVSP 1998

Integration of Audiovisually Compatible and Incompatible Consonants In Identification Experiments

Louis D. Braida, Kaoru Sekiyama, Ann K. Dix

A mathematical model that has successfully accounted for audiovisual integration of natural consonants with compatible audio and visual components is applied to artificial stimuli with incompatible components. Twenty eight Japanese subjects identified stimuli derived from the eight consonants /b,d,g,p,t,k,m,n/ under auditory, visual, and audiovisual conditions. The acoustic stimulus was degraded by lowpass filtering and additive noise. A single set of model parameters, derived from confusion matrices obtained in the unimodal experiments, was found to make fairly good predictions of the results obtained in the audiovisual experiment for both the compatible and incompatible stimuli. Limitations of the approach and implications of the results are discussed.