ISCA Archive SpeechProsody 2024
ISCA Archive SpeechProsody 2024

The Role of Auditory and Visual Modality in Perception of English Statements and Echoic Question by Chinese EFL Learners

Shanpeng Li, Yinuo Wang, Shifeng Xia, Zhiqiang Tang, Ping Tang, Yan Feng

Previous research underscored the role of auditory and visual cues in perceiving statements and questions, yet with conflicting conclusions regarding their relative significance. It was argued that English speakers relying predominantly on auditory cues, with limited impact from visual cues for intonation. Given the variability observed in language-specific utilization of auditory and visual modalities for interpreting statements and questions, as evidenced in studies involving Dutch and Catalan, the generalizability of findings to other languages remains uncertain. Therefore, the study aimed to investigate the influence of auditory and visual cues on the perception of English statements and questions among Chinese EFL learners. A total of 56 Chinese EFL learners participated in the audiovisual perception study, categorized into three blocks: audio-only (AO), visual-only (VO), and audiovisual (AV) conditions. Additionally, to explore the contribution of specific facial areas, the VO and AV conditions were subdivided into full-face, upper-face only, and lower-face only conditions. The result revealed that Chinese EFL learners lean towards visual cues, particularly upper face when perceiving English intonations. This inclination could be attributed to factors such as whispered condition and cultural inclinations. Recognizing and incorporating these influences into teaching approaches can significantly enhance the comprehension of intonation among EFL learners.