ISCA Archive issp 2024
ISCA Archive issp 2024

An experimental setup for capturing multimodal accommodation using dual electromagnetic articulography, audio, and video

Lena Pagel, Simon Roessig, Doris Mücke

When engaging in a conversation, interlocutors frequently accommodate to each other in their speech patterns and co- speech movements. However, only a small number of studies have investigated both domains in a multimodal approach. An additional challenge for studies is accounting for information structure, which not only influences the production of speech and co-speech motion in a speaker but also affects the patterns of accommodation between speakers. Due to the high complexity of the required experimental design, it has not yet been comprehensively studied whether speakers accommodate to each other in their strategies of encoding information structure. This paper present a methodological approach for capturing multimodal focus marking patterns in dyads, which allows to address this research question. We introduce DiCE, a cooperative game to elicit lexically and prosodically controlled data in German, and present details of the experimental setup involving dual EMA, audio, and video.