ISCA Archive SpeechProsody 2024
ISCA Archive SpeechProsody 2024

The role of prosodic structure in the planning of coordinated speech and manual gestures

Jelena Krivokapic, Mark Tiede, Martha Tyrone, Ruaridh Purse, Jungyun Seo

It is known that prosodic structure and co-speech gestures are temporally related to each other, but the exact nature of this coordination and how it arises is not understood. In this study, we test the hypothesis that the coordination is mediated by prosodic structure, with differing gestural types planned at different stages in the process of utterance generation, resulting in variation in the strength of coordination between speech and co-speech gesturing. An electromagnetic articulometer (EMA) study was conducted, testing the effect of gesture type (beat gesture, deictic gesture) and utterance planning difficulty (easy, difficult) on the temporal properties of manual gestures and their coordination with co-produced speech gestures. The hypothesis predicts that beat and deictic gestures will be affected differentially by planning difficulty, reflecting how early in the speech planning process they are integrated with speech. Data from six speakers were collected. Results provide qualified support for the hypothesis, reflecting in part individual speaker differences in planning strategies, and will be discussed in the context of their implications for models of speech planning.