ISCA Archive SpeechProsody 2006
ISCA Archive SpeechProsody 2006

An fMRI study of multimodal deixis: preliminary results on prosodic, syntactic, manual and ocular pointing

Francesca Carota, Hélène Loevenbruck, Coriandre Vilain, Monica Baciu, Christian Abry, Laurent Lamalle, Cédric Pichat, Christoph Segebarth

Deixis or pointing plays a crucial role in language acquisition and speech communication. In this paper we present an innovative fMRI approach in order to examine deixis. conceived as a unitary communicative strategy which employs different verbal and non-verbal speech devices to achieve the pragmatic goal of bringing relevant information to the interlocutors' attention. We designed a unified fMRI paradigm for multimodal deixis, integrating four conditions of verbal and non-verbal pointing: 1) prosodic focus, 2) syntactic extraction, 3) index finger pointing, 4) eye pointing. Sixteen subjects were examined while they gave oral, manual and ocular responses inside the 3T magnet imager. Preliminary results based on a random effect analysis with a group of 10 subjects show that all pointing conditions recruit a left temporo-parieto-frontal network, with respect to the control condition. Further analyses are being carried out to distinguish between different modalities of pointing.