ISCA Archive Interspeech 2024
ISCA Archive Interspeech 2024

Exploring the Complementary Nature of Speech and Eye Movements for Profiling Neurological Disorders

Yuzhe Wang, Anna Favaro, Thomas Thebaud, Jesus Villalba, Najim Dehak, Laureano Moro-Velazquez

Subtle changes in both speech and ocular movements often characterize neurodegenerative diseases (NDs). This study aims to profile various NDs using speech and ocular movements collected during the Cookie Thief Picture description task. We designed new multi-modal features centered around the Areas of Interest (AoIs) to quantify attention focus and search patterns. Through statistical analysis, we examined which features differentiated controls (CTLs) from subjects with various NDs, including Alzheimer’s disease (AD), Parkinson’s disease (PD), and Parkinson’s disease mimics (PDM). Results showed that subjects with AD and PDM had reduced AoI visits, indicating diminished search behavior. In contrast, subjects with PD displayed increased variability in visual search patterns, possibly reflecting disruptions in integrating visual information with verbal expression and loss of attention during the trial. These features effectively differentiated ND groups from CTLs and among themselves.