ISCA Archive SLaTE 2017
ISCA Archive SLaTE 2017

Was it something I said? Facial Expressions in Language Learning

John Sloan, Julie Carson-Berndsen

This paper describes an experiment to evaluate facial expressions of an animated avatar as a means of providing feedback to non-native English learners on language production. The aim of the study was to ascertain whether native and non-native English speakers interpret and respond differently to facial expressions and whether such expressions have a role to play as feedback in emerging language learning technologies. Native English speakers and non-native English learners took part in an experiment where facial expressions were presented as a response to their textual input sentences and were asked for their interpretation of the change in expression (or otherwise). Furthermore, it was investigated to what extent non-native learners subsequently altered their language behaviour in line with their perceived interpretation of the expression. The majority of non-native learners attributed a change in facial expression, where the avatar looked away, to errors in language production in the preceding sentence and they reduced the syntactic complexity of the following sentence accordingly. The results underpin the potential of facial expressions as a feedback mechanism for language learning and the insights will now be deployed in an effective and engaging personalised e-learning language platform.