ISCA Archive SLaTE 2013
ISCA Archive SLaTE 2013

Vizart3d – real-time system of visual articulatory feedback

Thomas Hueber, Gérard Bailly, Pierre Badin, Frédéric Elisei

We present recent developments on our system of visual articulatory feedback, named Vizartd3D, which is based on the 3D orofacial clone developed at GIPSA-lab. This system aims at providing any speaker with a feedback on his/her own articulation, especially by displaying the tongue movements. Application areas are computer-assisted pronunciation training (phonetic correction) for second-language learning and speech rehabilitation. In our system, the orofacial clone is animated in real-time from the speech audio signal, using a statistical mapping method based on multiple Gaussian mixture regressions (GMR). This approach combines in the same framework a spectral mapping step with an acousticarticulatory inversion step. All the modules of the system are implemented in the real-time environment Max/MSP (acoustic analysis, mapping and 3D rendering). More information is available at http://www.gipsa-lab.fr/projet/vizart3D/.