ISCA Archive Eurospeech 1997
ISCA Archive Eurospeech 1997

Continuous visual speech recognition using geometric lip-shape models and neural networks

Alexandrina Rogozan, Paul Deleglise

This paper describes a new approach for automatic speechreading. First, we use efficient, but effective representation of visible speech: a geometric lip-shape model. Then we present an automatic objective method to merge phonemes that appear visually similar into visemes for our speaker. In order to determine visemes, we trained SOM using the Kohonen algorithm on each phoneme extracted from our visual database. We go into the presentation of our visual speech recognition systems based on heuristics and neural networks (TDNN or JNN) trained to discriminate visual information. On a continuous spelling task, visual-alone recognition performance of about 37 % was achieved using the TDNN and about 33 % using the JNN one.


doi: 10.21437/Eurospeech.1997-530

Cite as: Rogozan, A., Deleglise, P. (1997) Continuous visual speech recognition using geometric lip-shape models and neural networks. Proc. 5th European Conference on Speech Communication and Technology (Eurospeech 1997), 1999-2002, doi: 10.21437/Eurospeech.1997-530

@inproceedings{rogozan97_eurospeech,
  author={Alexandrina Rogozan and Paul Deleglise},
  title={{Continuous visual speech recognition using geometric lip-shape models and neural networks}},
  year=1997,
  booktitle={Proc. 5th European Conference on Speech Communication and Technology (Eurospeech 1997)},
  pages={1999--2002},
  doi={10.21437/Eurospeech.1997-530},
  issn={1018-4074}
}