In this project, we have developed a sign language tutor that lets users learn isolated signs by watching recorded videos and by trying the same signs. The system records the user's video and analyses it. If the sign is recognized, both verbal and animated feedback is given to the user. The system is able to recognize complex signs that involve both hand gestures and head movements and expressions. Our performance tests yield a 99% recognition rate on signs involving only manual gestures and 85% recognition rate on signs that involve both manual and non manual components, such as head movement and facial expressions.

Index Terms: Gesture recognition, sign language recognition, head movement analysis, human body animation

@inproceedings{aran06_einterface,
  title     = {{Sign language tutoring tool}},
  author    = {Oya Aran and Ismail Ari and Alexandre Benoit and Ana Huerta Carrillo and François-Xavier Fanard and Pavel Campr and Lale Akarun and Alice Caplier and Michele Rombaut and Bülent Sankur},
  year      = {2006},
  booktitle = {{Summer Workshop on Multimodal Interfaces (eINTERFACE 2006)}},
  pages     = {23--33},
}