The computer vision community and industry have made great progress on face detection and recognition. The current popular approaches are now mostly based on single image analysis using sliding detection windows and boosted classifiers. They work impressively well for frontal face images. However, there are many more adjuvant application areas for facial analysis if we are able to accurately locate and describe faces in real-time through video sequences. This allows analysis of what a face is "doing", rather than just where it is and who it belongs to. This is a more appealing and difficult problem and the subject of this talk.