Real-time head tracking in a multimodal interface with changes in geometry and illumination

This paper describes a software solution for a real-time head tracker. The user's head/eyes can be detected and localized very precisely in video images supplied by a single video camera on a general purpose PC without any special signal-processing hardware. The critical points in the tracking process are to cope with changes in size, orientation and illumination. In order to robustly track the user's head/eyes, a luminance-adapted block matching technique using multiple eye patterns has been developed. This technique is both insensitive to size, orientation and illumination variations as well as computationally efficient. The head tracker is integrated into an interactive multimodal computer interface. The current 3D position of the user's head/eyes is calculated for use in multimedia applications.