This paper describes an approach for an Augmented Reality (AR) system based on the processing of stereoscopic images and focusing on a medical application area. One of the major challenges of an AR-system is the determination of the relative orientation between the viewer and the scene object used for registration. In medicine, this may be used to overlay a phyicians view of a patient with computer-generated images. Since magnetic and ultrasonic tracking systems are limited in relation to range, accuracy, and reliability, an optical tracking mechanism has been chosen. Stereo images are used for visualization as well as for tracking. To reduce the computational cost, depth information is calculated only for specific landmark positions after separated segmentation of each of the stereo images. A prediction mechanism aids the segmentation and the evaluation of segmentation results as well as the labeling for registration.
