Tracking with reference images: A real-time and markerless tracking solution for out-door augmented reality applications

This paper presents the optical tracking solution developed in theArcheoGuide project (The Augmented Reality based Cultural HeritageOn-site GUIDE, IST-1999-11306). The system enables torecover precisely the user head position and orientation at predetermined viewing location without the support of markers.The tracking approach is novel and bases on the real-time registration of the live video-image with so called "reference images" of the site. Once the matching between a live- and a reference-images could have been established, the virtual information can be presented correctly to the user, either using a 2D image warping transformation or after deduction of the current 3D position/orientation of the camera. The image matching algorithm represents the core of the tracking system. It must have real-time performance and be particulary robust to intensity and local changes. We opted for an analysis in frequency space and exploit the invariance properties of the Mellin-Fourier transform. The tracking system has been tested outdoor in the context of theArcheoguide project. It runs on a laptop at around 10 to 15 Hz and provides views of virtual monuments in a Head Mounted Display superposed to the ruins of the archaeological site.