Monocular camera trajectory optimization using LiDAR data

A well known problem in computer vision and photogrammetry is the precise online mapping of the surrounding scenery. Due to the nature of single projective sensor configurations with inherent 7-DoF, error accumulation and scale drift is still a problem for vision based systems. This is especially relevant for difficult motion trajectories. However, it is desirable to use cheap small form factor systems, e.g., small UAVs with a single camera setup. We propose a simple and efficient appearance based method for using LiDAR data in a monocular vision mapping system by using pose graph optimization. Provided laser scans are available, our system allows for a robust metric mapping and localization with single electro-optical sensors. We use large sets of synthetically generated 2-D LiDAR intensity views in order to globally register camera images. We especially provide insights for generating the synthetic intensity images and extracting features from such data. This enables the global appearance based 2-D/3-D registration of 2-D camera images to a metric 3-D point cloud data. As a result we are able to correct camera trajectories and estimate geo-referenced, metric structure from monocular camera images. Possible applications are numerous and include autonomous navigation, real-time map updating/extension or vision based indoor mapping.