Image analysis for 3D modeling, rendering, and virtual view generation

The generation of virtual environments for telepresence systems, interactive viewing, and immersion in remote 3D-scenarios is an expanding and very promising research field. The challenge of this technology is to design systems capable of synthesizing views from any desired perspective using a set of real-scene perspectives. In such a system, two main parts can be distinguished: multiview image-analysis and viewpoint synthesis. Among other methods, multiview image-analysis is comprised of methods for the solution of three commonly encountered difficult tasks in computer vision: estimation of pixel correspondence, object segmentation, and camera calibration. Moreover, the results of multiview image-analysis are the required foundations for image synthesis. In this paper, we present a robust framework for stereo-image analysis, 3D modeling, and viewpoint synthesis. The image analysis part comprises disparity estimation by an improved block-matching algorithm and depth-based object segmentation using morphology. The basic idea of the system is to solve each of these tasks in direct interaction with the others. The results obtained in the partial solution of the correspondence problem are used to substantially simplify the segmentation task and vice versa. In the image-synthesis part, two tasks are considered: direct generation of arbitrary intermediate views, and extraction of the 3D structure of the scene using disparity results and camera parameters. The system has been successfully demonstrated in the context of virtual-view generation and object manipulation through 3D modeling.