Layered representation of scenes based on multiview image- analysis

This paper describes a novel cooperative procedure for the segmentation of multiview image sequences exploiting multiple sources of information. Compared to other approaches, no a priori information is needed about the structure and the arrangement of objects in the scene. Three cameras in a particular unsymmetrical set-up are used in the system. The color distribution and the object contours in the constituent 2D images, the disparity information in stereo image pairs as well as motion information in subsequent images are analyzed and evaluated in a cooperative procedure to get reliable segmentation results. The scene is decomposed into a variable number of depth layers, with each layer showing a subset of the segmented regions. The layered representation can be used in a variety of applications. In this paper, the application aims at synthesizing 3D images for enhanced telepresence allowing the user to ""look around"" in natural scenes (intermediate views for interactive displays). In another application, 3D images showing a natural depth-of-focus are synthesized in order to improve viewing comfort with 3D displays.