Representation, coding and interactive rendering of high-resolution panoramic images and video using MPEG-4

In this paper we present an MPEG-4 compliant system for viewing 360° high-resolution spherical panoramic images and videos. The system allows user navigation within an MPEG-4 3D scene description. Here, two different scene geometries were implemented: One consisting of a spherical geometry around the viewpoint and the other having a cylindrical geometry with associated texture patches each. The textures are either real-world static high-resolution scene images or movie textures This scene dynamically deals with problems like bandwidth and video sizes to provide a real-time viewing experience by dividing the texture into a number of tiles. Thus, only the visible parts of the scene can be rendered. Furthermore, visibility sensors are associated to the texture tiles such that pre-fetching of texture tiles next the visible once is enabled during scene navigation. By varying the tile size and visibility sensor size, experiments for real-time capability of the rendering environment were performed. Additionally, a combination of head tracking device and head mounted display was investigated for a more comfortable scene navigation.