Bringing stage events with real actors to your home in VR - A low complexity solution for immersive media streaming

In this paper we demonstrate a novel method how to integrate stereo content of a live act into a virtual reality (VR) environment. This opens a new level of immersion, when watching the recording of a stage event using a head mounted display. In contrast to existing solutions, we offer proper depth perception, allow for a low complexity recording and distribution and still avoid any motion sickness caused by objects being close to the observer. To this end, we separate the digitization of the event location from the recording of the actor’s performance. As a main contribution of the paper, we show how to achieve high visual quality when compositing the stereo recording showing the actors with the reconstructed 3D-model of the event location. To do so, we have created a tool that allows to simulate the 3D perception of the stereo video depending on the selected stereo parameters such as inter-camera distance and focal length. By these means, we can ensure a smooth transition between the stereo video and the 3D model of the event location, leading to superior visual quality. Numerous lessons learned conclude our paper showing how we overcame practical problems encountered when recording a real theater play.