360° video cloud streaming & HTMLVideoElement extensions

Presented in the "360° video on the Web" Session: 360° video on the Web session identified needs for evolutions in streaming infrastructure (both on server and client side) to adapt to the heavy needs of 360° content streaming, and build understanding on what changes are needed to HTML media interfaces to make them suitable for 360° media content. In this presentation, a proof-of-concept 360° video cloud streaming solution that enables high quality 360° video experience on low capability devices such as Hybrid TVs or mobile devices was presented. Natural 360° video experience requires the spectator to be able to freely change her individual perspective of view, which requires the full spherical image of any direction of view to be available in every moment. The options discussed are to render the view on the server side and stream only the selected 360° video content to the end device, or to do the 360° video processing on the client side using the existing W3C Media Source Extensions API that makes it possible to implement the entire logic of the 360° player in the browser. The motion-to-photon-latency which depends on different factors like network latency, buffering strategies and segment duration is a known limitation. We also identified in this presentation a need for extensions to the Media Source Extensions API and HTMLVideoElement in order to provide a standards-based 360° experience, or alternatively need to provide hooks in HTMLMediaElement for a native 360° video player to integrate with, including metadata required to render the view correctly, an API to set and get the field of view with associated change events, and possibly zoom factor.