Nguyen, MinhMinhNguyenBusse, DominikDominikBussePham, StefanStefanPhamSteglich, StephanStephanSteglichArbanowski, StefanStefanArbanowskiHoltmeier, LeonLeonHoltmeier2025-07-302025-07-302025-03-20https://publica.fraunhofer.de/handle/publica/49009010.1145/3715675.3715820In recent years, the demand for remote rendering and cloud gaming solutions has grown significantly, pushing the boundaries of traditional game development and distribution methods. This work aims to explore and implement a novel approach to streaming rendered frames from the Unity game engine to web clients using the WebTransport protocol [4]. Unity, one of the world's leading game development platforms, provides a powerful environment for creating interactive 3D and 2D content. Its versatility and extensive feature set make it an ideal candidate for exploring advanced rendering and streaming techniques. While Unity already offers solutions to enable the direct streaming of rendered content, we aim to incorporate the new web standard "WebTransport" and test the feasibility of using this new method of data transfer. Built on top of HTTP/3 [3], WebTransport provides low-latency, bidirectional, and multiplexed data transfer between clients and servers. Its ability to handle both reliable and unreliable data streams makes it particularly suitable for real-time applications like game streaming. We utilized a multi-process architecture to achieve the goal of streaming rendered frames from Unity to a web client via WebTransport. The system comprised the following key components: (1) Unity Game Engine: This serves as the primary environment for rendering frames and managing the overall process flow. (2) WebTransport Server: We implemented a WebTransport server using Go, based on the webtransport-go-example [1]. This server facilitated the transmission of encoded video frames to the client via a unidirectional stream as well as the sending and receiving of simple messages via datagrams. (3) FFmpeg: This widespread video encoding tool was employed to encode the raw video frames generated by Unity into a fragmented MP4 format suitable for streaming. (4) Web Client: We developed a simple client application using HTML and JavaScript to display the streamed video. This client was also based on the aforementioned webtransport-go-example. These components are deployed on a Lenovo ThinkPad P1 Gen 5 with 32 GB memory and GPU Nvidia GA104 Geforce RTX 3070 Ti running Windows 11. To accurately assess the performance and latency of the streaming system, we implemented comprehensive timing measurements at various points along the data path. This approach allowed for a detailed investigation of the latency introduced by each component of the system. The following metrics were calculated and displayed on the client: Average time for frame requesting from the GPU (ms) Average time for frame encoding (ms) Average time for data transmission from Unity to the WebTransport server (ms) Average time for data transmission from the WebTransport server to the client (ms) Average time from frame encoding to display (ms) Additionally, when an action is triggered via a button on the client, a command is sent to Unity via a WebTransport datagram. The timing of this process is recorded and displayed on the video stream, providing insight into the responsiveness of the system to user inputs. This comprehensive timing system allowed for a thorough analysis of the latency at each stage of the streaming process, providing valuable data for optimizing the system and understanding its performance characteristics. Fig. 1 shows our user interface and preliminary results of a streaming session. The end-to-end latency is roughly 100 ms, which is reasonable with cloud gaming [2].enWebTransportlive streaminglow latencycloud gamingconference paper