Context-aware video encoding as a network-based media processing (NBMP) workflow

Leveraging processing capabilities and resources in a network is a trending approach in accomplishing complex media processing tasks. At the same time, efficiently utilizing available resources while ensuring the potential for scalability and distribution is key. However, deploying, operating and maintaining such complex media service workflows on different cloud services, at the edge or on-premise can be a very complex and time-consuming task. In this paper, we will present an approach that addresses these challenges by utilizing state-of-the-art technologies and standards for advanced multimedia services such as the MPEG Network-based Media Processing (NBMP) standard. We will apply the presented approach for implementing bandwidth reduction and optimization strategies by using context aware video encoding. Implemented as an automated NBMP workflow, the context aware encoding method with the support of machine learning models avoids computationally heavy test encodes. The models are trained on complex datasets composed of 40+ video attributes and generate an optimal encoding ladder as an output (bitrate/resolution pairs). In comparison to the conventional per-title encoding method, we observed significant savings in terms of storage and delivery costs, while maintaining the same visual quality.