Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Architecture of a Low Latency H.264/AVC Video Codec for robust ML based Image Classification

: Stabernack, B.; Steinert, F.


Association for Computing Machinery -ACM-:
DASIP 2021, Workshop on Design and Architectures for Signal and Image Processing - 14th edition : Jointly with the 16th HiPEAC Conference, January 18-20, 2021, Budapest, Hungary, virtual event
New York: ACM, 2021
ISBN: 978-1-4503-8901-3
Workshop on Design and Architectures for Signal and Image Processing (DASIP) <14, 2021, Online>
International Conference on High-Performance and Embedded Architectures and Compilers (HiPEAC) <16, 2021, Online>
Fraunhofer HHI ()

The use of neural networks represents the state of the art in the area of image classification. A large number of different networks are available for this purpose, which, appropriately trained, permit a high level of classification accuracy. Typically, these networks are applied to uncompressed image data, since a corresponding training was also carried out using image data of similar high quality. However, if image data contains image errors, the classification accuracy generally deteriorates drastically. This applies in particular to coding artifacts that arise from image and video compression. Typical application scenarios for this are narrowband transmission channels for which video coding is required but a subsequent classification is to be carried out on the receiver side. In this paper we present a special H.264/AVC-based video codec that allows certain regions of a picture to be coded with constant picture quality in order to allow a stable and robust classification using neural networks, whereas the remaining image regions will be coded using constant bit rate. We have combined this feature with the ability to run with lowest latency properties, which is usually also required in remote control applications scenarios. The codec has been implemented as a fully hardwired HD capable hardware architecture which is suitable for Field Programmable Gate Arrays.