Analysis of the Usability of IPG Movie in a Car Maker-Simulation for the Development of Perception Algorithms for Automated Driving

Before a perception algorithm finds its way on autonomous driving vehicles, it has to be tested thoroughly. As real world tests are laborious and reproducible tests are beneficial, simulations are used to accelerate this process. Most simulations are highly capable to be used along sensor system models to develop automated driving features. In contrast perception algorithms are mainly developed on real world data. In this work a methodology is developed to measure the usability of a simulation for the development of perception algorithms for automated driving - called realism. This is accomplished by comparing the output of an exemplary perception algorithm on real and recreated virtual test drives and comparing the outputs. With the used procedure a metric is defined to rate the realism of the simulation. Before applying this methodology special requirements on the simulation for the use case are elaborated. Furthermore challenges for the development of perception algorithms using simulations and possible influences on the evaluation are described. Finally the methodology is applied on the vehicle distance estimation and IPG CarMaker using the NVIDIA DriveNet module developed by Continental AG. The analysis shows that even with a minimal test coverage the scenario specific behaviour is present in the simulation, but the errors are leveraged.