Exploring the Impact of Geometric Model Fidelity on the Sim-to-Real Gap in Visual Robot Perception

In recent years, synthetic training data has proven its potential to overcome data scarcity as a major bottleneck in AI applications. Robotics benefits from this approach, as perceptual AI models can be trained safely and cost-effectively. However, it is impossible to fully reproduce reality, which results in a loss of performance when the models are applied to real-world data. To control the so-called sim-to-real gap, domain randomization is widely regarded as a solution. The variation of hard-to-control parameters, such as lighting conditions or object textures in the case of visual perception, forces models to focus on more robust shape-related information. Extensive research has been devoted to determining the appropriate level of randomization for computer vision models. Thereby, the availability of geometric object models is usually taken for granted. However, industrial practice shows that these models are often unavailable or incorrect. The effort required for geometric modelling correlates with model fidelity and can easily exceed the cost of actual synthetic data generation. To provide practical guidance, this paper investigates how the sim-to-real gap is influenced by model fidelity. In addition to conventional computer-aided design, 3D scans of real objects were considered as a cost-effective alternative for producing geometrically accurate models. The results indicate that high geometric model fidelity is not mandatory and that simplified models are a cost-effective alternative.