Benchmarking AI-Based Monocular Depth Estimators in Terms of Their Metrological Potential Following 3-D Sensor Guideline VDI/VDE 2634

Monocular depth estimation is a computer vision task in which a neural network is trained to estimate depth maps from given images. Recently, some estimators have reached remarkable results with potential to replace conventional 3-D sensors in certain applications. To investigate how they compare in terms of metrological performance, we applied the VDI/VDE 2634 evaluation guideline from the 'Verein Deutscher Ingenieure e.V.' This guideline is used to specify the probing and length measurement errors of a 3-D sensor by capturing data from a calibrated ball bar specimen in different orientations within a predefined measurement volume. We evaluated three recent monocular depth estimators Depth Anything V2, Depth Pro, and UniDepthV2 in different settings, which achieved probing and length measurement errors below 10 % under optimal conditions. However, under nonoptimal conditions, each of the three depth estimators showed significant errors. Adding everyday objects into the image scenes improved the overall results. Our image collection, the MD-VDI2634 dataset, enables the investigation and comparison of depth estimators regarding their metrological performance.