Point Cloud Quality Metrics for Incremental Image-based 3D Reconstruction

Image-based 3D reconstruction is a method that can reconstruct the geometry and texture of an object from images of that object. A crucial factor for the accuracy and completeness of the resulting reconstructed model is choosing from where images are captured. This task is called view planning. An iterative approach for autonomously capturing objects involves a feedback loop that switches between view planning and incremental reconstruction. Here, an intermediate 3D reconstruction based on the current images is used to plan the next set of poses for capturing images. These new images are, in turn, used to update the 3D reconstruction, which can subsequently be used to plan a new set of poses. The most important step for autonomously digitizing an object using a feedback loop is identifying which parts of an object are ”poorly reconstructed”, hence would benefit from being part of additional images. This thesis tests the hypothesis that calculating so-called point cloud quality metrics on a reconstructed point cloud can effectively provide this feedback. The hypothesis was confirmed in both simulated and real world scans. This work presents a comprehensive comparison of 24 local and six global point cloud quality metrics for 3D reconstruction. The local metrics assign a value to each point of the currently reconstructed point cloud. In comparison, the global metrics effectively quantify the overall reconstruction progress by calculating a single value for the whole point cloud. The local metrics were visually and mathematically compared based on their response to partially reconstructed objects, their behavior over time, their transferability between different reconstruction qualities, and their usefulness for view planning. Most importantly, a novel local metric was proposed that combines a selection of existing local metrics to provide a more effective feedback for view planning. This metric resulted in a superior performance compared to other metrics when used inside a simulated feedback loop. Additionally, this thesis introduces a novel image-based 3D reconstruction pipeline that is able to incrementally update a reconstruction whenever new images are captured. In contrast to existing reconstruction solutions, the proposed pipeline can efficiently update a sparse and dense reconstruction of an object without having to recompute either one from scratch. The robustness and accuracy of the pipeline was systematically evaluated on synthetic datasets, showing a high robustness against noise in the input data and a sub-millimeter accuracy of the reconstructed point clouds. The metrics and reconstruction pipeline were integrated into the autonomous 3D scanning system CultArm3D-P and tested inside an iterative feedback loop. Using this, a selection of models with varying sizes, complexities, and materials was autonomously captured based on the feedback provided by the proposed combined metric. In conclusion, this thesis represents an important step towards an efficient and fully autonomous reconstruction of 3D objects. Point cloud quality metrics are used to iteratively provide feedback based on incrementally updated reconstructions, which are calculated with a novel reconstruction pipeline. The effectiveness of this approach is shown in simulated and real world scans.