MACGaussian: Robust 3D Gaussian Splatting from Sparse Input Views Using High-Precision Measurement-Arm-Camera (MAC) Capture

Recent techniques like neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS) have led to significant improvements in novel view synthesis. Whereas the explicit scene representation of 3DGS in terms of Gaussians allows real-time rendering with state-of-the-art quality, this approach relies on the availability of many views to achieve a coherent scene representation. In this paper, we investigate the importance of accurate camera poses and demonstrate that this even allows for accurate scene representation based on 3D Gaussian Splatting in a sparse-view setting. For this purpose, we address accurate pose estimation by employing a measurement arm equipped with a camera, achieving precise camera-pose estimates with sub-millimeter accuracy. Based on a newly introduced dataset (Core dataset) with its accurate pose information, we demonstrate superior quality in terms of quality of rendered novel views in comparison to results achieved based on calibrations with Dust3R-based and COLMAP-based initializations of the 3D Gaussians. Thereby, our approach offers a reliable and effective solution to practical, sparse-view reconstruction for the preservation of cultural heritage artifacts, which is particularly relevant in applications like virtual museums and archaeology. Furthermore, we expect our Core dataset to serve as a reasonable benchmark, advancing the understanding and development of robust 3D reconstruction methods.