A Lightweight 3D Gaussian Splatting Based Virtual Object Insertion Strategy for Casually Captured Scenes

The physically plausible insertion of virtual objects into scenes captured by multi-view images can be applied to multiple domains such as urban planning, augmented reality and digital marketplaces. Recent advancements in Novel View Synthesis, such as 3D Gaussian Splatting provide photorealistic representations of casually captured scenes. This raises the question if these advancements can be incorporated into the virtual insertion of single objects. In this work, we propose a lightweight strategy that estimates the effects of light interaction between a virtually inserted mesh model with corresponding material properties and a scene represented as 3D Gaussian Splats. Our approach poses minimal constraints on the capturing setup, only relying on casually captured LDR multi-view scenes, i.e., by a smartphone, while demonstrating comparable visual fidelity to strategies that pose more constraints on the capturing process.