Generalizing Neural Radiance Fields for Robust 6D Pose Estimation of Unseen Appearances

Estimating the 6D pose of objects is a critical challenge for robotics and augmented reality applications. The problem is aggravated by the fact that critical attributes, such as an object’s texture and material, as well as the specific lighting conditions under which it must be identified, are often unknown. Neural Radiance Fields (NeRFs) and 3D Gaussian splatting (3DGS) are techniques that enable high-quality reconstruction of real-world scenes. By revising the scene fitting function, these representations can facilitate the estimation of an object’s pose within a given environment. However, a major complication is that the unique textures, materials, and lighting conditions are fixed within the scene, which can impair the accuracy of pose estimation. To address this, we adopt two alterations to the standard NeRF framework that enhance its ability to handle greatly varied object appearances such as material and texture. Our modified approaches are evaluated on the prevalent YCB-V object dataset, demonstrating their effectiveness. Our two proposed algorithms achieve mesh-free 6D Object Pose Estimation for objects with previously unseen appearances, requiring only a collection of input images to train the NeRF model.