3D shape reconstruction of the esophagus from gastroscopic video

In gastroscopy, video endoscopic imaging is applied for the assessment of the esophagus. Video sequences which provide a narrow two dimensional insight are thereby generated. Three dimensional shape reconstructions from such video sequences offer opportunities for intuitive and enhanced visualization of the esophagus, providing additional contextual and geometrical information. Due to lack of features and the variability of the scene, the shape reconstruction bears a challenge for computer vision. In this contribution, a three dimensional reconstruction from gastroscopic video is presented by first computing a panorama image of the esophagus wall using a novel shape from shading approach followed by a 3D alignment of thereby provided 2D contours of the esophagus wall. The resulting 3D point cloud is then registered contour-wise, leading to a regular triangulation which is then texturized using the panorama image and visualized.