Template-free shape from texture with perspective cameras

This paper formulates the Shape-from-Texture (SFT) problem of deriving the shape of an imaged surface from the distortion of its texture as a single-plane/multiple-view Structure-from-Motion (SFM) problem under full perspective projection. As in classical SFT formulations we approximate the surface as being piecewise planar. In contrast to many methods, our approach does not need a frontal view of the texture or the texture elements as reference, as it optimizes 3D patch positions and orientations from transformations between texture elements in the image. The reconstruction results in minimizing a large sparse linear least squares cost function based on the reprojection error, a planarity constraint and the estimated rigid motion between patches. Texture element positions in the image are estimated under the assumption of a regular texture from clustered feature points representing repeating appearances in the image. We present results obtained with synthetic data as well as real data to evaluate our method.