3D pose and shape estimation with deformable models in lifelike scenes

In real world scenes a large variety of objects can occur, that have to be recognized and localized by humanoids. In this paper we propose a new approach to initialize three dimensional pose and shape parameters for a large variety of objects, which is applicable to single images. In order to feed the recognition system with a priori knowledge, three-dimensional models are used and for the purpose of coping shape variations, deformable variants of these models are built. Thereby, three dimensional object descriptions, which incorporate shape parameters, are provided to the recognition system. By discretizing relevant shape and pose parameters, synthetic model views are created. A new method for the selection of the best fitting model view is proposed, which is realized, by applying a chain of filters on large sets of relevant model views. The pose parameters, that are associated with the selected model view, are enhanced in precision by means of a model-based parameter optimization technique. Overall, the approach allows to cope with strongly variable object shapes by combining the benefits of appearance based and deformable model-based approaches. We present experimental results, proving the high variability of the proposed method and its robustness against partial occlusions. Furthermore, the method was applied to a real world scene, where the estimated pose parameters and the three-dimensional model were provided to a tracking application.