Deformable object tracking using optical flow constraints

In this paper, we present a method for deformable object tracking that exploits the entire image information using the optical flow equation instead of working with discrete feature points. Our method starts from the optical flow constraint and first estimates global transformations in a hierarchical framework. Elastic deformations are then estimated separately using deformable meshes and spatial and temporal smoothing constraints. In both cases additional constraints to regularize the optical flow field are obtained from the motion model such that finding the best transformation amounts in minimizing an error function that can be solved in a least squares sense. Combing deformable meshes and the optical flow equation with a dedicated weighted smoothness constraint on the mesh deformation and estimating global transformations separately from elastic deformations is key to dealing with complex deformations such as cloth deformation as a person moves.