Background Modeling: Dealing with Pan, Tilt or Zoom in Videos

Even simple camera movements like pan, tilt or zoom constitute enormous problems for background subtraction algorithms since the modeling of the background works only under the assumption of a static camera. The problem has been mostly ignored and other algorithms have been used for videos with non-static cameras. Nonetheless, in this paper we introduce a method that adapts the background model to these camera movements by using affine
transformations in combination with a similarity metric, and thereby the algorithm makes background subtraction usable for these situations. Also, to keep the generality of this approach, we first apply a detection step to avoid unnecessary adaptions in videos with a static camera because even small adaptions might otherwise deteriorate the background model over time. The method is evaluated on the extensive changedetection.net data set and could reliably detect camera motion in all videos as well as precisely adapt the model of the background to that motion. This does improve the quality of the background models significantly which consequently leads to a higher accuracy of the segmentations.