Appearance and Motion Based Persistent Multiple Object Tracking in Wide Area Motion Imagery

Wide Area Motion Imagery (WAMI) data acquired by an airborne sensor for ground observation offers great potential for various applications ranging from the protection of borders and critical infrastructure to city monitoring and surveillance. Persistent multiple object tracking, which is a prerequisite for these applications, is generally based on moving object detection, as the characteristics of existing WAMI datasets, e.g. weak appearance of objects, impede the usage of appearance based features. Complex and computationally expensive strategies such as exploiting multiple trackers in parallel or classifier-based local search are typically utilized to detect slow and stopping vehicles that are missed by moving object detection. In this paper, we propose a novel and much simpler tracking-by-detection approach for persistent tracking in WAMI data, which avoids such strategies. To overcome limitations caused by image quality of existing WAMI datasets, our proposed tracker was developed on self-acquired WAMI data recorded with a state-of-the-art industrial camera. The improved image quality enables appearance based object detection by Convolutional Neural Networks (CNNs) in WAMI, which we fuse with motion detection to compensate for missed detections in image regions with partial occlusion or shadows. Our proposed tracker is an extension of Deep SORT with modified track management and data association, which is able to yield high recall even in such difficult image regions as well as for slow or stopping vehicles, outperforming state-of-the-art on our self-acquired dataset.