Target tracking in wireless sensor networks by data fusion with video-based object detection

Localization techniques based on wireless sensor networks (WSNs) are an increasingly popular approach for estimating object positions in a wide area of applications. Nevertheless, accuracy and reliability of the WSN position estimates need to be increased for some applications, e.g. automated people registration in public transport vehicles. This goal can be achieved by incorporating additional sensors like cameras in the localization process. In this paper we introduce a novel data fusion approach for combining WSN position estimates and object positions detected in camera images. We project image coordinates to the WSN coordinate frame and modify an Extended Kalman Filter (EKF) for data fusion. We show how the object positions that arise from 2D camera images are used to reduce the variance of the combined position estimate. We test our method by tracking the movement of a person using WSN positioning and additional measurements obtained by people detection in the corresponding video scene.