Towards a multi-purpose monocular vision-based high-level situation awareness system

In surveillance applications human operators are either confronted with a high cognitive load or monotonic time periods where the operator's attention rapidly decreases. Therefore, automatic high-level interpretation of image sequences gains increasing importance in assisting human operators. We present a generic hierarchical system that generates high-level logic-based situation descriptions in various domains. The system consists of two components. First, a vision component provides 3D spatial and temporal information about objects in scenes. Second, the situation recognition component uses knowledge encoded in Situation Graph Trees and a fuzzy graph traversal allowing exhaustive situation awareness. The system is tested with real video data comprising persons, their actions, and interactions. In order to show the domain independence we used recorded data from moving vehicles and static surveillance cameras. The results show that the system is usable with multi modal data and can easily be modified and extended.