Supporting fuzzy metric temporal logic based situation recognition by mean shift clustering

This contribution aims at assisting video surveillance operators with automatic understanding of situations in videos. The situations comprise many different agents interacting in groups. To this end we extended an existing situation recognition framework based on Situation Graph Trees and Fuzzy Metric Temporal Logic. Non-parametric meanshift clustering is utilized to support the logic-based inference process for such group-based situations, namely to improve effieciency. Additionally, the underlying knowledge base was augmented to also handle multiagent queries and the situation inference was adapted to also handle inference for group-based situations. For evaluation the publicly available BEHAVE video dataset was used consisting of partially annotated real video data of persons. The results show that the proposed system is capable of correctly and effieciently understanding such group-based situations.