High-level situation recognition using fuzzy metric temporal logic, case studies in surveillance and smart environments

Although computer vision and other machine perception have made great progress in recent years, corresponding high-level components have not progressed that fast. We present a general purpose framework for high-level situation recognition that is suited for arbitrary application domains and sensor setups. Our approach is hierarchical as opposed to monolithic and we focus on modeling expert knowledge with Fuzzy Metric Temporal Logic and Situation Graph Trees rather than learning from training data. To demonstrate the power and flexibility of our approach, we present case studies in two different settings: guiding the operator's attention in video surveillance and automatic report generation in smart environments. Our results show that this approach can yield a conceptually exhaustive situation recognition for diverse input modalities and application domains.