1 - 10 of 116
-
PublicationNutzung von Blickbewegungen für die Mensch-Computer-Interaktion mit dynamischen Bildinhalten am Beispiel der VideobildauswertungInteraktion mit dynamischen Bildinhalten ist für Systemnutzer herausfordernd bezüglich Wahrnehmung, Kognition und Motorik. Die vorliegende Arbeit identifiziert geeignete blickbasierte Interaktionstechniken zur Selektion bewegter Objekte in Bildfolgen mithilfe mehrerer Querschnitt- und einer Längsschnittstudie. Sie untersucht, wie blickbasierte Interaktion und automatische Verfahren bei der Videobildauswertung unterstützen und ob blickbasierte Klassifikation der Benutzertätigkeit möglich ist.
-
PublicationAttribute-Based Person Retrieval in Multi-Camera Networks( 2024)Attribute-based person retrieval is a crucial component in various realworld applications, including surveillance, retail, and smart cities. Contrary to image-based person identification or re-identification, individuals are searched for based on descriptions of their soft biometric attributes, such as gender, age, and clothing colors. For instance, attribute-based person retrieval enables law enforcement agencies to efficiently search enormous amounts of surveillance footage gathered from multi-camera networks to locate suspects or missing persons. This thesis presents a novel deep learning framework for attribute-based person retrieval. The primary objective is to research a holistic approach that is suitable for real-world applications. Therefore, all necessary processing steps are covered. Pedestrian attribute recognition serves as the base framework to address attribute-based person retrieval in this thesis. Various design characteristics of pedestrian attribute recognition approaches are systematically examined toward their suitability for attribute-based person retrieval. Following this analysis, novel techniques are proposed and discussed to further improve the performance. The PARNorm module is introduced to normalize the model’s output logits across both the batch and attribute dimensions to compensate for imbalanced attributes in the training data and improve person retrieval performance simultaneously. Strategies for video-based pedestrian attribute recognition are explored, given that videos are typically available instead of still images. Temporal pooling of the backbone features over time proves to be effective for the task. Additionally, this approach exhibits faster inference than alternative techniques. To enhance the reliability of attributebased person retrieval rankings and address common challenges such as occlusions, an independent hardness predictor is proposed that predicts the difficulty of recognizing attributes in an image. This information is utilized to remarkably improve retrieval results by down-weighting soft biometrics with an increased chance of classification failure. Additionally, three further enhancements to the retrieval process are investigated, including model calibration based on existing literature, a novel attribute-wise error weighting mechanism to balance the attributes’ influence on retrieval results, and a new distance measure that relies on the output distributions of the attribute classifier. Meaningful generalization experiments on pedestrian attribute recognition and attribute-based person retrieval are enabled for the first time. For this purpose, the UPAR dataset is proposed, which contributes 3.3 million binary annotations to harmonize semantic attributes across four existing datasets and introduces two evaluation protocols. Moreover, a new evaluation metric is suggested that is tailored to the task of attribute-based person retrieval. This metric evaluates the overlap between query attributes and the attributes of the retrieved samples to obtain scores that are consistent with the human perception of a person retrieval ranking. Combining the proposed approaches yields substantial improvements in both pedestrian attribute recognition and attribute-based person retrieval. State-of-the-art performance is achieved concerning both tasks and existing methods from the literature are surpassed. The findings are consistent across both specialization and generalization settings and across the well-established research datasets. Finally, the entire processing pipeline, from video feeds to the resulting retrieval rankings, is outlined. This encompasses a brief discussion on the topic of multi-target multi-camera tracking.
-
PublicationMethodische Evaluation von Verfahren zur Korrektur atmosphärischer Turbulenz in Bildsequenzen(KIT, 2023-03-01)Bei langen horizontalen Ausbreitungswegen in Bodennähe ist die Atmosphäre und nicht die Qualität moderner bildgebender Systeme ausschlaggebend für die Qualität aufgenommener Bilddaten. Besonders wird die Bildqualität durch atmosphärische Turbulenz beeinträchtigt, die je nach Schweregrad zeitlich und räumlich variierende Unschärfe, (scheinbare) Bildbewegungen und geometrische Deformationen, sowie Intensitätsfluktuationen (Szintillation), verringerten (Farb-)Kontrast und Rauschen verursacht. Korrekturverfahren haben entsprechend die Aufgabe, einen, mehrere oder ggfs. alle dieser Turbulenzeffekte in Bilddaten zu reduzieren und diese bestmöglich zu rekonstruieren. Im Idealfall wäre eine solche Rekonstruktion identisch mit einer Aufnahme am Diffraktionslimit ohne Turbulenz.
-
Publication3D Human Body Pose-Based Activity Recognition for Driver Monitoring Systems( 2023)Traffic accidents are unfortunately an unavoidable part of our modern transport system. In many cases the behavior of drivers is a contributing factor. Different studies show that distractions caused by secondary activities, like the use of mobile phones, play a large role. In addition, the trend to occupy oneself with other activities rises with increasing vehicle automation because drivers are less involved in the driving task. For this reason, driver monitoring systems will be required for future automated cars. In this thesis we therefore investigate how to detect the activities of drivers in automated cars using a modular recognition system based on 3D data. The proposed system consists of two stages. The first stage creates a 3D interior state model from camera data including the 3D body pose of the driver, the location of elements of the interior as well as the position of objects involved in certain activities. The second stage uses this representation to detect activities. We base this design on two hypotheses: First, a rich 3D interior state model including other elements in addition to the 3D driver body pose is important to discern fine-grained activities. Second, the interior state model, created by the first stage, is a sensor modality and sensor location independent representation allowing the second stage to successfully detect activities even with large changes to the camera system. To verify these assumptions, we contribute to the research field in three areas. The foundation of all our efforts are annotated datasets. Based on our extensive literature review we show that there are no suitable public automotive data sources. We therefore collect multiple datasets for different subtasks leading to our final dataset, published under the name Drive&Act. It includes a large-scale hierarchical activity recognition benchmark with multiple 3D input modalities for the task, like the 3D body pose of the driver, the location of interior elements like the steering wheel, as well as 3D trajectories of objects like smartphones. In addition, the dataset includes a public benchmark for 3D driver body pose estimation with challenging partial occlusion of the driver’s body. The main feature of the 3D interior state model is the body pose of the driver. Here we contribute methods for real-time 3D driver body pose estimation based on depth images. The primary challenge of using depth data for this task is occlusion of body parts. A second challenge is the lack of public automotive datasets to train these methods. In our first approach we therefore rely on simulated depth images with automated annotation for training. Our second approach handles this problem on the algorithm level. It uses a novel split of 2D body pose estimation followed by separate 3D keypoint regression guided by the depth image. This enables us to rely on advances in 2D body pose estimation using large-scale datasets from other domains. We can demonstrate the robustness of this method to partial occlusion on the 3D body pose benchmark of Drive&Act. While the primary input of many related activity recognition methods is just the 3D human body pose, we research how to expand the input to a complex 3D state model including elements of the surrounding vehicle interior as well as positions of objects relevant for certain activities. We follow one central paradigm and assume that the distance in 3D space of keypoints of the driver’s body to other elements in the state model is an important indicator of their relevance for the performed activity. Based on this hypothesis we develop different methods with increasing complexity of the interior state model. Our final method casts all parts of the interior state model into a spatio-temporal graph. To generate this graph, we rely on the distance of keypoints of the driver’s body to other parts of the state model to determine which nodes to include in the graph and what edges to create. We analyze this graph using a neural network based on graph convolutions. We can show the advantage of our graph creation method in selecting relevant interior elements and objects and the usefulness of object location data to discern activities represented by similar body poses of the driver. Consequently, we can prove our initial hypothesis that additional input modalities improve the detection of fine-grained activities based on 3D data and we can quantify their impact. We also investigate the overall performance of our modular system regarding sensor modality and viewpoint changes. We can demonstrate the capability to switch between creating the interior state model based on a multi-view camera system and creating it using data from a single depth sensor. We can show that our activity recognition approach can be trained on one of these representations and evaluated on the other with just a moderate performance drop. In addition, the overall system can generalize across different datasets recorded in different vehicles and in vastly different conditions, switching between data recorded in a simulator for automated driving and data recorded on a test track driving manually. This supports our second hypothesis that the 3D interior state model resulting from our first stage of algorithms is sensor independent to a large degree.
-
PublicationOn Diagnosing Cyber-Physical Systems( 2023)Cyber-physical systems are a class of technical systems that integrate mechanical components with intelligent, adaptable control devices and software. Nowadays, this integration enables high-performance, modular, and parameterized systems with high complexity, but low operating cost. Typical examples of cyber-physical systems are production machinery, cars, aeroplanes, and smart home appliances. In this thesis, the focus is on diagnosing faults within cyber-physical systems used in industrial production contexts. Faults occurring during production quickly lead to degrading product quality or production stops, which can be costly and may endanger human lives. Existing approaches to automated fault diagnosis are mostly defined on narrow use-cases or require a significant amount of expert knowledge. In this thesis, three different algorithms to automatically identify faults in cyber-physical systems are presented to mitigate these drawbacks. Therefore, this thesis makes four main contributions: (i) It introduces a novel diagnosis algorithm HySD to find faults in cyber-physical systems. (ii) It presents a new uninformed algorithm DDRC to learn diagnosis models from process data, using correlations in time-series data. (iii) It presents the new algorithm DDGD, which learns diagnosis models from time-series data supervised, using Granger Causality. (iv) It provides a novel theory to describe fault propagation in cyber-physical systems. More precise, the algorithm HySD uses satisfiability modulo linear arithmetic to combine process data with traditional symbolic consistency-based diagnosis algorithms. However, the algorithm heavily relies on models formulated by experts. Therefore, the algorithms DDRC and DDGD are introduced to learn diagnosis models from process data automatically. All algorithms build on the foundation of the theory of fault propagation. The algorithms were evaluated on internationally accepted benchmarks of tank systems, the well-known Tennessee Eastman Process, and two industrial use-cases. Throughout all empirical results, the algorithms exhibit good performance in learning suitable models and in diagnosing faults in synthetic and real fault scenarios.
-
PublicationMultimodal Panoptic Segmentation of 3D Point Clouds(KIT Scientific Publishing, 2023)The understanding and interpretation of complex 3D environments is a key challenge of autonomous driving. Lidar sensors and their recorded point clouds are particularly interesting for this challenge since they provide accurate 3D information about the environment. This work presents a multimodal approach based on deep learning for panoptic segmentation of 3D point clouds. It builds upon and combines the three key aspects multi view architecture, temporal feature fusion, and deep sensor fusion.
-
PublicationAnomaliedetektion in räumlich-zeitlichen Datensätzen(KIT Scientific Publishing, 2023)Eine Unterstützung des Menschen in Überwachungsaufgaben spielt eine immer wichtigere Rolle, da die schiere Menge der anfallenden Daten von heterogenen Sensoren eine Überforderung des Menschen zur Folge hat. Hierfür müssen dem Menschen in kritischen Entscheidungen die wichtigsten Informationen transparent dargebracht werden, um so das Situationsbewusstsein zu stärken. In dieser Arbeit wird der maritime Raum als Beispiel für die Entwicklung verschiedener Datenfusionsverfahren zu ebendiesem Zweck herangezogen. Der maritime Raum als Anwendungsszenario bietet durch seine enorme wirtschaftliche Bedeutung für den Welthandel, das Auftreten verschiedenster Anomalien und krimineller Handlungen wie Piraterie und illegaler Fischerei und die Verfügbarkeit von Datenquellen ein gut für die Erprobung der Verfahren geeignetes Umfeld. Die entwickelten und untersuchten Verfahren decken hierbei die gesamte Bandbreite von einfachen Positions- und kinematischen Anomalien, über kontextuelle Anomalien bis zu komplexen Anomalien ab. Für die Untersuchung werden verschiedene Datensätze mit realen Schiffsverkehrsinformationen genutzt. Außerdem werden die Verfahren teilweise in Live Trials mit Küstenwachen erprobt. Zur Entwicklung der Verfahren wird als Grundlage zunächst das objektorientierte Weltmodell um Verhaltensmodelle erweitert sowie das EUCISE-Datenmodell als Basis für die Modellierung des verfügbaren Hintergrundwissens identifiziert. Die ersten untersuchten Verfahren detektieren Anomalien in der Position und der Kinematik basierend auf einzelnen Datenpunkten oder ganzen Trajektorien. Hierbei wurde festgestellt, dass zwar Anomalien erkannt werden, die Korrektklassifikationsrate für einen tatsächlichen Einsatz aber deutlich zu hoch ausfällt sowie bestimmte Anomalien ohne Kontext nicht bestimmbar sind. Im nächsten Schritt wird ein Multiagentensystem aufgestellt, welches das Verhalten der beobachteten Objekte durch spieltheoretische Modelle simuliert. Die hierzu notwendigen Nutzenfunktionen werden sowohl wissensbasiert als auch datengetrieben hergeleitet. Mit den integrierten Kontextinformationen können echte Anomalien deutlich besser von normalem Verhalten abgegrenzt werden. Des Weiteren wird gezeigt, wie mit Hilfe von Merkmalen, die aus georeferenzierten Informationen abgeleitet werden, Kontextinformationen zur Klassifikation von Schiffstypen in neuronalen Netzen integriert werden können. Im letzten Schritt werden komplexe Anomalien in Form von spezifischen Situationen basierend auf dynamischen Bayes’schen Netzen modelliert und in Live Trials erprobt. Hierbei werden Kontextinformationen, wie das Wetter, sowie Datenquellen mit unterschiedlicher Zuverlässigkeit integriert, um Situationen in verschiedenen durch Endanwender/-innen mitgestalteten Anwendungsszenarien zu erkennen. Insgesamt wird gezeigt, dass mit automatischen Verfahren Anomalien unterschiedlicher Art erkannt werden können. Die Verfahren werden jeweils mit realen Daten evaluiert, um die Möglichkeit des tatsächlichen Einsatzes als Entscheidungsunterstützung für Menschen in realen Anwendungsszenarien aufzuzeigen.
-
PublicationÜber die Formalisierung und Analyse medizinischer Prozesse im Kontext von Expertenwissen und künstlicher Intelligenz(KIT Scientific Publishing, 2023)Die Digitalisierung hat bereits viele Bereiche der Wirtschaft und des gesellschaftlichen Lebens verändert. Auch unterliegen die Aspekte des Gesundheitswesens und der klinischen Praxis einem digitalen Wandel. Im Hinblick auf diese Entwicklungen beleuchtet die vorliegende Dissertation die Akquisition, Repräsentation und Nutzung von Prozesswissen im Kontext hybrider KI-Methoden. Zentraler Beitrag ist die strukturerhaltende Hin- und Rücktransformation von Prozessbäumen zu Prozessplänen.
-
PublicationNutzung von Blickbewegungen für die Mensch-Computer-Interaktion mit dynamischen Bildinhalten am Beispiel der Videobildauswertung( 2023)
-
PublicationAbbildende Ellipsometrie mit Lichtwegumkehrung für die optische Charakterisierung von gekrümmten Oberflächen(KIT Scientific Publishing, 2023)Die Ellipsometrie ist ein Messverfahren zur Oberflächencharakterisierung und Dünnschichtmessung von ebenen Oberflächen unter Verwendung von polarisiertem Licht. Ein neues Messprinzip basierend auf Lichtwegumkehrung und Retroreflexion ermöglicht jedoch die Erfassung von beliebigen Freiformflächen. Dieses neue Messprinzip und damit verbundene Fragestellungen zur Messabbildung, Auswertealgorithmik und Mehrdeutigkeiten sowie Freiheitsgrade der Lösungsmenge werden in dieser Arbeit untersucht.
