Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

1
Reset filters

Settings
Now showing 1 - 10 of 307
  • Publication
    Security Fence Inspection at Airports Using Object Detection
    ( 2024)
    Friederich, Nils
    ;
    Specker, Andreas
    ;
    Beyerer, Jürgen
    To ensure the security of airports, it is essential to protect the airside from unauthorized access. For this pur-pose, security fences are commonly used, but they require regular inspection to detect damages. However, due to the growing shortage of human specialists and the large man-ual effort, there is the need for automated methods. The aim is to automatically inspect the fence for damage with the help of an autonomous robot. In this work, we explore object detection methods to address the fence inspection task and localize various types of damages. In addition to evaluating four State-of-the-Art (SOTA) object detection models, we analyze the impact of several design criteria, aiming at adapting to the task-specific challenges. This in-cludes contrast adjustment, optimization of hyperparameters, and utilization of modern backbones. The experimental results indicate that our optimized You Only Look Once v5 (YOLOv5) model achieves the highest accuracy of the four methods with an increase of 6.9% points in Average Precision (AP) compared to the baseline. Moreover, we show the real-time capability of the model. The trained models are published on GitHub: hups://github.com/IN-Friederichlairport_fence_inspection.
  • Publication
    Pattern Recognition. Introduction, Features, Classifiers and Principles
    (De Gruyter, 2024)
    Beyerer, Jürgen
    ;
    Hagmanns, Raphael
    ;
    Stadler, Daniel
    The book offers a thorough introduction to Pattern Recognition aimed at master and advanced bachelor students of engineering and the natural sciences. Besides classification - the heart of Pattern Recognition - special emphasis is put on features: their typology, their properties and their systematic construction. Additionally, general principles that govern Pattern Recognition are illustrated and explained in a comprehensible way. Rather than presenting a complete overview over the rapidly evolving field, the book clarifies the concepts so that the reader can easily understand the underlying ideas and the rationale behind the methods. For this purpose, the mathematical treatment of Pattern Recognition is pushed so far that the mechanisms of action become clear and visible, but not farther. Therefore, not all derivations are driven into the last mathematical detail, as a mathematician would expect it. Ideas of proofs are presented instead of complete proofs. From the authors’ point of view, this concept allows to teach the essential ideas of Pattern Recognition with sufficient depth within a relatively lean book.
  • Publication
    Knowledge-Distillation-Based Label Smoothing for Fine-Grained Open-Set Vehicle Recognition
    ( 2024)
    Wolf, Stefan
    ;
    Loran, Dennis
    ;
    Beyerer, Jürgen
    Fine-grained vehicle classification describes the task of estimating the make and the model of a vehicle based on an image. It provides a useful tool for security authorities to find suspects in surveillance cameras. However, most research about fine-grained vehicle classification is only focused on a closed-set scenario which considers all possible classes to be included in the training. This is not realistic for real-world surveillance applications where the images fed into the classifier can be of arbitrary vehicle models and the large number of commercially available vehicle models renders learning all models impossible. Thus, we investigate fine-grained vehicle classification in an open-set recognition scenario which includes unknown vehicle models in the test set and expects these samples to be rejected. Our experiments highlight the importance of label smoothing for open-set recognition performance. Nonetheless, it lacks recognizing the different semantic distances between vehicle models which result in largely different confusion probabilities. Thus, we propose a knowledge-distillation-based label smoothing approach which considers these different semantic similarities and thus, improves the closed-set classification as well as the open-set recognition performance.
  • Publication
    Enhancing Skeleton-Based Action Recognition in Real-World Scenarios Through Realistic Data Augmentation
    ( 2024)
    Cormier, Mickael
    ;
    Schmid, Yannik
    ;
    Beyerer, Jürgen
    Skeleton-based action recognition is a prominent research area that provides a concise representation of human motion. However, real-world scenarios pose challenges to the reliability of human pose estimation, which is fundamental to such recognition. The existing literature mainly focuses on laboratory experiments with near-perfect skeletons, and fails to address the complexities of the real world. To address this, we propose simple yet highly effective data augmentation techniques based on the observation of erroneous human pose estimation, which enhance state-of-the-art methods for real-world skeleton-based action recognition. These techniques yield significant improvements (up to +4.63 accuracy) on the widely used UAV Human Dataset, a benchmark for evaluating real-world action recognition. Experimental results demonstrate the effectiveness of our augmentation techniques in compensating for erroneous and noisy pose estimation, leading to significant improvements in action recognition accuracy. By bridging the gap between laboratory experiments and real-world scenarios, our work paves the way for more reliable and practical skeleton-based action recognition systems. To facilitate reproducibility and further development, the Skelbumentations library is released at https://github.com/MickaelCormier/Skelbumentations. This library provides the code implementation of our augmentation techniques, enabling researchers and practitioners to easily augment skeleton sequences and improve the performance of skeleton-based action recognition models in real-world applications.
  • Publication
    A survey of the state of the art in sensor-based sorting technology and research
    ( 2024)
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.
  • Publication
    Particle-Specific Deflection Windows for Optical Sorting by Uncertainty Quantification
    ( 2024)
    Reith-Braun, Marcel
    ;
    Liang, Kevin
    ;
    Pfaff, Florian
    ;
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Bauer, Albert
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    ;
    Hanebeck, Uwe D.
    In current state of the art sensor-based sorting systems, the length of the deflection windows, i.e., the period of nozzle activation and the number of nozzles to be activated, is commonly determined solely by the size of the particles. However, this comes at the cost of the sorting process not accounting for model discrepancies between actual and presumed particle motion, as well as for situations where the available information does not allow for precise determination of nozzle activations. To achieve a desired sorting accuracy, in practice, one is therefore usually forced to enlarge the deflection window to a certain degree, which increases the number of falsely co-deflected particles and compressed air consumption. In this paper, we propose incorporating the uncertainty of the prediction of particle motion of each individual particle into the determination of the deflection windows. The method is based on the predictive tracking approach for optical sorting, which tracks the particles while they move toward the nozzle array based on images of an area-scan camera. Given the extracted motion information from the tracking, we propose an approximation for the distribution of arrival time and location of the particle at the nozzle array assuming nearly constant-velocity or nearly constantacceleration particle motion behavior. By evaluating the quantile function of both distributions, we obtain a confidence interval for the arrival time and location based on prediction uncertainty, which we then combine with the particle size to form the final deflection window. We apply our method to a real sorting task using a pilot-scale chute sorter. Our results obtained from extensive sorting trials show that sorting accuracies can be remarkably improved compared with state-of-the-art industrial sorters and enhanced even further compared with predictive tracking while having the potential to reduce compressed air consumption.
  • Publication
    Unsupervised 3D Skeleton-Based Action Recognition using Cross-Attention with Conditioned Generation Capabilities
    ( 2024)
    Lerch, David
    ;
    Zhong, Zeyun
    ;
    Martin, Manuel
    ;
    Voit, Michael
    ;
    Beyerer, Jürgen
    Human action recognition plays a pivotal role in various real-world applications, including surveillance systems, robotics, and occupant monitoring in the car interior. With such a diverse range of domains, the demand for generalization becomes increasingly crucial. In this work, we propose a cross-attention-based encoder-decoder approach for unsupervised 3D skeleton-based action recognition. Specifically, our model takes a skeleton sequence as input for the encoder and further applies masking and noise to the original sequence for the decoder. By training the model to reconstruct the original skeleton sequence, it simultaneously learns to capture the underlying patterns of actions. Extensive experiments on NTU and NW-UCLA datasets demonstrate the state-of-the-art performance as well as the impressive generalizability of our proposed approach. Moreover, our experiments reveal that our approach is capable of generating conditioned skeleton sequences, offering the potential to enhance small datasets or generate samples of under-represented classes in imbalanced datasets. Our code will be published on GitHub.
  • Publication
    Fusion between Event-Based and Line-Scan Camera for Sensor Based Sorting
    ( 2024)
    Bäcker, Paul
    ;
    Terzer, Nick
    ;
    Maier, Georg
    ;
    Hanebeck, Uwe D.
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    In sensor-based sorting systems, there is usually a time delay between the detection and separation of the material stream. This delay is required for the sensor data to be processed, i.e., to identify the objects that should be ejected. In this blind phase, the material stream continues to move. In most current systems, homogeneous movement for all objects is assumed, and actuation is timed accordingly. However, in many cases, this assumption does not hold true, for example, when unknown, foreign materials are present that have varying density and shapes, leading to inaccurate activation of the separation actuators and in turn lower sorting quality. Minimizing the blind phase by reducing the distance between the sensor and the actor is limited by the processing time of the detection process and may lead to interference between actuation and sensing. In this work, we address these issues by using an event-based camera placed between the sensor and actuator stages to track objects during the blind phase with minimal latency and small temporal increments between tracking steps. In our proposed setup, the event-based camera is used exclusively for tracking, while an RGB line-scan camera is used for classification. We propose and evaluate several approaches to combine the information of the two cameras. We benchmark our approach against the traditional method of using a fixed temporal offset by comparing simulated valve activation. Our method shows a drastic improvement in accuracy for our example application, improving the percentage of correctly deflected objects to 99.2% compared to 78.57% without tracking.
  • Publication
    UPAR Challenge 2024: Pedestrian Attribute Recognition and Attribute-Based Person Retrieval - Dataset, Design, and Results
    ( 2024)
    Cormier, Mickael
    ;
    Specker, Andreas
    ;
    Cezar Silveira Jacques Junior, Julio
    ;
    Moritz, Lennart
    ;
    Metzler, Jürgen
    ;
    Moeslund, Thomas B.
    ;
    Nasrollahi, Kamal
    ;
    Escalera, Sergio
    ;
    Beyerer, Jürgen
    Attribute-based person retrieval enables individuals to be searched and retrieved using their soft biometric features, for instance, gender, accessories, and clothing colors. The process has numerous practical use cases, such as surveillance, retail, or smart cities. Notably, attribute-based person retrieval empowers law enforcement agencies to efficiently comb through vast volumes of surveillance footage from extensive multi-camera networks, facilitating the swift localization of missing persons or criminals. However, for real-world application, attribute-based person retrieval is required to generalize to multiple settings in indoor and outdoor scenarios with their respective challenges. For its second edition, the WACV 2024 Pedestrian Attribute Recognition and Attribute-based Person Retrieval Challenge (UPAR-Challenge) aimed once again to spotlight the current challenges and limitations of existing methods to bridge the domain gaps in real-world surveillance contexts. Analogous to the first edition, two tracks are offered: pedestrian attribute recognition and attribute-based person retrieval. The UPAR-Challenge 2024 dataset extends the UPAR dataset with the introduction of harmonized annotations for the MEVID dataset, which is used as a novel test domain. To this aim, 1.1M additional annotations were manually labeled and validated. Each track evaluates the robustness of the competing methods to domain shifts by training and evaluating on data from entirely different domains. The challenge attracted 82 registered participants, which was considered a success from the organizers' perspective. While ten competing teams surpassed the baseline for track 1, no team managed to outperform the baseline on track 2, emphasizing the task's difficulty. This work describes the challenge design, the adopted dataset, obtained results, as well as future directions on the topic. The UPAR-Challenge dataset is available on GitHub: https:/github.com/speckean/upar_challenge.
  • Publication
    Factor Graph-Based Dense Mapping for Mobile Robot Teams Using VDB-Submaps
    ( 2024)
    Hagmanns, Raphael
    ;
    Emter, Thomas
    ;
    Garbe, Leo
    ;
    Beyerer, Jürgen
    A large number of works exist in the field of mobile robot based simultaneous localization and mapping. While the original SLAM problem has been considered solved for years, there still exist various environments, use cases, or robot configurations which require new approaches in order to successfully perform the task. This work addresses how a group of mobile robots can collaboratively create a dense 3D map that is globally consistent and accounts for uncertainties in measurement data and estimates. The main challenge is a compact representation of the robot-local submaps in order to minimize the data flow as well as a fast and accurate merging scheme to create a consistent global map. We leverage OpenVDB as underlying data structure to efficiently create submaps which are then fused in a factor graph-based backend. We extensively test and evaluate the framework and show that it is capable of creating dense 3D maps of challenging environments in real-time.