Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters
Reset filters

Settings
Now showing 1 - 10 of 286
No Thumbnail Available
Publication

Bridging the Gap Between IDS and Industry 4.0 - Lessons Learned and Recommendations for the Future

2024 , Alexopoulos, Kosmas , Bakopoulos, Emmanouil , Larrinaga Barrenechea, Felix , Castellvi, Silvia , Firouzi, Farshad , Luca, Gabriele de , Maló, Pedro , Marguglio, Angelo , Meléndez, Francisco , Meyer, Tom , Orio, Giovanni di , Pethig, Florian , Ruíz, Jesús , Treichel, Tagline , Usländer, Thomas , Volz, Friedrich , Watson, Kym , Stojanovic, Ljiljana

The Plattform Industrie 4.0 (PI4.0) and the International Data Spaces Association (IDSA) are two independent, parallel initiatives with clear focuses. While PI4.0 addresses communication and interaction between networked assets in a smart factory and/or supply chain across an asset or product lifecycle, IDSA is about a secure, sovereign system of data sharing in which all stakeholders can realize the full value of their data. Since data sharing between companies requires both interoperability and data sovereignty, the question emerges regarding the feasibility and rationality of integrating the expertise of PI4.0 and IDSA. The IDS-Industrial Community (IDS-I) is an extension of IDSA whose goal is to strengthen the cooperation between IDSA and PI4.0. Two fields of expertise could be combined: The Platform's know-how in the area of Industrie 4.0 (I4.0) and the IDSA's expertise in the areas of data sharing ecosystems and data sovereignty. In order to realize this vision, many aspects have to be taken into account, as there are discrepancies on multiple levels. Specifically, at the reference architecture level, we have the RAMI4.0 model on the PI4.0 side and the IDS Reference Architecture Model (IDS-RAM) on the IDSA side. While the existing I4.0 and IDS specifications are incompatible e.g. in terms of models (i.e., the AAS metamodel and the IDS information model) and APIs, there is also the issue of interoperability between I4.0 and IDS solutions. This position paper aims to bridge the gap between IDS and PI4.0 by not only analyzing how their existing concepts, tools, etc. have been "connected" in different contexts. Rather, this position paper makes recommendations on how different technologies could be combined in a generic way, independent of the concrete implementation of IDS and/or I4.0 relevant technology components. This paper could be used by both the IDS and I4.0 communities to further improve their specifications, which are still under development. The lessons learned and feedback from the initial joint use of technology components from both areas could provide concrete guidance on necessary improvements that could further strengthen or extend the specifications. Furthermore, it could help to promote the IDS architecture and specifications in the industrial production and smart manufacturing community and extend typical PI4.0 use cases to include data sovereignty by incorporating IDS aspects.

View
No Thumbnail Available
Publication

Towards a Digital Representation of Building Systems Controls

2024 , Ihlenburg, Moritz , Réhault, Nicolas , Herkel, Sebastian , Benndorf, Gesa

Energy monitoring and performance optimization processes play a significant role in ensuring energy-efficient building operation. However, the current implementation of these processes requires substantial time and effort due to the fragmented and non-digital nature of technical building equipment information. To address these challenges, we demonstrate the application of the PLCont methodology within an ongoing monitoring process in a real building. PLCont utilizes Semantic Web Technologies and established RDF ontologies to describe the control topology. The digital representation of control functions aligns closely with the IEC standard 61131-3, and the PLCont ontology establishes a connection between the topology and the control functions. We compare the conventional approach and the PLCont methodol-ogy using the supply air temperature control of an Air Handling Unit (AHU) as a use case. A graphical user interface built upon the PLCont backend provides a comprehensive represen-tation of the system's functionality. It displays the system topology as a digital HVAC schema, the function code exported from the PLC, time-series data plots, and textual descriptions of implemented functions. We demonstrate how this approach enhances transparency and facil-itates the identification and elimination of faulty system behavior, ultimately contributing to the optimization and energy efficiency of building systems.

View
No Thumbnail Available
Publication

Attribute-Based Person Retrieval in Multi-Camera Networks

2024 , Specker, Andreas , Beyerer, Jürgen

Attribute-based person retrieval is a crucial component in various realworld applications, including surveillance, retail, and smart cities. Contrary to image-based person identification or re-identification, individuals are searched for based on descriptions of their soft biometric attributes, such as gender, age, and clothing colors. For instance, attribute-based person retrieval enables law enforcement agencies to efficiently search enormous amounts of surveillance footage gathered from multi-camera networks to locate suspects or missing persons. This thesis presents a novel deep learning framework for attribute-based person retrieval. The primary objective is to research a holistic approach that is suitable for real-world applications. Therefore, all necessary processing steps are covered. Pedestrian attribute recognition serves as the base framework to address attribute-based person retrieval in this thesis. Various design characteristics of pedestrian attribute recognition approaches are systematically examined toward their suitability for attribute-based person retrieval. Following this analysis, novel techniques are proposed and discussed to further improve the performance. The PARNorm module is introduced to normalize the model’s output logits across both the batch and attribute dimensions to compensate for imbalanced attributes in the training data and improve person retrieval performance simultaneously. Strategies for video-based pedestrian attribute recognition are explored, given that videos are typically available instead of still images. Temporal pooling of the backbone features over time proves to be effective for the task. Additionally, this approach exhibits faster inference than alternative techniques. To enhance the reliability of attributebased person retrieval rankings and address common challenges such as occlusions, an independent hardness predictor is proposed that predicts the difficulty of recognizing attributes in an image. This information is utilized to remarkably improve retrieval results by down-weighting soft biometrics with an increased chance of classification failure. Additionally, three further enhancements to the retrieval process are investigated, including model calibration based on existing literature, a novel attribute-wise error weighting mechanism to balance the attributes’ influence on retrieval results, and a new distance measure that relies on the output distributions of the attribute classifier. Meaningful generalization experiments on pedestrian attribute recognition and attribute-based person retrieval are enabled for the first time. For this purpose, the UPAR dataset is proposed, which contributes 3.3 million binary annotations to harmonize semantic attributes across four existing datasets and introduces two evaluation protocols. Moreover, a new evaluation metric is suggested that is tailored to the task of attribute-based person retrieval. This metric evaluates the overlap between query attributes and the attributes of the retrieved samples to obtain scores that are consistent with the human perception of a person retrieval ranking. Combining the proposed approaches yields substantial improvements in both pedestrian attribute recognition and attribute-based person retrieval. State-of-the-art performance is achieved concerning both tasks and existing methods from the literature are surpassed. The findings are consistent across both specialization and generalization settings and across the well-established research datasets. Finally, the entire processing pipeline, from video feeds to the resulting retrieval rankings, is outlined. This encompasses a brief discussion on the topic of multi-target multi-camera tracking.

View
No Thumbnail Available
Publication

A Concept Study for Feature Extraction and Modeling for Grapevine Yield Prediction

2024 , Huber, Florian , Hofmann, Benedikt , Engler, Hannes , Gauweiler, Pascal , Fischer, Benedikt , Herzog, Katja , Kicherer, Anna , Töpfer, Reinhard , Gruna, Robin , Steinhage, Volker

Yield prediction in viticulture is an especially challenging research direction within the field of yield prediction. The characteristics that determine annual grapevine yields are plentiful, difficult to obtain, and must be captured multiple times throughout the year. The processes currently used in grapevine yield prediction are based mainly on manually captured data and rigid statistical measures derived from historical insights. Experts for data acquisition are scarce, and statistical models cannot meet the requirements of a changing environment, especially in times of climate change. This paper contributes a concept on how to overcome those drawbacks, by (1) proposing a deep learning driven approach for feature recognition and (2) explaining how Extreme Gradient Boosting (XGBoost) can be utilized for yield prediction based on those features, while being explainable and computationally inexpensive. The methods developed will be influential for the future of yield prediction in viticulture.

View
No Thumbnail Available
Publication

Depth estimation and 3D reconstruction from UAV-borne imagery: Evaluation on the UseGeo dataset

2024 , Hermann, Max , Weinmann, Martin , Nex, Francesco , Stathopoulou, E.K. , Remondino, F. , Jutzi, Boris , Ruf, Boitumelo

Depth estimation and 3D model reconstruction from aerial imagery is an important task in photogrammetry, remote sensing, and computer vision. To compare the performance of different image-based approaches, this study presents a benchmark for UAV-based aerial imagery using the UseGeo dataset. The contributions include the release of various evaluation routines on GitHub, as well as a comprehensive comparison of baseline approaches, such as methods for offline multi-view 3D reconstruction resulting in point clouds and triangle meshes, online multi-view depth estimation, as well as single-image depth estimation using self-supervised deep learning. With the release of our evaluation routines, we aim to provide a universal protocol for the evaluation of depth estimation and 3D reconstruction methods on the UseGeo dataset. The conducted experiments and analyses show that each method excels in a different category: the depth estimation from COLMAP outperforms that of the other approaches, ACMMP achieves the lowest error and highest completeness for point clouds, while OpenMVS produces triangle meshes with the lowest error. Among the online methods for depth estimation, the approach from the Plane-Sweep Library outperforms the FaSS-MVS approach, while the latter achieves the lowest processing time. And even though the particularly challenging nature of the dataset and the small amount of training data leads to a significantly higher error in the results of the self-supervised single-image depth estimation approach, it outperforms all other approaches in terms of processing time and frame rate. In our evaluation, we have also considered modern learning-based approaches that can be used for image-based 3D reconstruction, such as NeRFs. However, due to the significantly lower quality of the resulting 3D models, we have only included a qualitative comparison between NeRF-based and conventional approaches in the scope of this work.

View
No Thumbnail Available
Publication

Potential of Deep Learning methods for image processing in sensor-based sorting: data generation, training strategies and model architectures

2024 , Kronenwett, Felix , Maier, Georg , Längle, Thomas

The main component of a sensor-based sorting system is an imaging sensor and the associated data processing unit for detecting and classifying bulk material objects. High occupancy densities and objects with similar appearance lead to increasing problems for conventional image processing algorithms in object and class separation. Therefore, in this article, specialized Deep Learning approaches were applied to two datasets for instance segmentation. Due to the need for a large amount of training data for such models, a method for synthetic training data generation has been developed. Subsequently, established model architectures as well as an own approach specialized for the problem characteristics is presented and compared regarding their detection performance. Finally, the models are evaluated in terms of their speed and therefore their potential use in a sorting system. Our approach more than halves the inference time of the fastest model while achieving the best detection performance.

View
No Thumbnail Available
Publication

Regression via causally informed Neural Networks

2024 , Youssef, Shahenda , Doehner, Frank , Beyerer, Jürgen

Neural Networks have been successful in solving complex problems across various fields. However, they require significant data to learn effectively, and their decision-making process is often not transparent. To overcome these limitations, causal prior knowledge can be incorporated into neural network models. This knowledge improves the learning process and enhances the robustness and generalizability of the models. We propose a novel framework RCINN that involves calculating the inverse probability of treatment weights given a causal graph model alongside the training dataset. These weights are then concatenated as additional features in the neural network model. Then incorporating the estimated conditional average treatment effect as a regularization term to the model loss function, the potential influence of confounding variables can be mitigated, leading to bias minimization and improving the neural network model. Experiments conducted on synthetic and benchmark datasets using the framework show promising results.

View
No Thumbnail Available
Publication

Fortschrittsbericht zur Digitalisierung des Energiesystems

2024 , Bergsträßer, Jonathan , Berkhout, Volker , Klaiber, Stefan , Klobasa, Marian , Kohrs, Robert , Linnartz, Philipp , Naumann, Steffi , Nicolai, Steffen , Schmidt, Dietrich , Welisch, Marijke , Wende-von Berg, Sebastian , Wickert, Manuel , Wille-Haußmann, Bernhard

Die Digitalisierung ist ein hochgradig relevanter Schlüsselprozess für die Energiesystemtransformation - Details dazu hat der Fraunhofer Exzellenzcluster CINES im Jahr 2022 erforscht und in 14 Thesen zusammengefasst. In 2023 wurden in Zusammenarbeit mit Praxispartner:innen aus der Energiewirtschaft politische und regulatorischen Änderungen in der digitalen Energiewirtschaft analysiert und ausgewertet. Der Fortschrittsbericht zeigt die Fortschritte der Digitalisierung auf und erörtert Handlungsbedarfe und Weiterentwicklungspotenziale. Wichtige Erkenntnisse sind: Positive Fortschritte gibt es u.a. durch gesetzliche und regulatorische Neuerungen wie beispielsweise beim §14a EnWG oder dem Gesetz zum Neustart der Digitalisierung der Energiewende (GNDEW). Es mangelt an einem integrativen Zielbild - ein solches kann für mehr Klarheit und Orientierung bei der Digitalisierung des Energiesystems in Deutschland und Europa sorgen. Um Lücken, Handlungsbedarfe und positiven Fortschritt besser zu erkennen, erfordert es ein gemeinsames Verständnis für die Orientierung und Ausrichtung auf die Digitalisierung. Dafür ist es unter anderem notwendig ein handlungsanleitendes Zukunftsbild zu schaffen, die kommunikative Übersetzung und die Verständlichkeit von regulatorischen Änderungen zu verbessern, Kompetenzen für technologische Lösungen, bspw. für notwendige Cyberresilienz und kritischen Infrastrukturen, aufzubauen und Investitionen zu tätigen, um mehr finanzielle Mittel für die Digitalisierung der Energiesystemtransformation zur Verfügung zu haben.

View
No Thumbnail Available
Publication

Domänenadaptation für feingranulare Fahrzeugklassifikation mittels Domain-Adversarial-Learning

2024 , Wolf, Stefan , Beyerer, Jürgen

View
No Thumbnail Available
Publication

Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning

2024 , Kronenwett, Felix , Maier, Georg , Leiss, Norbert , Gruna, Robin , Thome, Volker , Längle, Thomas

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

View