Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters
Reset filters

Settings
Now showing 1 - 10 of 1513
No Thumbnail Available
Publication

Potential of Deep Learning methods for image processing in sensor-based sorting: data generation, training strategies and model architectures

2024 , Kronenwett, Felix , Maier, Georg , Längle, Thomas

The main component of a sensor-based sorting system is an imaging sensor and the associated data processing unit for detecting and classifying bulk material objects. High occupancy densities and objects with similar appearance lead to increasing problems for conventional image processing algorithms in object and class separation. Therefore, in this article, specialized Deep Learning approaches were applied to two datasets for instance segmentation. Due to the need for a large amount of training data for such models, a method for synthetic training data generation has been developed. Subsequently, established model architectures as well as an own approach specialized for the problem characteristics is presented and compared regarding their detection performance. Finally, the models are evaluated in terms of their speed and therefore their potential use in a sorting system. Our approach more than halves the inference time of the fastest model while achieving the best detection performance.

View
No Thumbnail Available
Publication

Metrics for the evaluation of learned causal graphs based on ground truth

2024 , Rehak, Josephine , Falkenstein, Alexander , Doehner, Frank , Beyerer, Jürgen

The self-guided learning of causal relations may contribute to the general maturity of artificial intelligence in the future. To develop such learning algorithms, powerful metrics are required to track advances. In contrast to learning algorithms, little has been done in regards to developing suitable metrics. In this work, we evaluate current state of the art metrics by inspecting their discovery properties and their considered graphs. We also introduce a new combination of graph notation and metric, which allows for benchmarking given a variety of learned causal graphs. It also allows the use of maximal ancestral graphs as ground truth.

View
No Thumbnail Available
Publication

Regression via causally informed Neural Networks

2024 , Youssef, Shahenda , Doehner, Frank , Beyerer, Jürgen

Neural Networks have been successful in solving complex problems across various fields. However, they require significant data to learn effectively, and their decision-making process is often not transparent. To overcome these limitations, causal prior knowledge can be incorporated into neural network models. This knowledge improves the learning process and enhances the robustness and generalizability of the models. We propose a novel framework RCINN that involves calculating the inverse probability of treatment weights given a causal graph model alongside the training dataset. These weights are then concatenated as additional features in the neural network model. Then incorporating the estimated conditional average treatment effect as a regularization term to the model loss function, the potential influence of confounding variables can be mitigated, leading to bias minimization and improving the neural network model. Experiments conducted on synthetic and benchmark datasets using the framework show promising results.

View
No Thumbnail Available
Publication

Retrofitting cyber-physical production systems with radio-based sensors and ML

2024 , Kühnert, Christian , Wallner, Steffen , Wessels, Lars , Wunsch, Andreas , Ziebarth, Mathias

Manufacturing companies usually isolate their production networks from other networks to ensure security against external attacks and to guarantee a fail-safe 24/7 operational service. However, these measures make it technically and organizationally complex to install new sensors or deploy new software in the production process. As a result, machine learning is only used to a limited extent in manufacturing, as these models require regular adaptations. To tackle this challenge, one possible solution is to install an additional network that is not connected to the production network. This network can be utilized for rapid prototyping of new sensors, advanced data analysis, or the deployment of machine learning models. One possible solution is to install a radio-based low-power, long-range network, having the property to capture data over large distances with only little power consumption. This paper examines the potential of retrofitting cyberphysical systems with such a network in combination with machine learning methods. The results are evaluated through three practical use cases: monitoring a workspace with a molding machine, monitoring the cycles of a washing machine, and predicting the daily consumption profile of a main water pipeline.

View
No Thumbnail Available
Publication

Root cause analysis using anomaly detection and temporal informed causal graphs

2024 , Rehak, Josephine , Youssef, Shahenda , Beyerer, Jürgen

In industrial processes, anomalies in the production equipment may lead to expensive failures. To avoid and avert such failures, the identification of the right root cause is crucial. Ideally, the search for a root cause is backed by causal information such as causal graphs. We have extended a framework that fuses causal graphs with anomaly detection to infer likely root causes. In this work, we add the use of temporal information to draw temporal valid conclusions about the potential propagation of anomalous information in causal graphs. The use of the framework is demonstrated on a robotic gripping process.

View
No Thumbnail Available
Publication

Domänenadaptation für feingranulare Fahrzeugklassifikation mittels Domain-Adversarial-Learning

2024 , Wolf, Stefan , Beyerer, Jürgen

View
No Thumbnail Available
Publication

Attribute-Based Person Retrieval in Multi-Camera Networks

2024 , Specker, Andreas , Beyerer, Jürgen

Attribute-based person retrieval is a crucial component in various realworld applications, including surveillance, retail, and smart cities. Contrary to image-based person identification or re-identification, individuals are searched for based on descriptions of their soft biometric attributes, such as gender, age, and clothing colors. For instance, attribute-based person retrieval enables law enforcement agencies to efficiently search enormous amounts of surveillance footage gathered from multi-camera networks to locate suspects or missing persons. This thesis presents a novel deep learning framework for attribute-based person retrieval. The primary objective is to research a holistic approach that is suitable for real-world applications. Therefore, all necessary processing steps are covered. Pedestrian attribute recognition serves as the base framework to address attribute-based person retrieval in this thesis. Various design characteristics of pedestrian attribute recognition approaches are systematically examined toward their suitability for attribute-based person retrieval. Following this analysis, novel techniques are proposed and discussed to further improve the performance. The PARNorm module is introduced to normalize the model’s output logits across both the batch and attribute dimensions to compensate for imbalanced attributes in the training data and improve person retrieval performance simultaneously. Strategies for video-based pedestrian attribute recognition are explored, given that videos are typically available instead of still images. Temporal pooling of the backbone features over time proves to be effective for the task. Additionally, this approach exhibits faster inference than alternative techniques. To enhance the reliability of attributebased person retrieval rankings and address common challenges such as occlusions, an independent hardness predictor is proposed that predicts the difficulty of recognizing attributes in an image. This information is utilized to remarkably improve retrieval results by down-weighting soft biometrics with an increased chance of classification failure. Additionally, three further enhancements to the retrieval process are investigated, including model calibration based on existing literature, a novel attribute-wise error weighting mechanism to balance the attributes’ influence on retrieval results, and a new distance measure that relies on the output distributions of the attribute classifier. Meaningful generalization experiments on pedestrian attribute recognition and attribute-based person retrieval are enabled for the first time. For this purpose, the UPAR dataset is proposed, which contributes 3.3 million binary annotations to harmonize semantic attributes across four existing datasets and introduces two evaluation protocols. Moreover, a new evaluation metric is suggested that is tailored to the task of attribute-based person retrieval. This metric evaluates the overlap between query attributes and the attributes of the retrieved samples to obtain scores that are consistent with the human perception of a person retrieval ranking. Combining the proposed approaches yields substantial improvements in both pedestrian attribute recognition and attribute-based person retrieval. State-of-the-art performance is achieved concerning both tasks and existing methods from the literature are surpassed. The findings are consistent across both specialization and generalization settings and across the well-established research datasets. Finally, the entire processing pipeline, from video feeds to the resulting retrieval rankings, is outlined. This encompasses a brief discussion on the topic of multi-target multi-camera tracking.

View
No Thumbnail Available
Publication

Cognitive User Modeling for Adaptivity in Serious Games

2024 , Streicher, Alexander , Bauer, Kolja

Accurate user models that capture information such as needs and knowledge levels are a central part of adaptive e-learning systems, which is all the more important in a post-pandemic world with more individualized learning. In this article, we report on the application of a Bayesian cognitive state modeling approach to adaptive educational serious games. Adaptivity needs information on the users as control variables, e.g., high or low cognitive load. Typically, this information is encoded in user models. One approach to building user models is to use tools from cognitive sciences such as Bayesian cognitive state modeling. However, cognitive modeling tools for adaptivity are sparse and can be difficult to implement. The main research question of this work is how to apply cognitive modeling tools to serious games to control adaptivity. The contribution of this article is the concept of how to implement cognitive modeling for adaptive serious games. Our approach makes use of standardized Experience API (xAPI) tracking data to facilitate applicability. We investigate how to compute quantitative measures of user performance to control adaptive responses. The implemented system has been evaluated in a user study with a serious game for image interpretation. The study results show a moderate correlation between self-assessed and computed variables.

View
No Thumbnail Available
Publication

Bridging the Gap Between IDS and Industry 4.0 - Lessons Learned and Recommendations for the Future

2024 , Alexopoulos, Kosmas , Bakopoulos, Emmanouil , Larrinaga Barrenechea, Felix , Castellvi, Silvia , Firouzi, Farshad , Luca, Gabriele de , Maló, Pedro , Marguglio, Angelo , Meléndez, Francisco , Meyer, Tom , Orio, Giovanni di , Pethig, Florian , Ruíz, Jesús , Treichel, Tagline , Usländer, Thomas , Volz, Friedrich , Watson, Kym , Stojanovic, Ljiljana

The Plattform Industrie 4.0 (PI4.0) and the International Data Spaces Association (IDSA) are two independent, parallel initiatives with clear focuses. While PI4.0 addresses communication and interaction between networked assets in a smart factory and/or supply chain across an asset or product lifecycle, IDSA is about a secure, sovereign system of data sharing in which all stakeholders can realize the full value of their data. Since data sharing between companies requires both interoperability and data sovereignty, the question emerges regarding the feasibility and rationality of integrating the expertise of PI4.0 and IDSA. The IDS-Industrial Community (IDS-I) is an extension of IDSA whose goal is to strengthen the cooperation between IDSA and PI4.0. Two fields of expertise could be combined: The Platform's know-how in the area of Industrie 4.0 (I4.0) and the IDSA's expertise in the areas of data sharing ecosystems and data sovereignty. In order to realize this vision, many aspects have to be taken into account, as there are discrepancies on multiple levels. Specifically, at the reference architecture level, we have the RAMI4.0 model on the PI4.0 side and the IDS Reference Architecture Model (IDS-RAM) on the IDSA side. While the existing I4.0 and IDS specifications are incompatible e.g. in terms of models (i.e., the AAS metamodel and the IDS information model) and APIs, there is also the issue of interoperability between I4.0 and IDS solutions. This position paper aims to bridge the gap between IDS and PI4.0 by not only analyzing how their existing concepts, tools, etc. have been "connected" in different contexts. Rather, this position paper makes recommendations on how different technologies could be combined in a generic way, independent of the concrete implementation of IDS and/or I4.0 relevant technology components. This paper could be used by both the IDS and I4.0 communities to further improve their specifications, which are still under development. The lessons learned and feedback from the initial joint use of technology components from both areas could provide concrete guidance on necessary improvements that could further strengthen or extend the specifications. Furthermore, it could help to promote the IDS architecture and specifications in the industrial production and smart manufacturing community and extend typical PI4.0 use cases to include data sovereignty by incorporating IDS aspects.

View
No Thumbnail Available
Publication

Fortschrittsbericht zur Digitalisierung des Energiesystems

2024 , Bergsträßer, Jonathan , Berkhout, Volker , Klaiber, Stefan , Klobasa, Marian , Kohrs, Robert , Linnartz, Philipp , Naumann, Steffi , Nicolai, Steffen , Schmidt, Dietrich , Welisch, Marijke , Wende-von Berg, Sebastian , Wickert, Manuel , Wille-Haußmann, Bernhard

Die Digitalisierung ist ein hochgradig relevanter Schlüsselprozess für die Energiesystemtransformation - Details dazu hat der Fraunhofer Exzellenzcluster CINES im Jahr 2022 erforscht und in 14 Thesen zusammengefasst. In 2023 wurden in Zusammenarbeit mit Praxispartner:innen aus der Energiewirtschaft politische und regulatorischen Änderungen in der digitalen Energiewirtschaft analysiert und ausgewertet. Der Fortschrittsbericht zeigt die Fortschritte der Digitalisierung auf und erörtert Handlungsbedarfe und Weiterentwicklungspotenziale. Wichtige Erkenntnisse sind: Positive Fortschritte gibt es u.a. durch gesetzliche und regulatorische Neuerungen wie beispielsweise beim §14a EnWG oder dem Gesetz zum Neustart der Digitalisierung der Energiewende (GNDEW). Es mangelt an einem integrativen Zielbild - ein solches kann für mehr Klarheit und Orientierung bei der Digitalisierung des Energiesystems in Deutschland und Europa sorgen. Um Lücken, Handlungsbedarfe und positiven Fortschritt besser zu erkennen, erfordert es ein gemeinsames Verständnis für die Orientierung und Ausrichtung auf die Digitalisierung. Dafür ist es unter anderem notwendig ein handlungsanleitendes Zukunftsbild zu schaffen, die kommunikative Übersetzung und die Verständlichkeit von regulatorischen Änderungen zu verbessern, Kompetenzen für technologische Lösungen, bspw. für notwendige Cyberresilienz und kritischen Infrastrukturen, aufzubauen und Investitionen zu tätigen, um mehr finanzielle Mittel für die Digitalisierung der Energiesystemtransformation zur Verfügung zu haben.

View