Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters
Reset filters

Settings
Now showing 1 - 10 of 103
  • Publication
    6D Pose Estimation on Point Cloud Data through Prior Knowledge Integration: A Case Study in Autonomous Disassembly
    ( 2024)
    Wu, Chengzhi
    ;
    Fu, Hao
    ;
    Kaiser, Jan-Philipp
    ;
    Tabuchi Barczak, Erik
    ;
    Pfrommer, Julius
    ;
    Lanza, Gisela
    ;
    Heizmann, Michael
    ;
    Beyerer, Jürgen
    The accurate estimation of 6D pose remains a challenging task within the computer vision domain, even when utilizing 3D point cloud data. Conversely, in the manufacturing domain, instances arise where leveraging prior knowledge can yield advancements in this endeavor. This study focuses on the disassembly of starter motors to augment the engineering of product life cycles. A pivotal objective in this context involves the identification and 6D pose estimation of bolts affixed to the motors, facilitating automated disassembly within the manufacturing workflow. Complicating matters, the presence of occlusions and the limitations of single-view data acquisition, notably when motors are placed in a clamping system, obscure certain portions and render some bolts imperceptible. Consequently, the development of a comprehensive pipeline capable of acquiring complete bolt information is imperative to avoid oversight in bolt detection. In this paper, employing the task of bolt detection within the scope of our project as a pertinent use case, we introduce a meticulously devised pipeline. This multi-stage pipeline effectively captures the 6D information with regard to all bolts on the motor, thereby showcasing the effective utilization of prior knowledge in handling this challenging task. The proposed methodology not only contributes to the field of 6D pose estimation but also underscores the viability of integrating domain-specific insights to tackle complex problems in manufacturing and automation.
  • Publication
    A survey of the state of the art in sensor-based sorting technology and research
    ( 2024)
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.
  • Publication
    Herrn Prof. Franz Mesch zum 90. Geburtstag
    ( 2023)
    Bretthauer, Georg
    ;
    Beyerer, Jürgen
    ;
    Stiller, Christoph
  • Publication
    Retroreflex ellipsometry for isotropic three-phase systems with nonplanar surfaces
    ( 2023)
    Chen, Chia Wei
    ;
    Hartrumpf, Matthias
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Ellipsometry is a long-established and highly-accurate method for characterizing materials and thin films in the semiconductor, optical coating and biology industry. However, the geometric shape of samples is usually limited to flat samples for conventional ellipsometers because the operation must fulfill the condition of the law of reflection. The retroreflex ellipsometer developed at Fraunhofer IOSB can overcome the shape constraints without realignment of the sample but information on surface inclinations is still necessary. In this work, the polarization model of nonplanar surfaces is analyzed by polarization ray tracing and a numerical inverse method is proposed to calculate the angle of incidence for isotropic three-phase systems with known refractive indices and ellipsometric parameters (Ψ,Δ). The experiments show excellent accuracy for the measurements of the incident and tilt angles and prove that the method is capable of measuring the film thickness of nonplanar surfaces.
  • Publication
    Finding optimal decision boundaries for human intervention in one-class machine-learning models for industrial inspection
    ( 2023)
    Zander, Tim
    ;
    Pan, Ziyan
    ;
    Birnstill, Pascal
    ;
    Beyerer, Jürgen
    Anomaly detection with machine learning in industrial inspection systems for manufactured products relies on labelled data. This raises the question of how the labelling by humans should be conducted. Moreover, such a system will most likely always be imperfect and potentially need a human fall-back mechanism for ambiguous cases. We consider the case where we want to optimise the cost of the combined inspection process done by humans together with a pre-trained algorithm. This gives improved combined performance and increases the knowledge of the performance of the pre-trained model. We focus on so-called one-class classification problems which produce a continuous outlier score. After establishing some initial setup mechanisms ranging from using prior knowledge to calibrated models, we then define some cost model for machine inspection with a possible second inspection of the sample done by a human. Further, we discuss in this cost model how to select two optimal boundaries of the outlier score, where in between these two boundaries human inspection takes place. Finally, we frame this established knowledge into an applicable algorithm and conduct some experiments for the validity of the model.
  • Publication
    Simulation study and experimental validation of a neural network-based predictive tracking system for sensor-based sorting
    ( 2023)
    Maier, Georg
    ;
    Reith-Braun, Marcel
    ;
    Bauer, Albert
    ;
    Gruna, Robin
    ;
    Pfaff, Florian
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Hanebeck, Uwe D.
    ;
    Beyerer, Jürgen
    Sensor-based sorting offers cutting-edge solutions for separating granular materials. The line-scanning sensors currently in use in such systems only produce a single observation of each object and no data on its movement. According to recent studies, using an area-scan camera has the potential to reduce both characterization and separation error in a sorting process. A predictive tracking approach based on Kalman filters makes it possible to estimate the followed paths and parametrize a unique motion model for each object using a multiobject tracking system. While earlier studies concentrated on physically-motivated motion models, it has been demonstrated that novel machine learning techniques produce predictions that are more accurate. In this paper, we describe the creation of a predictive tracking system based on neural networks. The new algorithm is applied to an experimental sorting system and to a numerical model of the sorter. Although the new approach does not yet fully reach the achieved sorting quality of the existing approaches, it allows the use of the general method without requiring expert knowledge or a fundamental understanding of the parameterization of the particle motion model.
  • Publication
    Deflectometry for specular surfaces: an overview
    ( 2023)
    Burke, Jan
    ;
    Pak, Alexey
    ;
    Höfer, Sebastian
    ;
    Ziebarth, Mathias
    ;
    Roschani, Masoud
    ;
    Beyerer, Jürgen
    Deflectometry as a technique to assess reflective surfaces has now existed for some 40 years. Its different aspects and variations have been studied in multiple theses and research articles; reviews are available for certain subtopics. Still a field of active development with many unsolved problems, deflectometry now encompasses a large variety of application domains, hardware setup types, and processing workflows for different purposes, and spans a range from qualitative defect inspection of large vehicles to precision measurements of microscopic optics. Over these years, many exciting developments have accumulated in the underlying theory, in the systems design, and in the implementation specifics. This diversity of topics is difficult to grasp for experts and non-experts alike and may present an obstacle to a wider acceptance of deflectometry as a useful tool for research and industrial applications. This paper presents an attempt to summarize the status of deflectometry and to map relations between its notable branches. Its aim is to provide a communication basis for experienced practitioners and also to offer a convenient entry point for those interested in learning about the method. The list of references introduces some prominent trends and established research groups in order to facilitate further self-directed exploration.
  • Publication
    Self-Supervised Generative-Contrastive Learning of Multi-Modal Euclidean Input for 3D Shape Latent Representations: A Dynamic Switching Approach
    ( 2023)
    Wu, Chengzhi
    ;
    Pfrommer, Julius
    ;
    Zhou, Mingyuan
    ;
    Beyerer, Jürgen
    We propose a combined generative and contrastive neural architecture for learning latent representations of 3D volumetric shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape. The main idea is to combine a contrastive loss between the resulting latent representations with an additional reconstruction loss. That helps to avoid collapsing the latent representations as a trivial solution for minimizing the contrastive loss. A novel dynamic switching approach is used to cross-train two encoders with a shared decoder. The switching approach also enables the stop gradient operation on a random branch. Further classification experiments show that the latent representations learned with our self-supervised method integrate more useful information from the additional input data implicitly, thus leading to better reconstruction and classification performance.
  • Publication
    GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs
    ( 2023)
    Reith-Braun, Marcel
    ;
    Bauer, Albert
    ;
    Staab, Maximilian
    ;
    Pfaff, Florian
    ;
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    ;
    Kruggel-Emden, Harald
    ;
    Hanebeck, Uwe D.
    Optical sorters separate particles of different classes by first detecting them while they are transported, e.g., on a conveyor belt, and subsequently bursting out particles of undesired classes using compressed air nozzles. Currently, the most promising results are achieved by predictive tracking, a multitarget tracking approach based on extracted midpoints from area-scan camera images that analyzes the particles’ motion and activates the nozzles accordingly. However, predictive tracking requires expert knowledge for setup and preceding object detection. Moreover, particle shapes are only considered implicitly, and the need to solve an association problem rises the computational complexity of the algorithm. In this paper, we present GridSort, an image-based approach that forecasts the scene at the nozzle array using a convolutional long short-term memory neural network and subsequently extracts nozzle activations, thus circumventing the aforementioned weaknesses. We show how GridSort can be trained in an unsupervised fashion and evaluate it using a coupled discrete element–computational fluid dynamics simulation of an optical sorter. We compare our method with predictive tracking in terms of sorting accuracy and demonstrate that it is an easy-to-apply alternative while achieving state-of-the-art results.
  • Publication
    An ontology for remanufacturing systems
    ( 2022)
    Pfrommer, Julius
    ;
    Klein, Jan-Felix
    ;
    Wurster, Marco
    ;
    Rapp, Simon
    ;
    Grauberger, Patric
    ;
    Lanza, Gisela
    ;
    Albers, Albert
    ;
    Matthiesen, Sven
    ;
    Beyerer, Jürgen
    A fully automated remanufacturing system needs to be very adaptive in order to adjust to different product variants and the condition of received products. To allow for such adaptive behavior across different subsystems and levels from the classical automation hierarchy, a common and uniform understanding regarding the core concepts that constitute a remanufacturing system is required. This article presents an ontology for the core concepts in an automated remanufacturing system.