Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

19
Reset filters

Settings
Now showing 1 - 10 of 19
  • Publication
    Simulation study and experimental validation of a neural network-based predictive tracking system for sensor-based sorting
    ( 2023)
    Maier, Georg
    ;
    Reith-Braun, Marcel
    ;
    Bauer, Albert
    ;
    Gruna, Robin
    ;
    Pfaff, Florian
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Hanebeck, Uwe D.
    ;
    Beyerer, Jürgen
    Sensor-based sorting offers cutting-edge solutions for separating granular materials. The line-scanning sensors currently in use in such systems only produce a single observation of each object and no data on its movement. According to recent studies, using an area-scan camera has the potential to reduce both characterization and separation error in a sorting process. A predictive tracking approach based on Kalman filters makes it possible to estimate the followed paths and parametrize a unique motion model for each object using a multiobject tracking system. While earlier studies concentrated on physically-motivated motion models, it has been demonstrated that novel machine learning techniques produce predictions that are more accurate. In this paper, we describe the creation of a predictive tracking system based on neural networks. The new algorithm is applied to an experimental sorting system and to a numerical model of the sorter. Although the new approach does not yet fully reach the achieved sorting quality of the existing approaches, it allows the use of the general method without requiring expert knowledge or a fundamental understanding of the parameterization of the particle motion model.
  • Publication
    Detecting Tar Contaminated Samples in Road-rubble using Hyperspectral Imaging and Texture Analysis
    ( 2023)
    Bäcker, Paul
    ;
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Polycyclic aromatic hydrocarbons (PAH) containing tar-mixtures pose a challenge for recycling road rubble, as the tar containing elements have to be extracted and decontaminated for recycling. In this preliminary study, tar, bitumen and minerals are discriminated using a combination of color (RGB) and Hyperspectral Short Wave Infrared (SWIR) cameras. Further, the use of an autoencoder for detecting minerals embedded inside tar- and bitumen mixtures is proposed. Features are extracted from the spectra of the SWIR camera and the texture of the RGB images. For classification, linear discriminant analysis combined with a k-nearest neighbor classification is used. First results show a reliable detection of minerals and positive signs for separability of tar and bitumen. This work is a foundation for developing a sensor-based sorting system for physical separation of tar contaminated samples in road rubble.
  • Publication
    Machine learning-based multiobject tracking for sensor-based sorting
    ( 2022)
    Maier, Georg
    ;
    Reith-Braun, Marcel
    ;
    Bauer, Albert
    ;
    Gruna, Robin
    ;
    Pfaff, Florian
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Hanebeck, Uwe
    ;
    Beyerer, Jürgen
  • Publication
    An Extended Modular Processing Pipeline for Event-Based Vision in Automatic Visual Inspection
    ( 2021)
    Beck, Moritz
    ;
    Maier, Georg
    ;
    Flitter, Merle
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Heizmann, Michael
    ;
    Beyerer, Jürgen
    Dynamic Vision Sensors differ from conventional cameras in that only intensity changes of individual pixels are perceived and transmitted as an asynchronous stream instead of an entire frame. The technology promises, among other things, high temporal resolution and low latencies and data rates. While such sensors currently enjoy much scientific attention, there are only little publications on practical applications. One field of application that has hardly been considered so far, yet potentially fits well with the sensor principle due to its special properties, is automatic visual inspection. In this paper, we evaluate current state-of-the-art processing algorithms in this new application domain. We further propose an algorithmic approach for the identification of ideal time windows within an event stream for object classification. For the evaluation of our method, we acquire two novel datasets that contain typical visual inspection scenarios, i.e., the inspection of objects on a conveyor belt and during free fall. The success of our algorithmic extension for data processing is demonstrated on the basis of these new datasets by showing that classification accuracy of current algorithms is highly increased. By making our new datasets publicly available, we intend to stimulate further research on application of Dynamic Vision Sensors in machine vision applications.
  • Publication
    Predictive tracking with improved motion models for optical belt sorting
    ( 2020)
    Pfaff, Florian
    ;
    Pieper, Christoph
    ;
    Maier, Georg
    ;
    Noack, Benjamin
    ;
    Gruna, Robin
    ;
    Kruggel-Emden, Harald
    ;
    Hanebeck, Uwe D.
    ;
    Wirtz, Siegmar
    ;
    Scherer, Viktor
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Optical belt sorters are a versatile means to sort bulk materials. In previous work, we presented a novel design of an optical belt sorter, which includes an area scan camera instead of a line scan camera. Line scan cameras, which are well-established in optical belt sorting, only allow for a single observation of each particle. Using multitarget tracking, the data of the area scan camera can be used to derive a part of the trajectory of each particle. The knowledge of the trajectories can be used to generate accurate predictions as to when and where each particle passes the separation mechanism. Accurate predictions are key to achieve high quality sorting results. The accuracy of the trajectories and the predictions heavily depends on the motion model used. In an evaluation based on a simulation that provides us with ground truth trajectories, we previously identified a bias in the temporal component of the prediction. In this paper, we analyze the simulation-based ground truth data of the motion of different bulk materials and derive models specifically tailored to the generation of accurate predictions for particles traveling on a conveyor belt. The derived models are evaluated using simulation data involving three different bulk materials. The evaluation shows that the constant velocity model and constant acceleration model can be outperformed by utilizing the similarities in the motion behavior of particles of the same type.
  • Publication
    Characterizing material flow in sensor-based sorting systems using an instrumented particle
    ( 2020)
    Maier, Georg
    ;
    Pfaff, Florian
    ;
    Bittner, Andrea
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    ;
    Noack, Benjamin
    ;
    Kruggel-Emden, Harald
    ;
    Hanebeck, Uwe D.
    Sensor-based sorting is a well-established single particle separation technology. It has found wide application as a quality assurance and control approach in food processing, mining, and recycling. In order to assure high sorting quality, a high degree of control of the motion of individual particles contained in the material stream is required. Several system designs, which are tailored to a sorting task at hand, exist. However, the suitability of a design for a sorting task is assessed by empirical observation. The required thorough experimentation is very time consuming and labor intensive. In this paper, we propose an instrumented bulk material particle for the characterization of motion behavior of the material stream in sensor-based sorting systems. We present a hardware setup including a 9-axis absolute orientation sensor that is used for data acquisition on an experimental sorting system. The presented results show that further processing of this data yields meaningful features of the motion behavior. As an example, we acquire and process data from an experimental sorting system consisting of several submodules such as vibrating conveyor channels and a chute. It is shown that the data can be used to train a model which enables predicting the submodule of a sorting system from which an unknown data sample originates. To our best knowledge, this is the first time that this IIoT-based approach has been applied for the characterization of material flow properties in sensor-based sorting.
  • Publication
    Automatic visual inspection based on trajectory data
    ( 2019)
    Maier, Georg
    ;
    Mürdter, N.
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Automatic inspection tasks have successfully been implemented in several industrial fields and are of growing importance. Visual inspection using optical sensors is wide spread due to the vast variety of different sensors, observable features and comparatively low prices. It seems obvious that corresponding systems are blind towards mechanical features and inspection of those typically requires highly specialized, inflexible and costly systems. Recently, we have shown in the context of sensor-based sorting that tracking objects over a time period allows deriving motion-based features which potentially enable discrimination of optically identical objects, although an optical sensor is used. In this paper, we take one step back from the specific application and study the classification of test objects based on their trajectories. The objects are observed while receiving a certain impulse. We further refrain from manually designing features but use raw coordinates as extracted from a series of images. The success of the method is demonstrated by discriminating spheres made of similar plastic types while bouncing off a plane.
  • Publication
    Real-time multitarget tracking for sensor-based sorting: A new implementation of the auction algorithm for graphics processing units
    ( 2019)
    Maier, Georg
    ;
    Pfaff, F.
    ;
    Wagner, M.
    ;
    Pieper, C.
    ;
    Gruna, Robin
    ;
    Noack, B.
    ;
    Kruggel-Emden, H.
    ;
    Längle, Thomas
    ;
    Hanebeck, U.D.
    ;
    Scherer, V.
    ;
    Beyerer, Jürgen
    Utilizing parallel algorithms is an established way of increasing performance in systems that are bound to real-time restrictions. Sensor-based sorting is a machine vision application for which firm real-time requirements need to be respected in order to reliably remove potentially harmful entities from a material feed. Recently, employing a predictive tracking approach using multitarget tracking in order to decrease the error in the physical separation in optical sorting has been proposed. For implementations that use hard associations between measurements and tracks, a linear assignment problem has to be solved for each frame recorded by a camera. The auction algorithm can be utilized for this purpose, which also has the advantage of being well suited for parallel architectures. In this paper, an improved implementation of this algorithm for a graphics processing unit (GPU) is presented. The resulting algorithm is implemented in both an OpenCL and a CUDA based environment. By using an optimized data structure, the presented algorithm outperforms recently proposed implementations in terms of speed while retaining the quality of output of the algorithm. Furthermore, memory requirements are significantly decreased, which is important for embedded systems. Experimental results are provided for two different GPUs and six datasets. It is shown that the proposed approach is of particular interest for applications dealing with comparatively large problem sizes.
  • Publication
    Application of area-scan sensors in sensor-based sorting
    ( 2018)
    Maier, Georg
    ;
    Pfaff, F.
    ;
    Pieper, C.
    ;
    Gruna, Robin
    ;
    Noack, B.
    ;
    Kruggel-Emden, H.
    ;
    Längle, Thomas
    ;
    Hanebeck, U.D.
    ;
    Wirtz, S.
    ;
    Scherer, V.
    ;
    Beyerer, Jürgen
    In the field of machine vision, sensor-based sorting is an important real-time application that enables the separation of a material feed into different classes. While state-of-the-art systems utilize scanning sensors such as line-scan cameras, advances in sensor technology have made application of area scanning sensors feasible. Provided a sufficiently high frame rate, objects can be observed at multiple points in time. By applying multiobject tracking, information about the objects contained in the material stream can be fused over time. Based on this information, our approach further allows predicting the position of each object for future points in time. While conventional systems typically apply a global, rather simple motion model, our approach includes an individual motion model for each object, which in turn allows estimating the point in time as well as the position when reaching the separation stage. In this contribution, we present results from our collaborative research project and summarize the present advances by discussing the potential of the application of area-scan sensors for sensor-based sorting. Among others, we introduce our simulation-driven approach and present results for physical separation efficiency for simulation-generated data, demonstrate the potential of using motion-based features for material classification and discuss real-time related challenges.
  • Publication
    Evaluierung und Vergleich von NIR-Multiprodukt-Kalibrierungsverfahren zur Brix-Bestimmung
    ( 2018)
    Kopf, M.
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Nahinfrarotspektroskopie ist eine etablierte Methode zur Qualitätsbestimmung von Obst und Gemüse. Neue Anwendungsgebiete, wie z. B. die mobile Lebensmittelanalyse mittels handgetragener und preisgünstiger Mikrospektrometer, verlangen nach neuen Ansätzen zur Multiprodukt-Kalibrierung. Zur produktspezifischen Kalibrierung existieren bereits geeignete Methoden wie partial least squares regression (PLSR). Der Versuch von Micklander et al. zeigt jedoch auf, dass die Multiprodukt-Kalibrierung noch eine ungelöste Herausforderung darstellt. Nichtlineare Ansätze wie neuronale Netze und lokale Regression erzielten hier bessere Ergebnisse als konventionelle Methoden wie PLSR. Vorläufige Untersuchungen zur Multiprodukt-Kalibrierung zur quantitativen Analyse von Lebensmitteln mittels NIR Spektroskopie lieferten vielversprechende Ergebnisse durch Memory-Based Learning (MBL) und Classification-Prediction-Hierarchy (CPH). In dieser Arbeit werden drei Ansätze zur Multiprodukt-Kalibrierung untersucht. Hierzu werden drei unterschiedliche Apfelsorten, Birnen und Tomaten mit bekanntem Zuckergehalt (in ○Brix) mittels bildgebender NIR Spektroskopie im Bereich von 900 nm bis 2400 nm analysiert. Die Genauigkeit eines linearen PLSR-Modells und zweier nichtlinearer Modelle (CPH und MBL) sowie unterschiedliche Vorverarbeitungsmethoden werden untersucht und evaluiert. Zur Bestimmung von Fehlermaßen dienen Leave-One-Out- und Leave-One-Product-Out-Kreuzvalidierungen.