Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

Reset filters

Settings
Now showing 1 - 10 of 69
  • Publication
    A survey of the state of the art in sensor-based sorting technology and research
    ( 2024)
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.
  • Publication
    Particle-Specific Deflection Windows for Optical Sorting by Uncertainty Quantification
    ( 2024)
    Reith-Braun, Marcel
    ;
    Liang, Kevin
    ;
    Pfaff, Florian
    ;
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Bauer, Albert
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    ;
    Hanebeck, Uwe D.
    In current state of the art sensor-based sorting systems, the length of the deflection windows, i.e., the period of nozzle activation and the number of nozzles to be activated, is commonly determined solely by the size of the particles. However, this comes at the cost of the sorting process not accounting for model discrepancies between actual and presumed particle motion, as well as for situations where the available information does not allow for precise determination of nozzle activations. To achieve a desired sorting accuracy, in practice, one is therefore usually forced to enlarge the deflection window to a certain degree, which increases the number of falsely co-deflected particles and compressed air consumption. In this paper, we propose incorporating the uncertainty of the prediction of particle motion of each individual particle into the determination of the deflection windows. The method is based on the predictive tracking approach for optical sorting, which tracks the particles while they move toward the nozzle array based on images of an area-scan camera. Given the extracted motion information from the tracking, we propose an approximation for the distribution of arrival time and location of the particle at the nozzle array assuming nearly constant-velocity or nearly constantacceleration particle motion behavior. By evaluating the quantile function of both distributions, we obtain a confidence interval for the arrival time and location based on prediction uncertainty, which we then combine with the particle size to form the final deflection window. We apply our method to a real sorting task using a pilot-scale chute sorter. Our results obtained from extensive sorting trials show that sorting accuracies can be remarkably improved compared with state-of-the-art industrial sorters and enhanced even further compared with predictive tracking while having the potential to reduce compressed air consumption.
  • Publication
    Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning
    ( 2024)
    Kronenwett, Felix
    ;
    Maier, Georg
    ;
    Leiss, Norbert
    ;
    Gruna, Robin
    ;
    Thome, Volker
    ;
    Längle, Thomas
    Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.
  • Publication
    Detecting Tar Contaminated Samples in Road-rubble using Hyperspectral Imaging and Texture Analysis
    ( 2023)
    Bäcker, Paul
    ;
    Maier, Georg
    ;
    Gruna, Robin
    ;
    Längle, Thomas
    ;
    Beyerer, Jürgen
    Polycyclic aromatic hydrocarbons (PAH) containing tar-mixtures pose a challenge for recycling road rubble, as the tar containing elements have to be extracted and decontaminated for recycling. In this preliminary study, tar, bitumen and minerals are discriminated using a combination of color (RGB) and Hyperspectral Short Wave Infrared (SWIR) cameras. Further, the use of an autoencoder for detecting minerals embedded inside tar- and bitumen mixtures is proposed. Features are extracted from the spectra of the SWIR camera and the texture of the RGB images. For classification, linear discriminant analysis combined with a k-nearest neighbor classification is used. First results show a reliable detection of minerals and positive signs for separability of tar and bitumen. This work is a foundation for developing a sensor-based sorting system for physical separation of tar contaminated samples in road rubble.
  • Publication
    Multi-sensor data fusion using deep learning for bulky waste image classification
    ( 2023)
    Bihler, Manuel
    ;
    Roming, Lukas
    ;
    Jiang, Yifan
    ;
    Afifi, Ahmed J.
    ;
    Aderhold, Jochen
    ;
    Cibiraite-Lukenskiene, Dovile
    ;
    Lorenz, Sandra
    ;
    Gloaguen, Richard
    ;
    Gruna, Robin
    ;
    Heizmann, Michael
    Deep learning techniques are commonly utilized to tackle various computer vision problems, including recognition, segmentation, and classification from RGB images. With the availability of a diverse range of sensors, industry-specific datasets are acquired to address specific challenges. These collected datasets have varied modalities, indicating that the images possess distinct channel numbers and pixel values that have different interpretations. Implementing deep learning methods to attain optimal outcomes on such multimodal data is a complicated procedure. To enhance the performance of classification tasks in this scenario, one feasible approach is to employ a data fusion technique. Data fusion aims to use all the available information from all sensors and integrate them to obtain an optimal outcome. This paper investigates early fusion, intermediate fusion, and late fusion in deep learning models for bulky waste image classification. For training and evaluation of the models, a multimodal dataset is used. The dataset consists of RGB, hyperspectral Near Infrared (NIR), Thermography, and Terahertz images of bulky waste. The results of this work show that multimodal sensor fusion can enhance classification accuracy compared to a single-sensor approach for the used dataset. Hereby, late fusion performed the best with an accuracy of 0.921 compared to intermediate and early fusion, on our test data.
  • Publication
    Increasing the reuse of wood in bulky waste using artificial intelligence and imaging in the VIS, IR, and terahertz ranges
    ( 2023)
    Roming, Lukas
    ;
    Gruna, Robin
    ;
    Aderhold, Jochen
    ;
    Schlüter, Friedrich
    ;
    Cibiraite-Lukenskiene, Dovile
    ;
    Gundacker, Dominik
    ;
    Friedrich, Fabian
    ;
    Bihler, Manuel
    ;
    Heizmann, Michael
    Bulky waste contains valuable raw materials, especially wood, which accounts for around 50% of the volume. Sorting is very time-consuming in view of the volume and variety of bulky waste and is often still done manually. Therefore, only about half of the available wood is used as a material, while the rest is burned with unsorted waste. In order to improve the material recycling of wood from bulky waste, the project ASKIVIT aims to develop a solution for the automated sorting of bulky waste. For that, a multi-sensor approach is proposed including: (i) Conventional imaging in the visible spectral range; (ii) Nearinfrared hyperspectral imaging; (iii) Active heat flow thermography; (iv) Terahertz imaging. This paper presents a demonstrator used to obtain images with the aforementioned sensors. Differences between the imaging systems are discussed and promising results on common problems like painted materials or black plastic are presented. Besides that, pre-examinations show the importance of near-infrared hyperspectral imaging for the characterization of bulky waste.
  • Publication
    Simulation study and experimental validation of a neural network-based predictive tracking system for sensor-based sorting
    ( 2023)
    Maier, Georg
    ;
    Reith-Braun, Marcel
    ;
    Bauer, Albert
    ;
    Gruna, Robin
    ;
    Pfaff, Florian
    ;
    Kruggel-Emden, Harald
    ;
    Längle, Thomas
    ;
    Hanebeck, Uwe D.
    ;
    Beyerer, Jürgen
    Sensor-based sorting offers cutting-edge solutions for separating granular materials. The line-scanning sensors currently in use in such systems only produce a single observation of each object and no data on its movement. According to recent studies, using an area-scan camera has the potential to reduce both characterization and separation error in a sorting process. A predictive tracking approach based on Kalman filters makes it possible to estimate the followed paths and parametrize a unique motion model for each object using a multiobject tracking system. While earlier studies concentrated on physically-motivated motion models, it has been demonstrated that novel machine learning techniques produce predictions that are more accurate. In this paper, we describe the creation of a predictive tracking system based on neural networks. The new algorithm is applied to an experimental sorting system and to a numerical model of the sorter. Although the new approach does not yet fully reach the achieved sorting quality of the existing approaches, it allows the use of the general method without requiring expert knowledge or a fundamental understanding of the parameterization of the particle motion model.
  • Publication
    Multi-modal image acquisition for AI-based bulky waste sorting (incl. terahertz synthetic aperture radar)
    ( 2023)
    Cibiraite-Lukenskiene, Dovile
    ;
    Gundacker, Dominik
    ;
    Bihler, M.
    ;
    Heizmann, M.
    ;
    Schlüter, Friedrich
    ;
    Aderhold, Jochen
    ;
    Roming, Lukas
    ;
    Gruna, Robin
    ;
    Jonuscheit, Joachim
    ;
    Friederich, Fabian
    This work presents the results of the initial acquisition of a multi-modal dataset that will be utilized to train and test a neural network for wood sorting. The aim of the project is to improve wood recycling from bulky waste by using four complementary sensing systems: visual, infrared, terahertz, and thermography. The four systems were combined to capture 57 multi-modal images of bulky waste samples moving on the conveyor belt at a speed of 10 cm/s. Early fusion results on THz show 0.77 accuracy, whereas the best multi-modal data fusion accuracy equals 0.921.
  • Publication
    Benchmarking a DEM‐CFD Model of an Optical Belt Sorter by Experimental Comparison
    ( 2023)
    Bauer, Albert
    ;
    Maier, Georg
    ;
    Reith-Braun, Marcel
    ;
    Kruggel-Emden, Harald
    ;
    Pfaff, Florian
    ;
    Gruna, Robin
    ;
    Hanebeck, Uwe
    ;
    Längle, Thomas
    A DEM-CFD (discrete element method - computational fluid dynamics) model of an optical belt sorter was extensively compared with experiments of a laboratory-scale sorter to assess the model's accuracy. Brick and sand-lime brick were considered as materials. First, the transport characteristics on the conveyor belt, involving mass flow, lateral particle distribution and proximity, were compared. Second, sorting results were benchmarked for varying mixture proportions at differing mass flows. It was found that the numerical model is able to reproduce the experimental results with high accuracy.
  • Publication
    Development of multilevel monitoring systems for the identification of phytoplasma diseases in German viticultural areas
    ( 2023)
    Jarausch, Barbara
    ;
    Alisaac, Elias
    ;
    Schumacher, Petra
    ;
    Gauweiler, Pascal
    ;
    Gruna, Robin
    ;
    Zabawa, Laura
    ;
    Klingbeil, Lasse
    ;
    Rechkemmer, Sonja
    ;
    Jarausch, Wolfgang
    ;
    Maixner, Michael
    ;
    Kicherer, Anna
    "Flavescence dorée" (FD) and "bois noir" (BN) are the most important phytoplasma diseases of grapevine. While BN is widespread in German winegrowing regions, only one single grapevine plant infected with FD phytoplasma has been reported and eradicated in 2020. Although the main vector of FD, Scaphoideus titanus, is not present in Germany, the FD phytoplasma is classified as quarantine pest because of its epidemic potential and a systematic monitoring of the disease and the vector is mandatory in Germany. Furthermore, a non-epidemic strain Palatinate grapevine yellows (PGY), with the same symptomatology as FD and BN is present in Germany. For large scale monitoring digital multilevel monitoring systems for grapevine yellows (GY) are being developed. Reference plots with infection by either BN or PGY in risk zones in southwestern Germany and Württemberg have been visually monitored and symptomatic grapevines were analysed by molecular means. In parallel, the same samples were examined with different sensor-based methods to define a correlation between phytoplasma presence and specific spectral signatures. A spectral distinction to similar symptoms due to virus infection, insect damage, and nutritional deficiencies was investigated for the system validation.