Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

Reset filters

Settings
Now showing 1 - 10 of 2668
  • Publication
    Informed Machine Learning - A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems
    ( 2023)
    Rueden, Laura von
    ;
    Mayer, Sebastian
    ;
    Beckh, Katharina
    ;
    Georgiev, Bogdan
    ;
    Giesselbach, Sven
    ;
    Heese, Raoul
    ;
    Kirsch, Birgit
    ;
    Walczak, Michal
    ;
    Pfrommer, Julius
    ;
    Pick, Annika
    ;
    Ramamurthy, Rajkumar
    ;
    Garcke, Jochen
    ;
    Bauckhage, Christian
    ;
    Schuecker, Jannis
    Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for informed machine learning which illustrates its building blocks and distinguishes it from conventional machine learning. We introduce a taxonomy that serves as a classification framework for informed machine learning approaches. It considers the source of knowledge, its representation, and its integration into the machine learning pipeline. Based on this taxonomy, we survey related research and describe how different knowledge representations such as algebraic equations, logic rules, or simulation results can be used in learning systems. This evaluation of numerous papers on the basis of our taxonomy uncovers key methods in the field of informed machine learning.
  • Publication
    Feasibility of artificial intelligence-supported assessment of bone marrow infiltration using dual-energy computed tomography in patients with evidence of monoclonal protein - a retrospective observational study
    ( 2022)
    Fervers, P.
    ;
    Fervers, Florian
    ;
    Lohneis, P.
    ;
    Pollman-Schweckhors, P.
    ;
    Zaytoun, H.
    ;
    Rinneburger, M.
    ;
    Maintz, D.
    ;
    Große Hokamp, N.
    Objectives To demonstrate the feasibility of an automated, non-invasive approach to estimate bone marrow (BM) infiltration of multiple myeloma (MM) by dual-energy computed tomography (DECT) after virtual non-calcium (VNCa) post-processing. Methods Individuals with MM and monoclonal gammopathy of unknown significance (MGUS) with concurrent DECT and BM biopsy between May 2018 and July 2020 were included in this retrospective observational study. Two pathologists and three radiologists reported BM infiltration and presence of osteolytic bone lesions, respectively. Bone mineral density (BMD) was quantified CT-based by a CE-certified software. Automated spine segmentation was implemented by a pre-trained convolutional neural network. The non-fatty portion of BM was defined as voxels > 0 HU in VNCa. For statistical assessment, multivariate regression and receiver operating characteristic (ROC) were conducted. Results Thirty-five patients (mean age 65 ± 12 years; 18 female) were evaluated. The non-fatty portion of BM significantly predicted BM infiltration after adjusting for the covariable BMD (p = 0.007, r = 0.46). A non-fatty portion of BM > 0.93% could anticipate osteolytic lesions and the clinical diagnosis of MM with an area under the ROC curve of 0.70 [0.49-0.90] and 0.71 [0.54-0.89], respectively. Our approach identified MM-patients without osteolytic lesions on conventional CT with a sensitivity and specificity of 0.63 and 0.71, respectively. Conclusions Automated, AI-supported attenuation assessment of the spine in DECT VNCa is feasible to predict BM infiltration in MM. Further, the proposed method might allow for pre-selecting patients with higher pre-test probability of osteolytic bone lesions and support the clinical diagnosis of MM without pathognomonic lesions on conventional CT.
  • Publication
    In memoriam Fernando Puente Leon
    ( 2022)
    Heizmann, M.
    ;
    Beyerer, J.
  • Publication
    Towards a Better Understanding of Machine Learning based Network Intrusion Detection Systems in Industrial Networks
    ( 2022)
    Borcherding, Anne
    ;
    Feldmann, Lukas
    ;
    Karch, Markus
    ;
    Meshram, Ankush
    ;
    Beyerer, Jürgen
    It is crucial in an industrial network to understand how and why a intrusion detection system detects, classifies, and reports intrusions. With the ongoing introduction of machine learning into the research area of intrusion detection, this understanding gets even more important since the used systems often appear as a black-box for the user and are no longer understandable in an intuitive and comprehensible way. We propose a novel approach to understand the internal characteristics of a machine learning based network intrusion detection system. This approach includes methods to understand which data sources the system uses, to evaluate whether the system uses linear or non-linear classification approaches, and to find out which underlying machine learning model is implemented in the system. Our evaluation on two publicly available industrial datasets shows that the detection of the data source and the differentiation between linear and non-linear models is possible with our approach. In addition, the identification of the underlying machine learning model can be accomplished with statistical significance for non-linear models. The information made accessible by our approach helps to develop a deeper understanding of the functioning of a network intrusion detection system, and contributes towards developing transparent machine learning based intrusion detection approaches.
  • Publication
    Siamese recurrent neural networks for the robust classification of grid disturbances in transmission power systems considering unknown events
    ( 2022)
    Kummerow, A.
    ;
    Monsalve, C.
    ;
    Bretschneider, P.
    The automated identification and localisation of grid disturbances is a major research area and key technology for the monitoring and control of future power systems. Current recognition systems rely on sufficient training data and are very error-prone to disturbance events, which are unseen during training. This study introduces a robust Siamese recurrent neural network using attention-based embedding functions to simultaneously identify and locate disturbances from synchrophasor data. Additionally, a novel double-sigmoid classifier is introduced for reliable differentiation between known and unknown disturbance types and locations. Different models are evaluated within an open-set classification problem for a generic power transmission system considering different unknown disturbance events. A detailed analysis of the results is provided and classification results are compared with a state-of-the-art open-set classifier.
  • Publication
    Composition and Symmetries - Computational Analysis of Fine-Art Aesthetics
    ( 2022)
    Zhuravleva, Olga A.
    ;
    Komarov, Andrei V.
    ;
    Zherdev, Denis A.
    ;
    Savkhalova, Natalie B.
    ;
    Demina, Anna L.
    ;
    Michaelsen, Eckart
    ;
    Nikonorov, Artem V.
    ;
    Nesterov, Alexander Y.
    This article deals with the problem of quantitative research of the aesthetic content of the fine-art object. The paper states that a fine-art object is a conceptually formed sequence of signs, and its composition is a structural form, that can be measured using mathematical models. The main approach is based on the perception of the formal order as a determinant of the aesthetic category of beauty. The composition of the image is directly related to the formation of aesthetic sensations and values, since it performs the function of controlling the viewer's perception of a work of art. The research is based on the studies of computational aesthetics by G. D. Birkhoff and M. Bense, as well as the studies of the receptive aesthetics of R. Ingarden, W. Iser, H. R. Jauss and Ya. Mukarzhovsky. The computational aesthetics methods, such as CNN-based object detectors, and gestalt-based symmetry analysis, are used to detect symmetry axes in fine-art images. Experimental analysis demonstrates that the applied computational approach is consistent with the philosophical analysis and the expert evaluations of the fine-art images, therefore it allows to obtain more detailed fine-art paintings description.
  • Publication
    MotorFactory: A Blender Add-on for Large Dataset Generation of Small Electric Motors
    ( 2022)
    Wu, Chengzhi
    ;
    Zhou, Kanran
    ;
    Kaiser, Jan-Philipp
    ;
    Mitschke, Norbert
    ;
    Klein, Jan-Felix
    ;
    Pfrommer, Julius
    ;
    Beyerer, Jürgen
    ;
    Lanza, Gisela
    ;
    Heizmann, Michael
    ;
    Furmans, Kai
    To enable automatic disassembly of different product types with uncertain condition and degree of wear in remanufacturing, agile production systems that can adapt dynamically to changing requirements are needed. Machine learning algorithms can be employed due to their generalization capabilities of learning from various types and variants of products. However, in reality, datasets with a diversity of samples that can be used to train models are difficult to obtain in the initial period. This may cause bad performances when the system tries to adapt to new unseen input data in the future. In order to generate large datasets for different learning purposes, in our project, we present a Blender add-on named MotorFactory to generate customized mesh models of various motor instances. MotorFactory allows to create mesh models which, complemented with additional add-ons, can be further used to create synthetic RGB images, depth images, normal images, segmentation ground truth masks and 3D point cloud datasets with point-wise semantic labels. The created synthetic datasets may be used for various tasks including motor type classification, object detection for decentralized material transfer tasks, part segmentation for disassembly and handling tasks, or even reinforcement learning-based robotics control or view-planning.
  • Publication
    Improving Semantic Image Segmentation via Label Fusion in Semantically Textured Meshes
    ( 2022)
    Fervers, Florian
    ;
    Breuer, Timo
    ;
    Stachowiak, Gregor
    ;
    Bullinger, Sebastian
    ;
    Bodensteiner, Christoph
    ;
    Arens, Michael
    Models for semantic segmentation require a large amount of hand-labeled training data which is costly and time-consuming to produce. For this purpose, we present a label fusion framework that is capable of improving semantic pixel labels of video sequences in an unsupervised manner. We make use of a 3D mesh representation of the environment and fuse the predictions of different frames into a consistent representation using semantic mesh textures. Rendering the semantic mesh using the original intrinsic and extrinsic camera parameters yields a set of improved semantic segmentation images. Due to our optimized CUDA implementation, we are able to exploit the entire c-dimensional probability distribution of annotations over c classes in an uncertainty-aware manner. We evaluate our method on the Scannet dataset where we improve annotations produced by the state-of-the-art segmentation network ESANet from 52.05% to 58.25% pixel accuracy. We publish the source code of our framework online to foster future research in this area (https://github.com/fferflo/semantic-meshes). To the best of our knowledge, this is the first publicly available label fusion framework for semantic image segmentation based on meshes with semantic textures.
  • Publication
    Fast and Lightweight Online Person Search for Large-Scale Surveillance Systems
    ( 2022)
    Specker, Andreas
    ;
    Moritz, Lennart
    ;
    Cormier, Mickael
    ;
    Beyerer, Jürgen
    The demand for methods for video analysis in the fieldof surveillance technology is rapidly growing due to theincreasing amount of surveillance footage available. Intelligent methods for surveillance software offer numerouspossibilities to support police investigations and crime prevention. This includes the integration of video processingpipelines for tasks such as detection of graffiti, suspiciousluggage, or intruders. Another important surveillance taskis the semi-automated search for specific persons-of-interestwithin a camera network. In this work, we identify the major obstacles for the development of person search systemsas the real-time processing capability on affordable hardware and the performance gap of person detection and reidentification methods on unseen target domain data. Inaddition, we demonstrate the current potential of intelligentonline person search by developing a real-world, largescale surveillance system. An extensive evaluation is provided for person detection, tracking, and re-identificationcomponents on affordable hardware setups, for which thewhole system achieves real-time processing up to 76 FPS.
  • Publication
    Where are we with Human Pose Estimation in Real-World Surveillance?
    ( 2022)
    Cormier, Mickael
    ;
    Clepe, Aris
    ;
    Specker, Andreas
    ;
    Beyerer, Jürgen
    The rapidly increasing number of surveillance cameras offers a variety of opportunities for intelligent video analytics to improve public safety. Among many others, the automatic recognition of suspicious and violent behavior poses a key task. To preserve personal privacy, prevent ethnic bias, and reduce complexity, most approaches first extract the pose or skeleton of persons and subsequently perform activity recognition. However, current literature mainly focuses on research datasets and does not consider real-world challenges and requirements of human pose estimation. We close this gap by analyzing these challenges, such as inadequate data and the need for real-time processing, and proposing a framework for human pose estimation in uncontrolled crowded surveillance scenarios. Our system integrates mitigation measures as well as a tracking component to incorporate temporal information. Finally, we provide a detailed quantitative and qualitative analysis on both a scientific and a real-world dataset to highlight improvements and remaining obstacles towards robust real-world human pose estimation in uncooperative scenarios.