Prof. Dr.-Ing.

Bauckhage, Christian

Filters

Reset filters

Settings
Now showing 1 - 10 of 55
  • Publication
    Anonymization of German financial documents using neural network-based language models with contextual word representations
    ( 2022-03)
    Biesner, David
    ;
    Ramamurthy, Rajkumar
    ;
    Loitz, Rüdiger
    ;
    Lübbering, Max
    ;
    Hillebrand, Lars Patrick
    ;
    Ladi, Anna
    ;
    Stenzel, Robin
    ;
    Pielka, Maren
    ;
    Bauckhage, Christian
    ;
    Sifa, Rafet
    The automatization and digitalization of business processes have led to an increase in the need for efficient information extraction from business documents. However, financial and legal documents are often not utilized effectively by text processing or machine learning systems, partly due to the presence of sensitive information in these documents, which restrict their usage beyond authorized parties and purposes. To overcome this limitation, we develop an anonymization method for German financial and legal documents using state-of-the-art natural language processing methods based on recurrent neural nets and transformer architectures. We present a web-based application to anonymize financial documents and a large-scale evaluation of different deep learning techniques.
  • Publication
    An Optimization for Convolutional Network Layers Using the Viola-Jones Framework and Ternary Weight Networks
    ( 2021-12-09)
    Agombar, Rhys
    ;
    Bauckhage, Christian
    ;
    Lübbering, Max
    ;
    Sifa, Rafet
    Neural networks have the potential to be extremely powerful for computer vision related tasks, but can be computationally expensive. Classical methods, by comparison, tend to be relatively light weight, albeit not as powerful. In this paper, we propose a method of combining parts from a classical system, called the Viola-Jones Object Detection Framework, with a modern ternary neural network to improve the efficiency of a convolutional neural net by replacing convolutional filters with a set of custom ones inspired by the framework. This reduces the number of operations needed for computing feature values with negligible effects on overall accuracy, allowing for a more optimized network.
  • Publication
    Decoupling Autoencoders for Robust One-vs-Rest Classification
    ( 2021-10-20)
    Lübbering, Max
    ;
    Gebauer, Michael
    ;
    Ramamurthy, Rajkumar
    ;
    Bauckhage, Christian
    ;
    Sifa, Rafet
    One-vs-Rest (OVR) classification aims to distinguish a single class of interest from other classes. The concept of novelty detection and robustness to dataset shift becomes crucial in OVR when the scope of the rest class extends from the classes observed during training to unseen and possibly unrelated classes. In this work, we propose a novel architecture, namely Decoupling Autoencoder (DAE) to tackle the common issue of robustness w.r.t. out-of-distribution samples which is prevalent in classifiers such as multi-layer perceptrons (MLP) and ensemble architectures. Experiments on plain classification, outlier detection, and dataset shift tasks show DAE to achieve robust performance across these tasks compared to the baselines, which tend to fail completely, when exposed to dataset shift. W hile DAE and the baselines yield rather uncalibrated predictions on the outlier detection and dataset shift task, we found that DAE calibration is more stable across all tasks. Therefore, calibration measures applied to the classification task could also improve the calibration of the outlier detection and dataset shift scenarios for DAE.
  • Publication
    ALiBERT: Improved automated list inspection (ALI) with BERT
    ( 2021-08-16)
    Ramamurthy, Rajkumar
    ;
    Pielka, Maren
    ;
    Stenzel, Marc Robin
    ;
    Bauckhage, Christian
    ;
    Sifa, Rafet
    ;
    Khameneh, Tim Dilmaghani
    ;
    Warning, Ulrich
    ;
    Kliem, Bernd
    ;
    Loitz, Rüdiger
    We consider Automated List Inspection (ALI), a content-based text recommendation system that assists auditors in matching relevant text passages from notes in financial statements to specific law regulations. ALI follows a ranking paradigm in which a fixed number of requirements per textual passage are shown to the user. Despite achieving impressive ranking performance, the user experience can still be improved by showing a dynamic number of recommendations. Besides, existing models rely on a feature-based language model that needs to be pre-trained on a large corpus of domain-specific datasets. Moreover, they cannot be trained in an end-to-end fashion by jointly optimizing with language model parameters. In this work, we alleviate these concerns by considering a multi-label classification approach that predicts dynamic requirement sequences. We base our model on pre-trained BERT that allows us to fine-tune the whole model in an end-to-end fashion, thereby avoiding the need for training a language representation model. We conclude by presenting a detailed evaluation of the proposed model on two German financial datasets.
  • Publication
    Utilizing Representation Learning for Robust Text Classification Under Datasetshift
    ( 2021)
    Lübbering, Max
    ;
    Gebauer, Michael
    ;
    Ramamurthy, Rajkumar
    ;
    Pielka, Maren
    ;
    Bauckhage, Christian
    ;
    Sifa, Rafet
    Within One-vs-Rest (OVR) classification, a classifier differentiates a single class of interest (COI) from the rest, i.e. any other class. By extending the scope of the rest class to corruptions (dataset shift), aspects of outlier detection gain relevancy. In this work, we show that adversarially trained autoencoders (ATA) representative of autoencoder-based outlier detection methods, yield tremendous robustness improvements over traditional neural network methods such as multi-layer perceptrons (MLP) and common ensemble methods, while maintaining a competitive classification performance. In contrast, our results also reveal that deep learning methods solely optimized for classification, tend to fail completely when exposed to dataset shift.
  • Publication
    Supervised autoencoder variants for end to end anomaly detection
    ( 2021)
    Lübbering, Max
    ;
    Gebauer, Michael
    ;
    Ramamurthy, Rajkumar
    ;
    Sifa, Rafet
    ;
    Bauckhage, Christian
    Despite the success of deep learning in various domains such as natural language processing, speech recognition, and computer vision, learning from a limited amount of samples and generalizing to unseen data still pose challenges. Notably, in the tasks of outlier detection and imbalanced dataset classification, the label of interest is either scarce or its distribution is skewed, causing aggravated generalization problems. In this work, we pursue the direction of multi-task learning, specifically the idea of using supervised autoencoders (SAE), which allows us to combine unsupervised and supervised objectives in an end to end fashion. We extend this approach by introducing an adversarial supervised objective to enrich the representations which are learned for the classification task. We conduct thorough experiments on a broad range of tasks, including outlier detection, novelty detection, and imbalanced classification, and study the efficacy of our method against standard baselines using autoencoders. Our work empirically shows that the SAE methods outperform one class autoencoders, adversarially trained autoencoders and multi layer perceptrons in terms of AUPR score comparison. Additionally, our analysis of the obtained representations suggests that the adversarial reconstruction loss functions enforce the encodings to separate into class-specific clusters, which was not observed for non-adversarial reconstruction loss functions.
  • Publication
    tanh Neurons are Bayesian Decision Makers
    ( 2021)
    Bauckhage, Christian
    ;
    Sifa, Rafet
    ;
    Hecker, Dirk
    The hyperbolic tangent (tanh) is a traditional choice for the activation function of the neurons of an artificial neural network. Here, we go through a simple calculation that shows that this modeling choice is linked to Bayesian decision theory. Our brief, tutorial-like discussion is intended as a reference to an observation rarely mentioned in standard textbooks.
  • Publication
    Tackling Contradiction Detection in German Using Machine Translation and End-to-End Recurrent Neural Networks
    ( 2021)
    Pielka, Maren
    ;
    Sifa, Rafet
    ;
    Hillebrand, Lars Patrick
    ;
    Biesner, David
    ;
    Ramamurthy, Rajkumar
    ;
    Ladi, Anna
    ;
    Bauckhage, Christian
    Natural Language Inference, and specifically Contradiction Detection, is still an unexplored topic with respect to German text. In this paper, we apply Recurrent Neural Network (RNN) methods to learn contradiction-specific sentence embeddings. Our data set for evaluation is a machine-translated version of the Stanford Natural Language Inference (SNLI) corpus. The results are compared to a baseline using unsupervised vectorization techniques, namely tf-idf and Flair, as well as state-of-the art transformer-based (MBERT) methods. We find that the end-to-end models outperform the models trained on unsupervised embeddings, which makes them the better choice in an empirical use case. The RNN methods also perform superior to MBERT on the translated data set.
  • Publication
    Towards Intelligent Food Waste Prevention: An Approach Using Scalable and Flexible Harvest Schedule Optimization with Evolutionary Algorithms
    ( 2021)
    Günder, Maurice
    ;
    Piatkowski, Nico
    ;
    Rüden, Laura von
    ;
    Sifa, Rafet
    ;
    Bauckhage, Christian
    In times of climate change, growing world population, and the resulting scarcity of resources, efficient and economical usage of agricultural land is increasingly important and challenging at the same time. To avoid disadvantages of monocropping for soil and environment, it is advisable to practice intercropping of various plant species whenever possible. However, intercropping is challenging as it requires a balanced planting schedule due to individual cultivation time frames. Maintaining a continuous harvest throughout the season is important as it reduces logistical costs and related greenhouse gas emissions, and can also help to reduce food waste. Motivated by the prevention of food waste, this work proposes a flexible optimization method for a full harvest season of large crop ensembles that complies with given economical and environmental constraints. Our approach applies evolutionary algorithms and we further combine our evolution strategy with a sophisticated hierarchical loss function and adaptive mutation rate. We thus transfer the multi-objective into a pseudo-single-objective optimization problem, for which we obtain faster and better solutions than those of conventional approaches.
  • Publication
    Toxicity Detection in Online Comments with Limited Data: A Comparative Analysis
    ( 2021)
    Lübbering, Max
    ;
    Pielka, Maren
    ;
    Das, Kajaree
    ;
    Gebauer, Michael
    ;
    Ramamurthy, Rajkumar
    ;
    Bauckhage, Christian
    ;
    Sifa, Rafet
    We present a comparative study on toxicity detection, focusing on the problem of identifying toxicity types of low prevalence and possibly even unobserved at training time. For this purpose, we train our models on a dataset that contains only a weak type of toxicity, and test whether they are able to generalize to more severe toxicity types. We find that representation learning and ensembling exceed the classification performance of simple classifiers on toxicity detection, while also providing significantly better generalization and robustness. All models benefit from a larger training set size, which even extends to the toxicity types unseen during training.