Prof. Dr.-Ing.

Bauckhage, Christian

Filters

5 2
Reset filters

Settings
Now showing 1 - 7 of 7
  • Publication
    Gradient Flows for L2 Support Vector Machine Training
    ( 2022-08-08)
    Bauckhage, Christian
    ;
    Schneider, Helen
    ;
    Wulff, Benjamin
    ;
    Sifa, Rafet
    We explore the merits of training of support vector machines for binary classification by means of solving systems of ordinary differential equations. We thus assume a continuous time perspective on a machine learning problem which may be of interest for implementations on (re)emerging hardware platforms such as analog- or quantum computers.
  • Publication
    Informed Pre-Training on Prior Knowledge
    ( 2022-05-23T16:24:40Z)
    Rüden, Laura von
    ;
    Houben, Sebastian
    ;
    Cvejoski, Kostadin
    ;
    Bauckhage, Christian
    ;
    Piatkowski, Nico
    When training data is scarce, the incorporation of additional prior knowledge can assist the learning process. While it is common to initialize neural networks with weights that have been pre-trained on other large data sets, pre-training on more concise forms of knowledge has rather been overlooked. In this paper, we propose a novel informed machine learning approach and suggest to pre-train on prior knowledge. Formal knowledge representations, e.g. graphs or equations, are first transformed into a small and condensed data set of knowledge prototypes. We show that informed pre-training on such knowledge prototypes (i) speeds up the learning processes, (ii) improves generalization capabilities in the regime where not enough training data is available, and (iii) increases model robustness. Analyzing which parts of the model are affected most by the prototypes reveals that improvements come from deeper layers that typically represent high-level features. This confirms that informed pre-training can indeed transfer semantic knowledge. This is a novel effect, which shows that knowledge-based pre-training has additional and complementary strengths to existing approaches.
  • Publication
    Informed Machine Learning - A Taxonomy and Survey of Integrating Knowledge into Learning Systems
    ( 2019-03-29)
    Rüden, Laura von
    ;
    Mayer, Sebastian
    ;
    Beckh, Katharina
    ;
    Georgiev, Bogdan
    ;
    Giesselbach, Sven
    ;
    Heese, Raoul
    ;
    Kirsch, Birgit
    ;
    Pfrommer, Julius
    ;
    Pick, Annika
    ;
    Ramamurthy, Rajkumar
    ;
    Schuecker, Jannis
    ;
    Garcke, Jochen
    ;
    Bauckhage, Christian
    ;
    Walczak, Michal
    Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process, which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. First, we provide a definition and propose a concept for informed machine learning, which illustrates its building blocks and distinguishes it from conventional machine learning. Second, we introduce a taxonomy that serves as a classification framework for informed machine learning approaches. It considers the source of knowledge, its representation, and its integration into the machine learning pipeline. Third, we survey related research and describe how different knowledge representations such as algebraic equations, logic rules, or simulation results can be used in learning systems. This evaluation of numerous papers on the basis of our taxonomy uncovers key methods in the field of informed machine learning.
  • Publication
    Recurrent Adversarial Service Times
    ( 2019)
    Ojeda, César
    ;
    Cvejosky, Kostadin
    ;
    Sánchez, Ramsés J.
    ;
    Schücker, Jannis
    ;
    Georgiev, Bogdan
    ;
    Bauckhage, Christian
    Service system dynamics occur at the interplay between customer behaviour and a service provider's response. This kind of dynamics can effectively be modeled within the framework of queuing theory where customers' arrivals are described by point process models. However, these approaches are limited by parametric assumptions as to, for example, inter-event time distributions. In this paper, we address these limitations and propose a novel, deep neural network solution to the queuing problem. Our solution combines a recurrent neural network that models the arrival process with a recurrent generative adversarial network which models the service time distribution. We evaluate our methodology on various empirical datasets ranging from internet services (Blockchain, GitHub, Stackoverflow) to mobility service systems (New York taxi cab).
  • Publication
    Neural conditional gradients
    ( 2018)
    Schramowski, Patrick
    ;
    Bauckhage, Christian
    ;
    Kersting, Kristian
    The move from hand-designed to learned optimizers in machine learning has been quite successful for gradient-based and -free optimizers. When facing a constrained problem, however, maintaining feasibility typically requires a projection step, which might be computationally expensive and not differentiable. We show how the design of projection-free convex optimization algorithms can be cast as a learning problem based on Frank-Wolfe Networks: recurrent networks implementing the Frank-Wolfe algorithm aka. conditional gradients. This allows them to learn to exploit structure when, e.g., optimizing over rank-1 matrices. Our LSTM-learned optimizers outperform hand-designed as well learned but unconstrained ones. We demonstrate this for training support vector machines and softmax classifiers.
  • Publication
    Maximum Entropy Models of Shortest Path and Outbreak Distributions in Networks
    ( 2015)
    Bauckhage, Christian
    ;
    Kersting, Kristian
    ;
    Hadiji, Fabian
    Properties of networks are often characterized in terms of features such as node degree distributions, average path lengths, diameters, or clustering coefficients. Here, we study shortest path length distributions. On the one hand, average as well as maximum distances can be determined therefrom; on the other hand, they are closely related to the dynamics of network spreading processes. Because of the combinatorial nature of networks, we apply maximum entropy arguments to derive a general, physically plausible model. In particular, we establish the generalized Gamma distribution as a continuous characterization of shortest path length histograms of networks or arbitrary topology. Experimental evaluations corroborate our theoretical results.
  • Publication
    A Comparison of Methods for Player Clustering via Behavioral Telemetry
    ( 2014)
    Drachen, Anders
    ;
    Thurau, Christian
    ;
    Sifa, Rafet
    ;
    Bauckhage, Christian
    The analysis of user behavior in digital games has been aided by the introduction of user telemetry in game development, which provides unprecedented access to quantitative data on user behavior from the installed game clients of the entire population of players. Player behavior telemetry datasets can be exceptionally complex, with features recorded for a varying population of users over a temporal segment that can reach years in duration. Categorization of behaviors, whether through descriptive methods (e.g. segmention) or unsupervised/supervised learning techniques, is valuable for finding patterns in the behavioral data, and developing profiles that are actionable to game developers. There are numerous methods for unsupervised clustering of user behavior, e.g. k-means/c-means, Non-negative Matrix Factorization, or Principal Component Analysis. Although all yield behavior categorizations, interpretation of the resulting categories in terms of actual play behavior can be difficult if not impossible. In this paper, a range of unsupervised techniques are applied together with Archetypal Analysis to develop behavioral clusters from playtime data of 70,014 World of Warcraft players, covering a five year interval. The techniques are evaluated with respect to their ability to develop actionable behavioral profiles from the dataset.