Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB

Filters

5 2
Reset filters

Settings
Now showing 1 - 7 of 7
  • Publication
    Validation of XAI Explanations for Multivariate Time Series Classification in the Maritime Domain
    ( 2022)
    Veerappa, Manjunatha
    ;
    Anneken, Mathias
    ;
    Burkart, Nadia
    ;
    Huber, Marco
    Due to the lack of explanation towards their internal mechanism, state-of-the-art deep learning-based classifiers are often considered as black-box models. For instance, in the maritime domain, models that classify the types of ships based on their trajectories and other features perform well, but give no further explanation for their predictions. To gain the trust of human operators responsible for critical decisions, the reason behind the classification is crucial. In this paper, we introduce explainable artificial intelligence (XAI) approaches to the task of classification of ship types. This supports decision-making by providing explanations in terms of the features contributing the most towards the prediction, along with their corresponding time intervals. In the case of the LIME explainer, we adapt the time-slice mapping technique (LimeforTime), while for Shapley additive explanations (SHAP) and path integrated gradient (PIG), we represent the relevance of each input variable to generate a heatmap as an explanation. In order to validate the XAI results, the existing perturbation and sequence analyses for classifiers of univariate time series data is employed for testing and evaluating the XAI explanations on multivariate time series. Furthermore, we introduce a novel evaluation technique to assess the quality of explanations yielded by the chosen XAI method.
  • Publication
    A Survey on the Explainability of Supervised Machine Learning
    ( 2021)
    Burkart, Nadia
    ;
    Huber, Marco F.
    Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or finance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Learning (SML). We conduct a state-of-the-art survey that reviews past and recent explainable SML approaches and classifies them according to the introduced definitions. Finally, we illustrate principles by means of an explanatory case study and discuss important future directions.
  • Publication
    Are you sure? Prediction revision in automated decision-making
    ( 2021)
    Burkart, Nadia
    ;
    Robert, Sebastian
    ;
    Huber, Marco
    With the rapid improvements in machine learning and deep learning, decisions made by automated decision support systems (DSS) will increase. Besides the accuracy of predictions, their explainability becomes more important. The algorithms can construct complex mathematical prediction models. This causes insecurity to the predictions. The insecurity rises the need for equipping the algorithms with explanations. To examine how users trust automated DSS, an experiment was conducted. Our research aim is to examine how participants supported by an DSS revise their initial prediction by four varying approaches (treatments) in a between-subject design study. The four treatments differ in the degree of explainability to understand the predictions of the system. First we used an interpretable regression model, second a Random Forest (considered to be a black box [BB]), third the BB with a local explanation and last the BB with a global explanation. We noticed that all participants improved their predictions after receiving an advice whether it was a complete BB or an BB with an explanation. The major finding was that interpretable models were not incorporated more in the decision process than BB models or BB models with explanations.
  • Publication
    Supported Decision-Making by Explainable Predictions of Ship Trajectories
    ( 2021)
    Burkart, Nadia
    ;
    Huber, Marco
    ;
    Anneken, Mathias
    Machine Learning and Deep Learning models make accurate predictions based on a specifically trained task. For instance, models that classify ship vessel types based on their trajectory and other features. This can support human experts while they try to obtain information on the ships, e.g., to control illegal fishing. Besides the support in predicting a certain ship type, there is a need to explain the decision-making behind the classification. For example, which features contributed the most to the classification of the ship type. This paper introduces existing explanation approaches to the task of ship classification. The underlying model is based on a Residual Neural Network. The model was trained on an AIS data set. Further, we illustrate the explainability approaches by means of an explanatory case study and conduct a first experiment with a human expert.
  • Publication
    Explanation Framework for Intrusion Detection
    ( 2021)
    Burkart, Nadia
    ;
    Franz, Maximilian
    ;
    Huber, Marco F.
    Machine learning and deep learning are widely used in various applications to assist or even replace human reasoning. For instance, a machine learning based intrusion detection system (IDS) monitors a network for malicious activity or specific policy violations. We propose that IDSs should attach a sufficiently understandable report to each alert to allow the operator to review them more efficiently. This work aims at complementing an IDS by means of a framework to create explanations. The explanations support the human operator in understanding alerts and reveal potential false positives. The focus lies on counterfactual instances and explanations based on locally faithful decision-boundaries.
  • Publication
    Batch-wise Regularization of Deep Neural Networks for Interpretability
    ( 2020)
    Burkart, Nadia
    ;
    Faller, Philipp M.
    ;
    Peinsipp, Elisabeth
    ;
    Huber, Marco
    Fast progress in the field of Machine Learning and Deep Learning strongly influences the research in many application domains like autonomous driving or health care. In this paper, we propose a batch-wise regularization technique to enhance the interpretability for deep neural networks (NN) by means of a global surrogate rule list. For this purpose, we introduce a novel regularization approach that yields a differentiable penalty term. Compared to other regularization approaches, our approach avoids repeated creating of surrogate models during training of the NN. The experiments show that the proposed approach has a high fidelity to the main model and also results in interpretable and more accurate models compared to some of the baselines.
  • Publication
    Forcing Interpretability for Deep Neural Networks through Rule-based Regularization
    ( 2019)
    Burkart, Nadia
    ;
    Huber, Marco
    ;
    Faller, Philipp M.
    Remarkable progress in the field of machine learning strongly drives the research in many application domains. For some domains, it is mandatory that the output of machine learning algorithms needs to be interpretable. In this paper, we propose a rule-based regularization technique to enforce interpretability for neural networks (NN). For this purpose, we train a rule-based surrogate model simultaneously with the NN. From the surrogate, a metric quantifying its degree of explainability is derived and fed back to the training of the NN as a regularization term. We evaluate our model on four datasets and compare it to unregularized models as well as a decision tree (DT) based baseline. The rule-based regularization approach achieves interpretability and competitive accuracy.