Fraunhofer-Institut für Angewandte und Integrierte Sicherheit AISEC

Filters

19
14 14 14
19
Reset filters

Settings
Now showing 1 - 10 of 19
  • Publication
    DA3G: Detecting Adversarial Attacks by Analysing Gradients
    ( 2021)
    Schulze, J.-P.
    ;
    Sperl, P.
    ;
    Böttinger, K.
    Deep learning models are vulnerable to specifically crafted inputs, called adversarial examples. In this paper, we present DA3G, a novel method to reliably detect evasion attacks on neural networks. We analyse the behaviour of the network under test on the given input sample. Compared to the benign training data, adversarial examples cause a discrepancy between visual and causal perception. Although visually close to a benign input class, the output is shifted at the attacker's will. DA3G detects these changes in the pattern of the gradient using an auxiliary neural network. Our end-to-end approach readily integrates with a variety of existing architectures. DA3G reliably detects known as well as unknown attacks and increases the difficulty of adaptive attacks.
  • Publication
    Adversarial Vulnerability of Active Transfer Learning
    ( 2021)
    Müller, N.M.
    ;
    Böttinger, K.
    Two widely used techniques for training supervised machine learning models on small datasets are Active Learning and Transfer Learning. The former helps to optimally use a limited budget to label new data. The latter uses large pre-trained models as feature extractors and enables the design of complex, non-linear models even on tiny datasets. Combining these two approaches is an effective, state-of-the-art method when dealing with small datasets. In this paper, we share an intriguing observation: Namely, that the combination of these techniques is particularly susceptible to a new kind of data poisoning attack: By adding small adversarial noise on the input, it is possible to create a collision in the output space of the transfer learner. As a result, Active Learning algorithms no longer select the optimal instances, but almost exclusively the ones injected by the attacker. This allows an attacker to manipulate the active learner to select and include arbitrary images into the data set, even against an overwhelming majority of unpoisoned samples. We show that a model trained on such a poisoned dataset has a significantly deteriorated performance, dropping from 86% to 34% test accuracy. We evaluate this attack on both audio and image datasets and support our findings empirically. To the best of our knowledge, this weakness has not been described before in literature.
  • Publication
    Towards Resistant Audio Adversarial Examples
    ( 2020)
    Dörr, T.
    ;
    Markert, K.
    ;
    Müller, N.M.
    ;
    Böttinger, K.
    Adversarial examples tremendously threaten the availability and integrity of machine learning-based systems. While the feasibility of such attacks has been observed first in the domain of image processing, recent research shows that speech recognition is also susceptible to adversarial attacks. However, reliably bridging the air gap (i.e., making the adversarial examples work when recorded via a microphone) has so far eluded researchers. We find that due to flaws in the generation process, state-of-the-art adversarial example generation methods cause overfitting because of the binning operation in the target speech recognition system (e.g., Mozilla Deepspeech). We devise an approach to mitigate this flaw and find that our method improves generation of adversarial examples with varying offsets. We confirm the significant improvement with our approach by empirical comparison of the edit distance in a realistic over-the-air setting. Our approach states a significant step towards over-the-air attacks. We publish the code and an applicable implementation of our approach.
  • Publication
    Data Poisoning Attacks on Regression Learning and Corresponding Defenses
    ( 2020)
    Müller, N.
    ;
    Kowatsch, D.
    ;
    Böttinger, K.
    Adversarial data poisoning is an effective attack against machine learning and threatens model integrity by introducing poisoned data into the training dataset. So far, it has been studied mostly for classification, even though regression learning is used in many mission critical systems (such as dosage of medication, control of cyber-physical systems and managing power supply). Therefore, in the present research, we aim to evaluate all aspects of data poisoning attacks on regression learning, exceeding previous work both in terms of breadth and depth. We present realistic scenarios in which data poisoning attacks threaten production systems and introduce a novel black-box attack, which is then applied to a real-word medical use-case. As a result, we observe that the mean squared error (MSE) of the regressor increases to 150 percent due to inserting only two percent of poison samples. Finally, we present a new defense strategy against the novel and previous attacks and evaluate it thoroughly on 26 datasets. As a result of the conducted experiments, we conclude that the proposed defence strategy effectively mitigates the considered attacks.
  • Publication
    Distributed anomaly detection of single mote attacks in RPL networks
    ( 2019)
    Müller, N.M.
    ;
    Debus, P.
    ;
    Kowatsch, D.
    ;
    Böttinger, K.
    RPL, a protocol for IP packet routing in wireless sensor networks, is known to be susceptible to a wide range of attacks. Especially effective are 'single mote attacks', where the attacker only needs to control a single sensor node. These attacks work by initiating a 'delayed denial of service', which depletes the motes' batteries while maintaining otherwise normal network operation. While active, this is not detectable on the application layer, and thus requires detection on the network layer. Further requirements for detection algorithms are extreme computational and resource efficiency (e.g. avoiding communication overhead) and the use of machine learning (if the drawbacks of signature based detection are not acceptable). In this paper, we present a system for anomaly detection of these kinds of attacks and constraints, implement a prototype in C, and evaluate it on different network topologies against three 'single mote attacks'. We make our system highly resource and energy effic ient by deploying pre-trained models to the motes and approximating our choice of ML algorithm (KDE) via parameterized cubic splines. We achieve on average 84.91 percent true-positives and less than 0.5 percent false-positives. We publish all data sets and source code for full reproducibility.
  • Publication
    A Unified Architecture for Industrial IoT Security Requirements in Open Platform Communications
    ( 2019)
    Hansch, G.
    ;
    Schneider, P.
    ;
    Fischer, K.
    ;
    Böttinger, K.
    We present a unified communication architecture for security requirements in the industrial internet of things. Formulating security requirements in the language of OPC UA provides a unified method to communicate and compare security requirements within a heavily heterogeneous landscape of machines in the field. Our machine-readable data model provides a fully automatable approach for security requirement communication within the rapidly evolving fourth industrial revolution, which is characterized by high-grade interconnection of industrial infrastructures and self-configuring production systems. Capturing security requirements in an OPC UA compliant and unified data model for industrial control systems enables strong use cases within modern production plants and future supply chains. We implement our data model as well as an OPC UA server that operates on this model to show the feasibility of our approach. Further, we deploy and evaluate our framework within a reference project realized by 14 industrial partners and 7 research facilities within Germany.
  • Publication
    On GDPR Compliance of Companies' Privacy Policies
    ( 2019)
    Müller, N.M.
    ;
    Kowatsch, D.
    ;
    Debus, P.
    ;
    Mirdita, D.
    ;
    Böttinger, K.
    We introduce a data set of privacy policies containing more than 18,300 sentence snippets, labeled in accordance to five General Data Protection Regulation (GDPR) privacy policy core requirements. We hope that this data set will enable practitioners to analyze and detect policy compliance with the GDPR legislation in various documents. In order to evaluate our data set, we apply a number of NLP and other classification algorithms and achieve an F1 score between 0.52 and 0.71 across the five requirements. We apply our trained models to over 1200 real privacy policies which we crawled from companies' websites, and find that over 76% do not contain all of the requirements, thus potentially not fully complying with GDPR.
  • Publication
  • Publication
    Deep Reinforcement Fuzzing
    ( 2018)
    Böttinger, K.
    ;
    Godefroid, P.
    ;
    Singh, R.
    Fuzzing is the process of finding security vulnerabilities in input-processing code by repeatedly testing the code with modified inputs. In this paper, we formalize fuzzing as a reinforcement learning problem using the concept of Markov decision processes. This in turn allows us to apply state-of-the-art deep Q -learning algorithms that optimize rewards, which we define from runtime properties of the program under test. By observing the rewards caused by mutating with a specific set of actions performed on an initial program input, the fuzzing agent learns a policy that can next generate new higher-reward inputs. We have implemented this new approach, and preliminary empirical evidence shows that reinforcement fuzzing can outperform baseline random fuzzing.
  • Publication
    Deep Reinforcement Fuzzing
    ( 2018)
    Böttinger, K.
    ;
    Godefroid, P.
    ;
    Singh, R.
    Fuzzing is the process of finding security vulnerabilities in input-processing code by repeatedly testing the code with modified inputs. In this paper, we formalize fuzzing as a reinforcement learning problem using the concept of Markov decision processes. This in turn allows us to apply state-of-the-art deep Q -learning algorithms that optimize rewards, which we define from runtime properties of the program under test. By observing the rewards caused by mutating with a specific set of actions performed on an initial program input, the fuzzing agent learns a policy that can next generate new higher-reward inputs. We have implemented this new approach, and preliminary empirical evidence shows that reinforcement fuzzing can outperform baseline random fuzzing.