• English
  • Deutsch
  • Log In
    Password Login
    or
  • Research Outputs
  • Projects
  • Researchers
  • Institutes
  • Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Artikel
  4. Double Deep Q-Learning with Prioritized Experience Replay for Anomaly Detection in Smart Environments
 
  • Details
  • Full
Options
2022
Journal Article
Titel

Double Deep Q-Learning with Prioritized Experience Replay for Anomaly Detection in Smart Environments

Abstract
Anomaly detection in smart environments is important when dealing with rare events, which can be safety-critical to individuals or infrastructure. Safety-critical means in this case, that these events can be a threat to the safety of individuals (e.g. a person falling to the ground) or to the security of infrastructure (e.g. unauthorized access to protected facilities). However, recognizing abnormal events in smart environments is challenging, because of the complex and volatile nature of the data recorded by monitoring sensors. Methodologies proposed in the literature are frequently domain-specific and are subject to biased assumptions about the underlying data. In this work, we propose the adaption of a deep reinforcement learning algorithm, namely double deep q-learning (DDQN), for anomaly detection in smart environments. Our proposed anomaly detector directly learns a decision-making function, which can classify rare events based on multivariate sequential time series data. With an emphasis on improving the performance in rare event classification tasks, we extended the algorithm with a prioritized experience replay (PER) strategy, and showed that the PER extension yields an increase in detection performance. The adaption of the improved version of the DDQN reinforcement learning algorithm for anomaly detection in smart environments is the major contribution of this work. Empirical studies on publicly available real-world datasets demonstrate the effectiveness of our proposed solution. Here specifically, we use a dataset for fall and for occupancy detection to evaluate the solution proposed in this work. Our solution yields comparable detection performance to previous work, and has the additional advantages of being adaptable to different environments and capable of online learning.
Author(s)
Fährmann, Daniel
Fraunhofer-Institut fĂĽr Graphische Datenverarbeitung IGD
Jorek, Nils
Frankfurt Univ. of Applied Sciences
Damer, Naser
Fraunhofer-Institut fĂĽr Graphische Datenverarbeitung IGD
Kirchbuchner, Florian orcid-logo
Fraunhofer-Institut fĂĽr Graphische Datenverarbeitung IGD
Kuijper, Arjan orcid-logo
Fraunhofer-Institut fĂĽr Graphische Datenverarbeitung IGD
Zeitschrift
IEEE access
Project(s)
Secure Urban Infrastructures
Funder
Bundesministerium fĂĽr Bildung und Forschung -BMBF-
DOI
10.1109/ACCESS.2022.3179720
File(s)
Double_Deep_Q-Learning_With_Prioritized_Experience_Replay_for_Anomaly_Detection_in_Smart_Environments.pdf (1.48 MB)
Language
English
google-scholar
Fraunhofer-Institut fĂĽr Graphische Datenverarbeitung IGD
Tags
  • Lead Topic: Digitized Work

  • Lead Topic: Individual Health

  • Lead Topic: Smart City

  • Smart environments

  • Smart home

  • Machine learning

  • ATHENE

  • CRISP

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Send Feedback
© 2022