• English
  • Deutsch
  • Log In
    or
  • Research Outputs
  • Projects
  • Researchers
  • Institutes
  • Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Artikel
  4. PocketNet: Extreme Lightweight Face Recognition Network Using Neural Architecture Search and Multistep Knowledge Distillation
 
  • Details
  • Full
Options
2022
Journal Article
Titel

PocketNet: Extreme Lightweight Face Recognition Network Using Neural Architecture Search and Multistep Knowledge Distillation

Abstract
Deep neural networks have rapidly become the mainstream method for face recognition (FR). However, this limits the deployment of such models that contain an extremely large number of parameters to embedded and low-end devices. In this work, we present an extremely lightweight and accurate FR solution, namely PocketNet. We utilize neural architecture search to develop a new family of lightweight face-specific architectures. We additionally propose a novel training paradigm based on knowledge distillation (KD), the multi-step KD, where the knowledge is distilled from the teacher model to the student model at different stages of the training maturity. We conduct a detailed ablation study proving both, the sanity of using NAS for the specific task of FR rather than general object classification, and the benefits of our proposed multi-step KD. We present an extensive experimental evaluation and comparisons with the state-of-the-art (SOTA) compact FR models on nine different benchmarks including large-scale evaluation benchmarks such as IJB-C and MegaFace. PocketNets have consistently advanced the SOTA FR performance on nine mainstream benchmarks when considering the same level of model compactness. With 0.92M parameters, our smallest network PocketNetS-128 achieved very competitive results to recent SOTA compacted models that contain up to 4M parameters. Training codes and pre-trained models are public.
Author(s)
Boutros, Fadi
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Siebke, Patrick
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Klemt, Marcel
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Damer, Naser
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Kirchbuchner, Florian
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Kuijper, Arjan
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Zeitschrift
IEEE access
Project(s)
Next Generation Biometric Systems
Funder
Bundesministerium für Bildung und Forschung -BMBF-
Thumbnail Image
DOI
10.1109/ACCESS.2022.3170561
Language
English
google-scholar
Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Tags
  • Lead Topic: Digitized...

  • Lead Topic: Smart Cit...

  • Lead Topic: Visual Co...

  • Research Line: Comput...

  • Research Line: Human ...

  • Research Line: Machin...

  • Biometrics

  • Machine learning

  • Face recognition

  • Deep learning

  • ATHENE

  • CRISP

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Send Feedback
© 2022