• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Konferenzschrift
  4. AdaDistill: Adaptive Knowledge Distillation for Deep Face Recognition
 
  • Details
  • Full
Options
2025
Conference Paper
Title

AdaDistill: Adaptive Knowledge Distillation for Deep Face Recognition

Abstract
Knowledge distillation (KD) aims at improving the performance of a compact student model by distilling the knowledge from a high-performing teacher model. In this paper, we present an adaptive KD approach, namely AdaDistill, for deep face recognition. The proposed AdaDistill embeds the KD concept into the softmax loss by training the student using a margin penalty softmax loss with distilled class centers from the teacher. Being aware of the relatively low capacity of the compact student model, we propose to distill less complex knowledge at an early stage of training and more complex one at a later stage of training. This relative adjustment of the distilled knowledge is controlled by the progression of the learning capability of the student over the training iterations without the need to tune any hyper-parameters. Extensive experiments and ablation studies show that AdaDistill can enhance the discriminative learning capability of the student and demonstrate superiority over various state-of-the-art competitors on several challenging benchmarks, such as IJB-B, IJB-C, and ICCV2021-MFR (https://github.com/fdbtrs/AdaDistill).
Author(s)
Boutros, Fadi  orcid-logo
Fraunhofer-Institut für Graphische Datenverarbeitung IGD  
Štruc, Vitomir
Univ. Ljubljana
Damer, Naser  
Fraunhofer-Institut für Graphische Datenverarbeitung IGD  
Mainwork
Computer Vision - ECCV 2024. Proceedings. Part LV  
Project(s)
Next Generation Biometric Systems  
Next Generation Biometric Systems  
Funder
Bundesministerium für Bildung und Forschung -BMBF-  
Hessisches Ministerium für Wissenschaft und Kunst -HMWK-  
Conference
European Conference on Computer Vision 2024  
DOI
10.1007/978-3-031-73001-6_10
Language
English
Fraunhofer-Institut für Graphische Datenverarbeitung IGD  
Keyword(s)
  • Branche: Information Technology

  • Research Line: Computer vision (CV)

  • Research Line: Human computer interaction (HCI)

  • Research Line: Machine learning (ML)

  • LTA: Interactive decision-making support and assistance systems

  • LTA: Machine intelligence, algorithms, and data structures (incl. semantics)

  • Machine learning

  • Deep learning

  • Face recognition

  • Biometrics

  • ATHENE

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024