• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Artikel
  4. Detecting Voice Cloning and Text to Speech Audio in Real Time on Mobile Devices
 
  • Details
  • Full
Options
February 2, 2025
Journal Article
Title

Detecting Voice Cloning and Text to Speech Audio in Real Time on Mobile Devices

Abstract
In this paper, we present a method that analyzes an audio stream in real time and provides an indication of whether the voice is synthetic generated by a voice clone or a text to speech model. Unlike state-of-the-art techniques that rely on self-supervised (SSL) or non-self-supervised learning, this method is deterministic and focuses on the analysis of tonal and non-tonal components within an audio stream. By leveraging principles from the MPEG-1 global masking threshold, the algorithm systematically evaluates tonal and noise components within a defined frequency range. The underlying hypothesis is that synthesized audio exhibits distinct tonal and non-tonal characteristics compared to original human speech, which can be quantified for classification. This interpretable, deterministic framework addresses key limitations of existing SSL-based approaches, including high computational costs and limited transparency. Beyond detecting synthesized speech, the method provides insights into the likely model used for generation. Experimental evaluations demonstrate the algorithm’s effectiveness, revealing distinct and consistent patterns across various TTS and voice conversion (VC) models, thereby offering a reliable and computationally efficient solution for audio authenticity verification. The proposed algorithm is developed and tested on a small dataset and show an excellent separation between different solution providers and genuine voices.
Author(s)
Berchtold, Waldemar  
Fraunhofer-Institut für Sichere Informationstechnologie SIT  
Heeger, Julian  
Fraunhofer-Institut für Sichere Informationstechnologie SIT  
Bugert, Simon  
Fraunhofer-Institut für Sichere Informationstechnologie SIT  
Steinebach, Martin  
Fraunhofer-Institut für Sichere Informationstechnologie SIT  
Journal
Electronic imaging. Online journal  
Conference
International Symposium on Electronic Imaging 2025  
Symposium "Mobile Devices and Multimedia - Enabling Technologies, Algorithms, and Applications" 2025  
DOI
10.2352/EI.2025.37.3.MOBMU-310
Additional full text version
Landing Page
Language
English
Fraunhofer-Institut für Sichere Informationstechnologie SIT  
  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024