Beyond Binary: Multi-Class Classification of Audio Deepfakes by TTS and Vocoder Types

With advances in AI, audio deepfakes are getting more and more convincing. Audio deepfakes can be generated using Text-to-Speech synthesis (TTS) or Voice conversion. Audio deepfake detection is, currently, mostly performed as binary classification task, dividing between spoof and bona-fide audio recordings. But different generation methods exist, introducing different artefacts in the generated recordings. In this work, we performed multi-class classification using various TTS-methods and vocoder types as spoofing classes, allowing the classifier to learn the specifies of the various models. We used three detectors, RawNet3, RawNet3 with an SSL-based front-end and an SSL-based front-end with a classification layer. The multi-class detectors outperformed the binary baseline on the in-domain and out-of-domain (in-the-wild) test sets. The detectors trained on classifying the vocoder types outperformed the classifiers trained on the TTS models. RawNet3 outperformed the SSL-based classifiers, reaching a 0% EER on the in-domain test set.