Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Training phoneme models for singing with "songified" speech data

: Kruspe, Anna M.

Fulltext (PDF; )

Müller, M.; Wiering, F. ; International Society for Music Information Retrieval -ISMIR-:
16th International Society for Music Information Retrieval Conference, ISMIR 2015. Proceedings : Málaga, Spain, October 26-30, 2015
Málaga, 2015
ISBN: 978-84-606-8853-2
International Society for Music Information Retrieval (ISMIR Conference) <16, 2015, Málaga>
Conference Paper, Electronic Publication
Fraunhofer IDMT ()
vocal analysis; phoneme recognition

Speech recognition in singing is a task that has not been widely researched so far. Singing possesses several characteristics that differentiate it from speech. Therefore, algorithms and models that were developed for speech usually perform worse on singing.
One of the bottlenecks in many algorithms is the recognition of phonemes in singing. We noticed that this recognition step can be improved when using singing data in model training, but to our knowledge, there are no large datasets of singing data annotated with phonemes. However, such data does exist for speech.
We therefore propose to make phoneme recognition models more robust for singing by training them on speech data that has artificially been made more “song-like”. We test two main modifications on speech data: Time stretching and pitch shifting. Artificial vibrato is also tested. We then evaluate models trained on different combinations of these modified speech recordings. The utilized modeling algorithms are Neural Networks and Deep Belief Networks.