Analysis and Visualisation of Music

Music is characterised by the ensemble of rhythm, harmony and melody. There are many meaningful ways to transcribe these parts into human-readable formats. Typically, they require a certain degree of knowledge in music for interpretation. For people who would like to understand more of music, but lack the necessary time to gain more knowledge, these formats are unsuited. In this work, we aspire to create a more direct connection between the listener and the parts that make up music by visualising music. We create a system capable of decomposing music into its parts and then use lights which follow the singing voices and melodic instruments of a song. For this matter, we conduct a study into source separation methods to derive possible future applications in automated (unsupervised) music visualisation. Our goal is achieved using algorithms based on non-negative matrix factorisation for the separation of sources and a pitch detector to estimate fundamental frequencies at predetermined time intervals. For each isolated instrument, the frequency data, in relation to these time intervals, then controls our visualisation in sync with the audio. We present a prototype showcasing the general idea, and evaluate to what extent it fulfills our vision of enhancing the music listening experience. The results indicate that the system is generally successful and has great potential, but needs optimisation for a more unsupervised context.