improving music mood classification using lyrics, audio and social tags

the ultimate judge. Thus ground truth datasets in existing experiments were mostly built by
recruiting human assessors to manually label music pieces, and then selecting pieces with (near)
consensus on human judgments. However, judgments on music are very subjective, and it is hard
to achieve agreements across assessors (Skowronek, McKinney, & van de Par, 2006). This has
seriously limited the sizes of experimental datasets and necessary validation on inter-assessor
credibility. As a result, experimental datasets usually consist of merely several hundreds of
music pieces, and each piece is judged by at most three human assessors, and in many cases, by
only one assessor (e.g., Trohidis, Tsoumakas, Kalliris, & Vlahavas, 2008; Li & Ogihara, 2003;
Lu, Liu, & Zhang, 2006).
The situation is worsened by the intellectual property regulations on music materials, which
effectively prevents sharing ground truth datasets with audio content among MIR researchers
affiliated with different institutions. Therefore, it is clear that to enhance the development and
evaluation in music mood classification, and in MIR research in general, a sound method is
much in need to build ground truth sets of reliable quality in an efficient manner.
1.2.3 Multi-modal Classification
Until recent years, MIR systems have focused on single-modal representation of music,
mostly on audio content and some on symbolic scores. The seminal work of Aucouturier and
Pachet (2004) pointed out that there appeared to be a “glass ceiling” in audio-based MIR, due to
the fact that some high-level music features with semantic meanings might be too difficult to be
derived from audio using current technology. Hence, researchers started paying attention to
multi-modal classification systems that combine audio and text (e.g., Neumayer & Rauber, 2007;
Aucouturier, Pachet, Roy, & Beurivé, 2007; Dhanaraj & Logan, 2005; Muller, Kurth, Damm,
4