Music emotion recognition: A state of the art review

Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR 2010

Volumn , Issue , 2010, Pages 255-266

  Kim, Youngmoo E ^a   Schmidt, Erik M ^a   Migneco, Raymond ^a   Morton, Brandon G ^a   Richardson, Patrick ^a   Scott, Jeffrey ^a   Speck, Jacquelin A ^a   Turnbull, Douglas ^a  

^a Drexel University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATED SYSTEMS; AUTOMATIC EMOTION RECOGNITION; CONTENT-BASED APPROACH; GENRE IDENTIFICATION; MULTIPLE FEATURES; MUSIC EMOTIONS; MUSIC INFORMATION RETRIEVAL; PAPER SURVEYS; STATE OF THE ART; STATE-OF-THE ART REVIEWS; TEXT INFORMATION;

AUTOMATION; INFORMATION RETRIEVAL;

BEHAVIORAL RESEARCH;

EID: 84873591302     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (81)

1. C. C. Pratt, Music as the language of emotion. The Library of Congress, December 1950.
2. J. S. Downie, "The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research, " Acoustical Science and Technology, vol. 29, no. 4, pp. 247-255, 2008.
3. X. Hu, J. Downie, C. Laurier, M. Bay, and A. Ehmann, "The 2007 MIREX audio mood classification task: Lessons learned, " in Proc. of the Intl. Conf. on Music Information Retrieval, Philadelphia, PA, 2008.
4. J. A. Russell, "A circumspect model of affect, " Journal of Psychology and Social Psychology, vol. 39, no. 6, p. 1161, 1980.
5. P. Juslin and P. Luakka, "Expression, perception, and induction of musical emotions: A review and questionnaire study of everyday listening, " Journal of New Music Research, vol. 33, no. 3, p. 217, 2004.
6. A. Mehrabian and J. A. Russell, An Approach to Environmental Psychology. MIT Press, 1974.
7. C. McKay, "Emotion and music: Inherent responses and the importance of empirical cross-cultural research, " Course Paper. McGill University, 2002.
8. L.-L. Balkwill and W. F. Thompson, "A cross-cultural investigation of the perception of emotion in music, " Music Perception, vol. 17, no. 1, pp. 43-64, 1999.
9. T. Fritz, S. Jentschke, N. Gosselin, D. Sammler, I. Peretz, R. Turner, A. D. Friederici, and S. Koelsch, "Universal recognition of three basic emotions in music, " Current Biology, vol. 19, no. 7, pp. 573 - 576, 2009.
10. K. Hevner, "Experimental studies of the elements of expression in music, " American Journal of Psychology, vol. 48, no. 2, pp. 246-267, 1936.
11. P. Juslin and J. Sloboda, Music and Emotion: theory and research. Oxford Univ. Press, 2001.
12. E. Schubert, "Update of the Hevner adjective checklist, " Perceptual and Motor Skills, vol. 96, pp. 1117-1122, 2003.
13. M. Zentner, D. Grandjean, and K. R. Scherer, "Emotions evoked by the sound of music: Characterization, classification, and measurement." Emotion, vol. 8, p. 494, 2008.
14. C. Laurier, M. Sordo, J. Serra, and P. Herrera, "Music mood representation from social tags, " in Proc. of the Intl. Society for Music Information Conf., Kobe, Japan, 2009.
15. T. Eerola, O. Lartillot, and P. Toiviainen, "Prediction of multidimensional emotional ratings in music from audio using multivariate regression models, " in Proc. of the Intl. Society for Music Information Conf., Kobe, Japan, 2009.
16. L. Mion and G. D. Poli, "Score-independent audio features for description of music expression, " IEEE Transactions on Audio, Speech and Language Processing, vol. 16, no. 2, pp. 458-466, 2008.
17. E. Bigand, "Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts, " Cognition and Emotion, vol. 19, no. 8, p. 1113, 2005.
18. U. Schimmack and R. Reisenzein, "Experiencing activation: energetic arousal and tense arousal are not mixtures of valence and activation, " Emotion, vol. 2, no. 4, p. 412, 2002.
19. D. Watson and L. Clark, PANAS-X: Manual for the Positive and Negative Affect Schedule, expanded form ed., University of Iowa, 1994.
20. A. Tellegen, D. Watson, and L. A. Clark, "On the dimensional and hierarchical structure of affect, " Psychological Science, vol. 10, no. 4, pp. 297-303, 1999.
21. J. Libeks and D. Turnbull, "Exploring artist image using content-based analysis of promotional photos, " in Proc. of the Int. Computer Music Conf., 2010.
22. D. Turnbull, L. Barrington, D. Torres, and G. Lanckriet, "Semantic annotation and retrieval of music and sound effects, " IEEE Transactions on Audio, Speech and Language Processing, vol. 16, no. 2, 2008.
23. K. Trohidis, G. Tsoumakas, G. Kalliris, and I. Vlahavas, "Multilabel classification of music into emotion, " in Proc. of the Intl. Conf. on Music Information Retrieval, Philadelphia, PA, 2008.
24. F. Miller, M. Stiksel, and R. Jones, "Last.fm in numbers, " Last.fm press material, February 2008.
25. P. Lamere and O. Celma, "Music recommendation tutorial notes, " ISMIR Tutorial, September 2007.
26. L. von Ahn, "Games with a purpose, " Computer, vol. 39, no. 6, pp. 92-94, 2006.
27. M. I. Mandel and D. P. W. Ellis, "A web-based game for collecting music metadata, " in Proc. of the Intl. Conf. on Music Information Retrieval, Vienna, Austria, 2007, pp. 365-366.
28. D. Turnbull, R. Liu, L. Barrington, and G. Lanckriet, "A game-based approach for collecting semantic annotations of music, " in Proc. Intl. Conf. on Music Information Retrieval, Vienna, Austria, 2007, pp. 535-538.
29. E. L. M. Law, L. von Ahn, R. B. Dannenberg, and M. Crawford, "TagATune: A game for music and sound annotation, " in Proc. of the Intl. Conf. on Music Information Retrieval, Vienna, Austria, 2007.
30. Y. Kim, E. Schmidt, and L. Emelle, "Moodswings: A collaborative game for music mood label collection, " in Proc. Intl. Conf. on Music Information Retrieval, Philadelphia, PA, September 2008.
31. B. G. Morton, J. A. Speck, E. M. Schmidt, and Y. E. Kim, "Improving music emotion labeling using human computation, " in HCOMP 2010: Proc. of the ACM SIGKDD Workshop on Human Computation, Washington, D.C., 2010.
32. J. P. Bachorik, M. Bangert, P. Loui, K. Larke, J. Berger, R. Rowe, and G. Schlaug, "Emotion in motion: Investigating the time-course of emotional judgments of musical stimuli, " Music Perception, vol. 26, no. 4, pp. 355-364, April 2009.
33. L. Barrington, D. Turnbull, D. O'Malley, and G. Lanckriet, "User-centered design of a social game to tag music, " ACM KDD Workshop on Human Computation, 2009.
34. P. Knees, T. Pohle, M. Schedl, D. Schnitzer, and K. Seyerlehner, A Document-Centered Approach to a Natural Language Music Search Engine. Springer Berlin/Heidelberg, 2008, pp. 627-631.
35. O. Celma, P. Cano, and P. Herrera, "Search sounds: An audio crawler focused on weblogs, " in Proc. of the Intl. Conf. on Music Information Retrieval, Victoria, Canada, 2006.
36. B. Whitman and D. Ellis, "Automatic record reviews, " in Proc. of the Intl. Conf. on Music Information Retrieval, Barcelona, Spain, 2004, pp. 470-477.
37. M. Levy and M. Sandler, "A semantic space for music derived from social tags, " in Proc. of the Intl. Conf. on Music Information Retrieval, Vienna, Austria, 2007.
38. P. Knees, T. Pohle, M. Schedl, and G. Widmer, "A music search engine built upon audio-based and web-based similarity measures, " in ACM SIGIR, 2007.
39. B. Whitman, "Combining musical and cultural features for intelligent style detection, " in Proc. of the Intl. Conf. on Music Information Retrieval, Paris, France, 2002, pp. 47-52.
40. P. Knees, E. Pampalk, and G. Widmer, "Artist classification with web-based data, " in Proc. of the Intl. Symposium on Music Information Retrieval, Barcelona, Spain, 2004, pp. 517- 524.
41. K. Bischoff, C. S. Firan, R. Paiu, W. Nejdl, C. Laurier, and M. Sordo, "Music mood and theme classification-a hybrid approach, " in Proc. of the Intl. Society for Music Information Retrieval Conf., Kobe, Japan, 2009.
42. K. Bischoff, C. Firan, W. Nejdl, and R. Paiu, "How do you feel about 'Dancing Queen'?: Deriving mood & theme annotations from user tags, " in Proc. of the ACM/IEEE-CS Joint Conf. on Digital Libraries, Austin, TX, Jan 2009.
43. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, "Indexing by latent semantic analysis, " Journal of the American Society for Information Science, vol. 41, pp. 391-407, 1990.
44. B. Logan, A. Kositsky, and P. Moreno, "Semantic analysis of song lyrics, " in Proc. IEEE Intl. Conf. on Multimedia and Expo, Taipei, Taiwan, June 27-30 2004.
45. M. M. Bradley and P. J. Lang, "Affective norms for English words (ANEW), " The NIMH Center for the Study of Emotion and Attention, University of Florida, Tech. Rep., 1999.
46. Y. Hu, X. Chen, and D. Yang, "Lyric-based song emotion detection with affective lexicon and fuzzy clustering method, " in Proc. of the Intl. Society for Music Information Conf., Kobe, Japan, 2009.
47. R. H. Chen, Z. L. Xu, Z. X. Zhang, and F. Z. Luo, "Content based music emotion analysis and recognition, " in Proc. of the Intl. Workshop on Computer Music and Audio Technology, 2006.
48. Y. Xia, L. Wang, K. Wong, and M. Xu, "Sentiment vector space model for lyric-based song sentiment classification, " in Proc. of the Association for Computational Linguistics. Columbus, Ohio, U.S.A: ACL-08, 2008, pp. 133-136.
49. O. C. Meyers, "A mood-based music classification and exploration system, " Master's thesis, Massachusetts Institute of Technology, June 2007.
50. Y.-H. Yang, Y.-C. Lin, H.-T. Cheng, I.-B. Liao, Y.-C. Ho, and H. Chen, Advances in Multimedia Information Processing - PCM 2008, ser. Lecture Notes in Computer Science. Springer Berlin/Heidelberg, December 2008, ch. 8, pp. 70- 79.
51. S. Davis and P. Mermelstein, "Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences, " IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 28, no. 4, pp. 357- 366, 1980.
52. B. Logan, "Mel frequency cepstral coefficients for music modeling, " in Proc. of the Intl. Symposium on Music Information Retrieval, Plymouth, MA, September 2000.
53. K. F. MacDorman, S. Ough, and C.-C. Ho, "Automatic emotion prediction of song excerpts: Index construction, algorithm design, and empirical comparison, " Journal of New Music Research, vol. 36, no. 4, pp. 281-299, 2007.
54. E. M. Schmidt, D. Turnbull, and Y. E. Kim, "Feature selection for content-based, time-varying musical emotion regression, " in MIR '10: Proc. of the Intl. Conf. on Multimedia Information Retrieval, Philadelphia, PA, 2010, pp. 267-274.
55. E. M. Schmidt and Y. E. Kim, "Prediction of time-varying musical mood distributions from audio, " in Proc. of the Int. Society for Music Information Conf., Utrecht, Netherlands, August 2010.
56. T. Fujishima, "Realtime chord recognition of musical sound: A system using Common Lisp music." in Proc. of the Intl. Computer Music Conf., 1999.
57. G. Tzanetakis and P. Cook, "Musical genre classification of audio signals, " IEEE Transactions on Speech and Audio Processing, vol. 10, no. 5, pp. 293- 302, 2002.
58. D. Jiang, L. Lu, H. Zhang, J. Tao, and L. Cai, "Music type classification by spectral contrast feature, " in Proc. Intl. Conf. on Multimedia and Expo, vol. 1, 2002, pp. 113-116.
59. L. Lu, D. Liu, and H. J. Zhang, "Automatic mood detection and tracking of music audio signals, " IEEE Transactions on Audio, Speech and Language Processing, vol. 14, no. 1, pp. 5-18, 2006.
60. T. Li and M. Ogihara, "Detecting emotion in music, " in Proc. of the Intl. Conf. on Music Information Retrieval, Baltimore, MD, October 2003.
61. M. I. Mandel, G. E. Poliner, and D. P.W. Ellis, "Support vector machine active learning for music retrieval, " Multimedia Systems, vol. 12, no. 1, pp. 3-13, Aug 2006.
62. J. Skowronek, M. McKinney, and S. van de Par, "A demonstrator for automatic music mood estimation, " in Proc. Intl. Conf. on Music Information Retrieval, Vienna, Austria, 2007.
63. G. Tzanetakis, "Marsyas submissions to MIREX 2007, " MIREX 2007.
64. G. Peeters, "A generic training and classification system for MIREX08 classification tasks: Audio music mood, audio genre, audio artist and audio tag, " MIREX 2008.
65. C. Cao and M. Li, "Thinkit's submissions for MIREX2009 audio music classification and similarity tasks." ISMIR, MIREX 2009.
66. A. Smola and B. Schölkopf, "A tutorial on support vector regression, " Statistics and Computing, vol. 14, no. 3, pp. 199- 222, 2004.
67. D. Shrestha and D. Solomatine, "Experiments with AdaBoost. RT, an improved boosting scheme for regression, " Neural Computation, vol. 18, no. 7, pp. 1678-1710, 2006.
68. D. Cabrera, S. Ferguson, and E. Schubert, "Psysound3: software for acoustical and psychoacoustical analysis of sound recordings, " in Proc. of the Intl. Conf. on Auditory Display, Montreal, Canada, June 26-29 2007, pp. 356-363.
69. G. Tzanetakis and P. Cook, "Marsyas: a framework for audio analysis, " Organized Sound, vol. 4, no. 3, pp. 169-175, 1999.
70. B. Han, S. Rho, R. B. Dannenberg, and E. Hwang, "SMERS: Music emotion recognition using support vector regression, " in Proc. of the Intl. Society for Music Information Conf., Kobe, Japan, 2009.
71. J.-J. Aucouturier and F. Pachet, "Improving timbre similarity: How high is the sky?" Journal of Negative Results in Speech and Audio Sciences, vol. 1, no. 1, 2004.
72. J. Cohn and G. Katz, "Bimodal expression of emotion by face and voice, " in ACM Intl. Multimedia Conf., 1998.
73. L. Silva and P. Ng, "Bimodal emotion recognition, " in Proc. of the IEEE Intl. Conf. on Automatic Face and Gesture Recognition, 2000, pp. 332-335.
74. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions, " in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 1, 2009, pp. 39-58.
75. R. Neumayer and A. Rauber, "Integration of text and audio features for genre classification in music information retrieval, " in Proc. of the European Conf. on Information Retrieval, 2007.
76. D. Yang and W. Lee, "Disambiguating music emotion using software agents, " in Proc. of the Intl. Conf. on Music Information Retrieval. Barcelona, Spain: Universitat Pompeu Fabra, October 2004.
77. C. Laurier, J. Grivolla, and P. Herrera, "Multimodal music mood classification using audio and lyrics, " in Proc. of the Intl. Conf. on Machine Learning and Applications. Universitat Pompeu Fabra, 2008, pp. 1-6.
78. X. Hu, J. S. Downie, and A. F. Ehmann, "Lyric text mining in music mood classification, " in Proc. of the Intl. Society for Music Information Retrieval Conf., Kobe, Japan, 2009.
79. B. Schuller, J. Dorfner, and G. Rigoll, "Determination of nonprototypical valence and arousal in popular music: Features and performances, " EURASIP Journal on Audio, Speech, and Music Processing, vol. 2010, pp. 1-20, Jan 2010.
80. D. Turnbull, L. Barrington, M.Yazdani, and G. Lanckriet, "Combining audio content and social context for semantic music discovery, " ACM SIGIR, 2009.
81. P. Dunker, S. Nowak, A. Begau, and C. Lanz, "Content-based mood classification for photos and music: A generic multimodal classification framework and evaluation approach, " in Proc. of the Intl. Conf. on Multimedia Information Retrieval. ACM, 2008.