Ranking-based emotion recognition for music organization and retrieval

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 4, 2011, Pages 762-774

  Yang, Yi Hsuan ^a   Chen, Homer H ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Arousal; learning to rank; music emotion recognition; music emotion tournament; ranking; rating; valence

Indexed keywords

AROUSAL; LEARNING-TO-RANK; MUSIC EMOTION RECOGNITION; MUSIC EMOTION TOURNAMENT; RANKING; VALENCE;

ALGORITHMS; RATING; SEMANTICS;

BEHAVIORAL RESEARCH;

EID: 79951628231     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2064164     Document Type: Article

References (67)

1. L. B. Meyer, Emotion and Meaning in Music. Chicago, IL: Univ. of Chicago Press, 1956.
2. J. A. Sloboda and P. N. Juslin, "Psychological perspectives on music and emotion," in Music and Emotion: Theory and Research, P.N. Juslin and J. A. Sloboda, Eds. New York: Oxford Univ. Press, 2001.
3. S. Hallam, I. Cross, and M. Thaut, The Oxford Handbook of Music Psychology. New York: Oxford Univ. Press, 2008.
4. A. Gabrielsson, "Emotion perceived and emotion felt: Same or different?," Musicae Scientiae, pp. 123-147, 2002, special issue.
5. L. Lu, D. Liu, and H. Zhang, "Automatic mood detection and tracking of music audio signals," IEEE Tra (Pubitemid 43863448)
6. X. Hu, J. S. Downie, C. Laurier,M. Bay, and A. F. Ehmann, "The 2007 MIREX audio mood classification task: Lessons learned," in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 462-467.
7. H. Katayose, M. Imai, and S. Inokuchi, "Sentiment extraction in music," in Proc. Int. Conf. Pattern Recognition, 1998, pp. 1083-1087.
8. M.-Y.Wang, N.-Y. Zhang, and H.-C. Zhu, "User-adaptive music emotion recognition," in Proc. IEEE Int. Conf. Signal Process., 2004, pp. 1352-1355.
9. T. Li and M. Ogihara, "Detecting emotion in music," in Proc. Int. Conf. Music Inf. Retrieval, 2003, pp. 239-240.
10. K. Trohidis, G. Tsoumakas, G. Kalliris, and I. Vlahavas, "Multi-label classification of music into emotions," in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 325-330.
11. Y.-H. Yang, C. C. Liu, and H. H. Chen, "Music emotion classification: A fuzzy approach," in Proc. ACM Int. Conf. Multimedia, 2006, pp. 81-84. (Pubitemid 47113509)
12. Y.-H. Yang, Y.-C. Lin, H.-T. Cheng, I.-B. Liao, Y.-C. Ho, and H.-H. Chen, "Toward multi-modal music emotion classification," in Proc. Pacific-Rim Conf. Multimedia, 2008, pp. 70-79.
13. H.-T. Cheng, Y.-H. Yang, Y.-C. Lin, I.-B. Liao, and H.-H. Chen, "Automatic chord recognition for music classification and retrieval," in Proc. IEEE Int. Conf. Multimedia Expo., 2008, pp. 1505-1508.
14. Y.-C. Lin, Y.-H. Yang, and H.-H. Chen, "Exploiting genre for music emotion classification," in Proc. IEEE Int. Conf. Multimedia Expo., 2009, pp. 618-621.
15. C. Laurier, J. Grivolla, and P. Herrera, "Multimodal music mood classification using audio and lyrics," in Proc. Int. Conf. Mach. Learn. Applicat., 2008, pp. 105-111.
16. P. N. Juslin and P. Laukka, "Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening," J. New Music Res., vol. 33, no. 3, pp. 217-238, 2004.
17. J. A. Russell, "A circumplex model of affect," J. Personal. Social Sci., vol. 39, no. 6, pp. 1161-1178, 1980.
18. R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford Univ. Press, 1989.
19. Y.-H. Yang, Y.-C. Lin, Y.-F. Su, and H. H. Chen, "A regression approach to music emotion recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 2, pp. 448-457, Feb. 2008.
20. Y.-H. Yang and H. H. Chen, "Music emotion ranking," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2009, pp. 1657-1660.
21. Y.-H.Yang, Y.-C. Lin, H.-T. Cheng, and H. H. Chen, "Mr. Emo: Music retrieval in the emotion plane," in Proc. ACM Int. Conf. Multimedia, 2008, pp. 1003-1004.
22. Y.-H. Yang, Y.-F. Su, Y.-C. Lin, and H. H. Chen, "Music emotion recognition: The role of individuality," in Proc. ACM Int. Workshop Human-Centered Multimedia, 2007, pp. 13-21.
23. Y.-H. Yang, Y.-C. Lin, and H. H. Chen, "Personalized music emotion recognition," in Proc. ACMInt. Conf. Inf. Retrieval, 2009, pp. 748-749.
24. S. Jun, S. Rho, B.-J. Han, and E. Hwang, "A fuzzy inference-based music emotion recognition system," in Proc. Vis. Inf. Eng., 2008, pp. 673-677.
25. K. F. MacDorman, S. Ough, and C.-C. Ho, "Automatic emotion prediction of song excerpts: Index construction, algorithm design, and empirical comparison," J. New Music Res., vol. 36, no. 4, pp. 281-299, 2007.
26. T. Eerola, O. Lartillot, and P. Toiviainen, "Prediction of multidimensional emotional ratings in music from audio using multivariate regression models," in Proc. Int. Conf. Music Inf. Retrieval, 2009, pp. 621-626.
27. M. D. Korhonen, D. A. Clausi, and M. E. Jernigan, "Modeling emotional content of music using system identification," IEEE Trans. Syst., Man, Cybern., vol. 36, no. 3, pp. 588-599, Jun. 2006. (Pubitemid 43820863)
28. E. Schubert, "Measurement and time series analysis of emotion in music," Ph.D. dissertation, School of Music Education, Univ. New South Wales, Sydney, Australia, 2006.
29. D. Yang and W.-S. Lee, "Disambiguating music emotion using software agents," in Proc. Int. Conf. Music Inf. Retrieval, 2004, pp. 52-58.
30. S. Ovadia, "Ratings and rankings: Reconsidering the structure of values and their measurement," Int. J. Social Res. Methodol., vol. 7, no. 5, pp. 403-414, 2004. (Pubitemid 40031837)
31. A. Sen and M. Srivastava, Regression Analysis: Theory, Methods, and Applications. New York: Springer, 1990.
32. Y. E. Kim, E. Schmidt, and L. Emelle, "Moodswings: A collaborative game for music mood label collection," in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 231-236.
33. M. Mandel and D. Ellis, "A web-based game for collecting music metadata," J. New Music Res., vol. 37, no. 2, pp. 151-165, 2008.
34. E. Law, K. West, M. Mandel, M. Bay, and J. S. Downie, "Evaluation of algorithms using games: The case of music annotation," in Proc. Int. Conf. Music Inf. Retrieval, 2009, pp. 387-392.
35. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
36. W. W. Cohen, R. E. Schapire, and Y. Singer, "Learning to order things," J. Artif. Intell. Res., vol. 10, pp. 243-270, 1999. (Pubitemid 30573909)
37. R. Herbrich, T. Graepel, and K. Obermayer, "Support vector learning for ordinal regression," in Proc. Int. Conf. Artif. Neural Netw., 1999, pp. 97-102. (Pubitemid 30519921)
38. F. Xia, T.-Y. Liu, J.Wang, W.-S. Zhang, and H. Li, "Listwise approach to learning to rank: Theory and algorithm," in Proc. IEEE Int. Conf. Mach. Learn., 2008, pp. 1192-1199.
39. S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
40. "Electronic Statistics Textbook" StatSoft, Tulsa, OK, 2010 [Online]. Available: http://www.statsoft.com/textbook/, StatSoft
41. Y.-H. Yang, W.-H. Hsu, and H.-H. Chen, "Online reranking via ordinal informative concepts for context fusion in concept detection and video search," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 12, pp. 1880-1890, Dec. 2009.
42. F. Pachet and P. Roy, "Improving multilabel analysis of music titles: A large-scale validation of the correction approach," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 2, pp. 335-343, Feb. 2009.
43. N. B. Karayiannis and M. Randolph-Gips, "On the construction and training of reformulated radial basis function neural networks," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 835-846, Jul. 2003.
44. T. Chen and H. Chen, "Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks," IEEE Trans. Neural Netw., vol. 6, no. 4, pp. 904-910, Jul. 1995.
45. S. Granqvist, "The visual sort and rate method for perceptual evaluation in listening tests," Logopedics Phoniatrics Vocology, vol. 28, no. 3, pp. 109-116, 2003. (Pubitemid 37322181)
46. N. Scaringella, G. Zoia, and D. Mlynek, "Automatic genre classification of music content: A survey," IEEE Signal Process. Mag., vol. 23, no. 2, pp. 133-141, Mar. 2006.
47. J. Fornari and T. Eerola, "The pursuit of happiness in music: Retrieving valence with high-level musical descriptors," in Proc. Comput. Music Modeling Retrieval, 2008.
48. C. Krumhansl, "Music: A link between cognition and emotion," Current Directions Psychol. Sci., vol. 11, no. 2, pp. 45-50, 2002.
49. B. Schuller, S. Steidl, and A. Batliner, "The Interspeech 2009 emotion challenge," in Proc. Interspeech, 2009.
50. D. Ververidis and C. Kotropoulos, "Emotional speech recognition: Resources, features, and methods," Speech Commun., vol. 48, no. 9, pp. 1162-1181, 2006. (Pubitemid 44128615)
51. P. N. Juslin, "Cue utilization in communication of emotion in music performance: Relating performance to perception," J. Exper. Psychol.: Human Percept. Perf., vol. 16, no. 6, pp. 1797-1813, 2000.
52. K. Hevner, "Expression in music: A discussion of experimental studies and theories," Psychol. Rev., vol. 48, no. 2, pp. 186-204, 1935.
53. A. Gabrielsson and E. Lindström, "The influence of musical structure on emotional expression," in Music and Emotion: Theory and Research, P. N. Juslin and J. A. Sloboda, Eds. New York: Oxford Univ. Press, 2001.
54. O. Lartillot and P. Toiviainen, "MIR in Matlab (II): A toolbox for musical feature extraction from audio," in Proc. Int. Conf. Music Inf. Retrieval, 2007, pp. 127-130 [Online]. Available: http://users.jyu.fi/lartillo/ mirtoolbox/.
55. A. Meng, P. Ahrendt, J. Larsen, and L. K. Hansen, "Temporal feature integration for music genre classification," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 5, pp. 1654-1663, Jul. 2007.
56. G. Tzanetakis and P. Cook, "Musical genre classification of audio signals," IEEE Trans. Speech Audio Process. vol. 10, no. 5, pp. 293-302, Jul. 2002 [Online]. Available: http://marsyas.sness.net/ (Pubitemid 34950067)
57. E. Pampalk, "A Matlab toolbox to compute music similarity from audio," in Proc. Int. Conf. Music Inf. Retrieval, 2004 [Online]. Available: http://www.ofai.at/elias.pampalk/ma/.
58. T. Tolonen and M. Karjalainen, "A computationally efficient multipitch analysis model," IEEE Trans. Speech Audio Process., vol. 8, no. 6, pp. 708-716, Nov. 2000.
59. E. Gómez, "Tonal description of music audio signal," Ph.D. dissertation, Univ. Pompeu Fabra, Barcelona, Spain, 2006.
60. C. Harte, M. Sandler, and M. Gasser, "Detecting harmonic change in musical audio," in Proc. ACM Workshop Audio Music Comput. Multimedia, 2006, pp. 21-26. (Pubitemid 47168287)
61. E. Allamanche, J. Herre, O. Helmuth, B. Fröba, T. Kasten, and M. Cremer, "Content-based identification of audio material using MPEG-7 low level description," in Proc. Int. Conf. Music Inf. Retrieval, 2001, pp. 197-204.
62. A. Klapuri, "Sound onset detection by applying psychoacoustic knowledge," in Proc. Int. Conf. Acoust., Speech, Signal Process., 1999, pp. 3089-3092.
63. R. Cohen and M. Swerdlik, Psychological Testing and Assessment: An Introduction to Tests and Measurement. Mountain View, CA: Mayfield, 2002.
64. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," 2001 [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/ libsvm.
65. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, 2003.
66. X. Geng, T.-Y. Liu, T. Qin, and H. Li, "Feature selection for ranking," in Proc. ACM Int. Conf. Inf. Retrieval, 2007, pp. 407-414. (Pubitemid 350164987)
67. M. R. S¡konja and I. Kononenko, "Theoretical and empirical analysis of ReliefF and RReliefF," Mach. Learn., vol. 53, pp. 23-69, 2003.