Prediction of the Distribution of Perceived Music Emotions Using Discrete Samples

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 7, 2011, Pages 2184-2196

  Yang, Yi Hsuan ^a   Chen, Homer H ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Arousal; emotion distribution prediction; music emotion recognition; regression; subjectivity; valence

Indexed keywords

EID: 85008559668     PISSN: 15587916     EISSN: 15587924     Source Type: Journal    
DOI: 10.1109/TASL.2011.2118752     Document Type: Article

References (81)

1. R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
2. L. B. Meyer, Emotion and Meaning in Music. Chicago, IL: Univ. of Chicago Press, 1956.
3. P. N. Juslin and J. A. Sloboda, Music and Emotion: Theory and Research. New York: Oxford Univ. Press, 2001.
4. S. Hallam, I. Cross, and M. Thaut, The Oxford Handbook of Music Psychology. New York: Oxford Univ. Press, 2008.
5. A. Gabrielsson, “Emotion perceived and emotion felt: Same or different?,” Musicae Scientiae, pp. 123–147, 2002.
6. D. Huron, Sweet Anticipation: Music and the Psychology of Expectation. Cambridge, MA: MIT Press, 2006.
7. H. Katayose, M. Imai, and S. Inokuchi, “Sentiment extraction in music,” in Proc. Int. Conf. Pattern Recognition, 1998, pp. 1083–1087.
8. X. Hu, J. S. Downie, C. Laurier, M. Bay, and A. F. Ehmann, “The 2007 MIREX audio mood classification task: Lessons learned,” in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 462–467.
9. L. Lu, D. Liu, and H. Zhang “Automatic mood detection and tracking of music audio signals,” IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 5–18, Jan. 2006.
10. M.-Y. Wang, N.-Y. Zhang, and H.-C. Zhu, “User-adaptive music emotion recognition,” in Proc. IEEE Int. Conf. Signal Process., 2004, pp. 1352–1355.
11. T. Li and M. Ogihara, “Detecting emotion in music,” in Proc. Int. Conf. Music Inf. Retrieval, 2003, pp. 239–240.
12. K. Trohidis, G. Tsoumakas, G. Kalliris, and I. Vlahavas, “Multilabel classification of music into emotions,” in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 325–330.
13. Y.-H. Yang, C. C. Liu, and H. H. Chen, “Music emotion classification: A fuzzy approach,” in Proc. ACM Int. Conf. Multimedia, 2006, pp. 81–84.
14. Y. E. Kim, E. M. Schmidt, R. Migneco, B. G. Morton, P. Richardson, J. Scott, J. Speck, and D. Turnbull, “Music emotion recognition: A state of the art review,” in Proc. Int. Conf. Music Inf. Retrieval, 2010, pp. 255–266.
15. P. N. Juslin and P. Laukka “Expression, perception and induction of musical emotions: A review and a questionnaire study of everyday listening,” J. New Music Res., vol. 33, no. 3, pp. 217–238, 2004.
16. J. A. Russell “A circumplex model of affect,” J. Personal. Social Sci., vol. 39, no. 6, pp. 1161–1178, 1980.
17. Y.-H. Yang, Y.-F. Su, Y.-C. Lin, and H. H. Chen, “Music emotion recognition: The role of individuality,” in Proc. ACM Int. Workshop Human-Centered Multimedia, 2007, pp. 13–21.
18. Y.-H. Yang, Y.-C. Lin, Y.-F. Su, and H. H. Chen “A regression approach to music emotion recognition,” IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 2, pp. 448–457, Feb. 2008.
19. Y.-H. Yang, Y.-C. Lin, H.-T. Cheng, and H. H. Chen, “Mr. Emo: Music retrieval in the emotion plane,” in Proc. ACM Int. Conf. Multimedia, 2008, pp. 1003–1004.
20. Y.-H. Yang and H. H. Chen “Ranking-based emotion recognition for music organization and retrieval,” IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 4, pp. 762–774, 2011.
21. Y.-H. Yang, Y.-C. Lin, and H. H. Chen, “Personalized music emotion recognition,” in Proc. ACM Int. Conf. Inf. Retrieval, 2009, pp. 748–749.
22. K. MacDorman, S. Ough, and C.-C. Ho “Automatic emotion prediction of song excerpts: Index construction, algorithm design and empirical comparison,” J. New Music Res., vol. 36, no. 4, pp. 281–299, 2007.
23. T. Eerola, O. Lartillot, and P. Toiviainen, “Prediction of multidimensional emotional ratings in music from audio using multivariate regression models,” in Proc. Int. Conf. Music Inf. Retrieval, 2009, pp. 621–626.
24. A. Huq, J. P. Bello, and R. Rowe “Automated music emotion recognition: A systematic evaluation,” J. New Music Res., vol. 39, no. 3, pp. 227–244, 2010.
25. M. D. Korhonen, D. A. Clausi, and M. E. Jernigan “Modeling emotional content of music using system identification,” IEEE Trans. Syst., Man, Cybern., vol. 36, no. 3, pp. 588–599, Jun. 2006.
26. E. Schubert, “Measurement and time series analysis of emotion in music,” Ph.D. dissertation, School of Music Education, Univ. New South Wales, Sydney, Australia, 1999.
27. E. M. Schmidt and Y. E. Kim, “Prediction of time-varying musical mood distributions from audio,” in Proc. Int. Conf. Music Inf. Retrieval, 2010, pp. 465–470.
28. E. M. Schmidt, D. Turnbull, and Y. E. Kim, “Feature selection for content-based, time-varying musical emotion regression,” in Proc. ACM Int. Conf. Multimedia Inf. Retrieval, 2010, pp. 267–274.
29. P. Ekman “An argument for basic emotions,” Cogn. Emot., vol. 6, no. 3, pp. 169–200, 1992.
30. K. Hevner “Expression in music: A discussion of experimental studies and theories,” Psychol. Rev., vol. 48, no. 2, pp. 186–204, 1935.
31. R. E. Thayer, The Biopsychology of Mood and Arousal. New York: Oxford Univ. Press, 1989.
32. G. Collier “Beyond valence and activity in the emotional connotations of music,” Psychol. Music, vol. 35, no. 1, pp. 110–131, 2007.
33. R. A. Calvo and S. D'Mello, “Affect detection: An interdisciplinary review of models, methods and their applications,” IEEE Trans. Affective Comput., vol. 1, no. 1, pp. 18–37, Jan. 2010.
34. C. Laurier, M. Sordo, J. Serra, and P. Herrera, “Music mood representations from social tags,” in Proc. Int. Conf. Music Inf. Retrieval, 2009, pp. 381–386.
35. H.-T. Cheng, Y.-H. Yang, Y.-C. Lin, I.-B. Liao, and H.-H. Chen, “Automatic chord recognition for music classification and retrieval,” in Proc. IEEE Int. Conf. Multimedia Expo., 2008, pp. 1505–1508.
36. Y.-C. Lin, Y.-H. Yang, and H.-H. Chen, “Exploiting genre for music emotion classification,” in Proc. IEEE Int. Conf. Multimedia Expo., 2009, pp. 618–621.
37. Y.-H. Yang, Y.-C. Lin, H.-T. Cheng, I.-B. Liao, Y.-C. Ho, and H.-H. Chen, “Toward multi-modal music emotion classification,” in Proc. Pacific-Rim Conf. Multimedia, 2008, pp. 70–79.
38. Y.-H. Yang and H. H. Chen, “Machine recognition of music emotion: A review,” ACM Trans. Intell. Syst. Technol., 2011, to be published.
39. X. Hu and J. S. Downie, “When lyrics outperform audio for music mood classification: A feature analysis,” in Proc. Int. Conf. Music Inf. Retrieval, 2010, pp. 619–624.
40. Q. Lu, X. Chen, D. Yang, and J. Wang, “Boosting for multi-modal music emotion classification,” in Proc. Int. Conf. Music Inf. Retrieval, 2010, pp. 105–110.
41. M. Zaanen and P. Kanters, “Automatic mood classification using tf*idf based on lyrics,” in Proc. Int. Conf. Music Inf. Retrieval, 2010, pp. 75–80.
42. B. Schuller, C. Hage, D. Schuller, and G. Rigoll “Mister D.J., cheer me up!': musical and textual features for automatic mood classification,” J. New Music Res., vol. 39, no. 1, pp. 13–34, 2010.
43. B. Schuller, J. Dorfner, and G. Rigoll, “Determination of nonprototypical valence and arousal in popular music: Features and performances,” EURASIP J. Audio, Speech, Music Process., p. 19, 2010, Article ID 735854.
44. A. Sen and M. Srivastava, Regression Analysis: Theory, Methods and Applications. New York: Springer, 1990.
45. A. W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data Analysis. New York: Oxford Univ. Press, 1997.
46. Z. I. Botev, J. F. Grotowski, and D. P. Kroese, “Kernel density estimation via diffusion,” Ann. Statist., 2009.
47. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2000.
48. C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press, 2006 [Online]. Available: http://www.gaussianprocess.org/gpml/
49. C. Cortes and V. Vapnik “Support vector networks,” Mach. Learn., vol. 20, no. 3, pp. 273–297, 1995.
50. B. Scholkopf, A. Smola, R. Williamson, and P. Bartlett “New support vector algorithms,” Neural Comput., vol. 12, pp. 1207–1245, 2000.
51. S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
52. J. Kittler, M. Hatef, R. Duin, and J. Matas “On combining classifiers,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 3, pp. 226–239, Mar. 1998.
53. O. R. Terrades, E. Valveny, and S. Tabbone “Optimal classifier fusion in a non-Bayesian probabilistic framework,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 9, pp. 1630–1644, Sep. 2009.
54. S. A. Banawan and N. M. Zeidat, “A comparative study of load sharing in heterogeneous multicomputersystems,” in Proc. Annu. Simulat. Symp., 1992, pp. 22–31.
55. D. C. Montgomery, G. C. Runger, and N. F. Hubele, Engineering Statistics. New York: Wiley, 1998.
56. F. E. Grubbs, “Procedures for detecting outlying observations in samples,” Technometrics, vol. 11, no. 1, pp. 1–21, 1969.
57. P. N. Juslin “Cue utilization in communication of emotion in music performance: Relating performance to perception,” J. Exper. Psychol.: Human Percept. Perform., vol. 16, no. 6, pp. 1797–1813, 2000.
58. C. Krumhansl, “Music: A link between cognition and emotion,” Current Direct. Psychol. Sci., vol. 11, no. 2, pp. 45–50, 2002.
59. A. Gabrielsson and E. Lindström, “The influence of musical structure on emotional expression,” in Music and Emotion: Theory and Research, P. N. Juslin and J. A. Sloboda, Eds. New York: Oxford Univ. Press, 2001.
60. O. Lartillot and P. Toiviainen, “MIR in Matlab (II): A toolbox for musical feature extraction from audio,” in Proc. Int. Conf. Music Inf. Retrieval, 2007, pp. 127–130 [Online]. Available: http://users.jyu.fi/lartillo/mirtoolbox/
61. A. Meng, P. Ahrendt, J. Larsen, and L. K. Hansen “Temporal feature integration for music genre classification,” IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 5, pp. 1654–1663, Jul. 2007.
62. G. Tzanetakis and P. Cook, “Musical genre classification of audio signals,” IEEE Trans. Speech Audio Process., vol. 10, no. 5, pp. 293–302, Jul. 2002.
63. E. Pampalk, “A Matlab toolbox to compute music similarity from audio,” in Proc. Int. Conf. Music Information Retrieval, 2004 [Online]. Available: http://www.ofai.at/~elias.pampalk/ma/
64. E. Benetos, M. Kotti, and C. Kotropoulos, “Large scale musical instrument identification,” in Proc. Int. Conf. Music Inf. Retrieval, 2007 [Online]. Available: http://www.ifs.tuwien.ac.at/mir/muscle/del/audio tools.html#SoundDescrToolbox
65. T. Lidy and A. Rauber, “Evaluation of feature extractors and psychoacoustic transformations for music genre classification,” in Proc. Int. Conf. Music Inf. Retrieval, 2005, pp. 34–41 [Online]. Available: http://www.ifs.tuwien.ac.at/mir/audiofeatureextraction.html
66. T. Hofmann, “Probabilistic latent semantic indexing,” in Proc. ACM Int. Conf. Inf. Retrieval, 1999, pp. 50–57.
67. T. Tolonen and M. Karjalainen “A computationally efficient multipitch analysis model,” IEEE Trans. Speech Audio Process., vol. 8, no. 6, pp. 708–716, Nov. 2000.
68. E. Gómez, “Tonal description of music audio signal,” Ph.D. dissertation, Univ. Pompeu Fabra, Barcelona, Spain, 2006.
69. C. Harte, M. Sandler, and M. Gasser, “Detecting harmonic change in musical audio,” in Proc. ACM Workshop Audio Music Comput. Multimedia, 2006, pp. 21–26.
70. E. Allamanche, J. Herre, O. Helmuth, B. Froba, T. Kasten, and M. Cremer, “Content-based identification of audio material using MPEG-7 low level description,” in Proc. Int. Conf. Music Inf. Retrieval, 2001, pp. 197–204.
71. W. F. Thompson and B. Robitaille, “Can composers express emotions through music?,” Empirical Studies of the Arts, vol. 10, pp. 79–89, 1992.
72. S. O. Ali “Songs and emotions: Are lyrics and melodies equal partners,” Psychol. Music, vol. 34, no. 4, pp. 511–534, 2006.
73. F. Sebastiani “Machine learning in automated text categorization,” ACM Comput. Surveys, vol. 34, no. 1, pp. 1–47, 2002.
74. Y. Rubner, C. Tomasi, and L. J. Guibas “The earth mover's distance as a metric for image retrieval,” Int. J. Comput. Vis., vol. 40, no. 2, pp. 99–121, 2000.
75. C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” 2001 [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm
76. A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” Proc. Neural Inf. Process. Syst., pp. 849–856, 2002.
77. J. Ellson, E. R. Gansner, E. Koutsofios, S. C. North, and G. Woodhull, “Graphviz—open source graph drawing tools,” in Proc. Graph Drawing, 2001, pp. 483–484 [Online]. Available: http://www.graphviz.org/
78. S. Schaefer, T. McPhail, and J. Warren “Image deformation using moving least squares,” ACM Trans. Graphics, vol. 25, no. 3, pp. 533–540, 2006.
79. A. Hanjalic and L.-Q. Xu, “Affective video content representation and modeling,” IEEE Trans. Multimedia, vol. 7, pp. 143–154, 2005.
80. E. V. Bonilla, K. M. A. Chai, and C. K. I. Williams, “Multitask Gaussian process prediction,” in Proc. Conf. Neural Inf. Process. Syst., 2008, pp. 164–170.
81. P. Boyle and M. Frean, “Dependent Gaussian processes,” in Proc. Conf. Neural Inf. Process. Syst., 2004, pp. 217–224.