Affective visualization and retrieval for music video

IEEE Transactions on Multimedia

Volumn 12, Issue 6, 2010, Pages 510-522

  Zhang, Shiliang ^a   Huang, Qingming ^b   Jiang, Shuqiang ^a   Gao, Wen ^a   Tian, Qi ^c  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^c University of Texas at San Antonio   (United States)

Author keywords

Affective content analysis; affective visualization; dimensional affective model; support vector regression

Indexed keywords

AFFECTIVE MODEL; AFFECTIVE RESPONSE; AFFECTIVE STATE; AFFECTIVE VIDEO CONTENT ANALYSIS; AFFECTIVE VISUALIZATION; COMMONLY USED; COMPREHENSIVE EXPERIMENTS; CONTENT ANALYSIS; DATA SETS; INTEGRATED SYSTEMS; MODERN TIME; MUSIC VIDEO; SUPPORT VECTOR REGRESSION; USER STUDY;

AUDIO ACOUSTICS; INTEGRATED OPTICS;

VISUALIZATION;

EID: 77956736936     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2010.2059634     Document Type: Article

References (44)

1. S. Moncrieff, C. Dorai, and S. Venkatesh, "Affect computing in film through sound energy dynamics," ACM Multimedia, pp. 525-527, 2001.
2. H. B. Kang, "Analysis of scene context related with emotional events," ACM Multimedia, pp. 311-314, 2002.
3. H. B. Kang, "Emotional event detection using relevance feedback," in Proc. IEEE ICIP, 2003, pp. 721-724.
4. H. B. Kang, "Affective content detection using HMMs," ACM Multimedia, pp. 259-262, 2003.
5. M. Xu, L. T. Chia, and J. Jin, "Affective content analysis in comedy and horror videos by audio emotional event detection," in Proc. IEEE ICME, 2005, pp. 622-625.
6. M. Xu, S. Luo, and J. Jin, "Video adaptation based on affective content with MPEG-21 DIA framework," in Proc. IEEE SCIISP, 2007, pp. 386-390.
7. M. Xu, J. Jin, and S. H. Luo, "Hierarchical movie affective content analysis based on arousal and valence features," ACM Multimedia, pp. 677-680, 2008.
8. L. Lu, D. Liu, and H. J. Zhang, "Automatic mood detection and tracking of music audio signals," IEEE Trans. Audio, Speech, Language Process., vol.14, no.1, pp. 5-18, Jan. 2006.
9. A. Hanjalic, "Extracting moods from pictures and sounds: Towards truly personalized TV," IEEE Signal Process. Mag., vol.23, no.2, pp. 90-100, 2006.
10. A. Hanjalic and L. Q. Xu, "Affective video content representation and modeling," IEEE Trans. Multimedia, vol.7, no.1, pp. 143-154, Feb. 2005.
11. A. Hanjalic, "Adaptive extraction of highlights from a sport video based on excitement modeling," IEEE Trans. Multimedia, vol.7, no.6, pp. 1114-1122, Dec. 2005.
12. S. Arifin and P. Y. K. Cheung, "A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional information," ACM Multimedia, pp. 68-77, 2007.
13. S. Arifin and P. Y. K. Cheung, "User attention based arousal content modeling," in Proc. IEEE ICIP, 2006, pp. 433-436.
14. H. L.Wang and L. F. Cheong, "Affective understanding in film," IEEE Trans. Circuits Syst. Video Technol., vol.16, no.6, pp. 689-704, Jun. 2006.
15. C. H. Chan and G. J. F. Jones, "Affect-based indexing and retrieval of films," ACM Multimedia, pp. 427-430, 2005.
16. Y. H. Yang, Y. F. Su, Y. C. Lin, and H. H. Chen, "Music emotion recognition: the role of individuality," in Proc. ACM Int. Workshop Human-Centered Multimedia, 2007, pp. 13-22.
17. [Online]. Available: http://www.google.cn/music/songscreener
18. P. J. Lang, "The network model of emotion: motivational connections," in Advances in Social Cognition. Hillsdale, NJ: Lawrence Erlbaum Assoc., 1993, pp. 109-133.
19. J. Russell and A. Mehrabian, "Evidence for a three-factor theory of emotions," J. Res. Personality, vol.11, pp. 273-294, 1977.
20. H. Schlosberg, "Three dimensions of emotion," Psycholog. Rev., vol.61, no.2, pp. 81-88, Mar. 1954.
21. K. Isbister, K. Hook, M. Sharp, and J. Laaksolahti, "The sensual evaluation instrument: Developing an affective evaluation tool," in Proc. SIGCHI Conf. Human Factors in Computing Syst., 2006, pp. 1163-1172.
22. A. J. Smola and B. Scholkopf, "A tutorial on support vector regression," Stat. Computing, vol.14, no.3, pp. 199-222, Aug. 2004.
23. S. L. Zhang, Q. Huang, Q. Tian, S. Jiang, and W. Gao, "i.MTV-An integrated system for MTV affective analysis," ACM Multimedia, pp. 985-986, 2008.
24. S. L. Zhang, Q. Tian, S. Q. Jiang, Q. M. Huang, and W. Gao, "Affective MTV analysis based on arousal and valence features," in Proc. IEEE ICME, 2008, pp. 1369-1372.
25. A. Mehrabian, "Pleasure-arousal-dominance: A general framework for describing and measuring individual difference for describing and measuring individual differences in temperament," Current Psychol., vol.14, no.4, pp. 261-292, Dec. 1996.
26. N. Dimitrova, L. Agnihotri, R. Jasinschi, J. Zimmerman, G. Marmaropoulos, T. McGee, and S. Dagtas, "Scouting demonstration: smart content selection and recording," ACM Multimedia, pp. 499-500, 2000, BVideo.
27. B. Adams and S. Venkatesh, "An embedded suggestive interface for making home videos," in Proc. IEEE ICME, 2006, pp. 2025-2028.
28. C. Dorai, R. Farrell, A. Katriel, G. Kofman, Y. Li, and Y. Park, "BMAGICAL demonstration: System for automated metadata generation for instructional content," ACM Multimedia, pp. 491-492, 2006.
29. R. Jana, J. Murray, and C. W. Rice, "IMS-TV: An IMS-based architecture for interactive, personalized IPTV," IEEE Commun. Mag., vol.46, no.11, pp. 156-163, Nov. 2008.
30. D. O. Sullivan, B. Smyth, D. C.Wilson, K. McDonald, and A. Smeaton, "Improving the quality of the personalized electronic program guide," User Modeling and User-Adapted Interaction, vol.14, no.1, pp. 5-36, Feb. 2004.
31. L. Chen and K. Sycara, "WebMate: Personal agent for browsing and searching," in Proc. Int. Conf. Autonomous Agents, 1998, pp. 132-139.
32. B. Magnini and C. Strapparava, "User modeling for news web sites with word sense based techniques," User Modeling and User-Adapted Interaction, vol.14, no.2-3, pp. 239-257, Jun. 2004.
33. S. Venkatesh, B. Adams, D. Phung, C. Dorai, R. G. Farrell, L. Agnihotri, and N. Dimitrova, ""You Tube and I Find"-Personalizing multimedia content access," Proc. IEEE, vol.96, no.4, pp. 697-711, Apr. 2008.
34. N. Sebe and Q. Tian, "Personalized multimedia retrieval: the new trend?," in Proc. ACM Int. Workshop MIR, 2007, pp. 299-306.
35. L. Agnihotri, J. Kender,N. Dimitrova, and J. Zimmerman, "Framework for personalized multimedia summarization," in Proc. ACM Int. Workshop MIR, 2005, pp. 31-38.
36. G. I. Webb, M. J. Pazzani, and D. Billsus, "Machine learning for user modeling," User Modeling and User-Adapted Interaction, pp. 19-29, Mar. 2001.
37. H. Hirsh, C. Basu, and B. Davison, "Learning to personalize," Commun. ACM, vol.43, no.8, pp. 102-106, Aug. 2000.
38. T. Zhang and C. C. J. Kuo, "Audio content analysis for online audiovisual data segmentation and classification," IEEE Trans. Audio, Speech, Language Process., vol.9, no.4, pp. 441-457, Dec. 2001.
39. N. C. Maddage, C. S. Xu, M. S. Kankanhalli, and X. Shao, "Contentbased music structure analysis with applications to music semantics understanding," ACM Multimedia, pp. 112-119, 2004.
40. D. Bordwell and K. Thompson, Film Art: An Introduction, 7th ed. New York: McGraw-Hill, 2004.
41. H. Zettl, Sight Sound Motion: Applied Media Aesthetics, 3rd ed. Belmont, CA: Wadsworth, 1998.
42. C. J. C. Burges, "A tutorial on support vector machine for pattern recognition," Data Mining Knowledge Discovery, pp. 121-167, Jun. 1998.
43. G. P. Nguyen and M. Worring, "Optimization of interactive visual similarity based search," ACM Trans. Multimedia Comput., Commun., Appl., vol.4, no.1, Jan. 2008.
44. X. L. Liu, T. Mei, X. S. Hua, B. Yang, and H. Q. Zhou, "Video collage," ACM Multimedia, pp. 461-462, 2007.