Hierarchical affective content analysis in arousal and valence dimensions

Signal Processing

Volumn 93, Issue 8, 2013, Pages 2140-2150

  Xu, Min ^a,b   Xu, Changsheng ^b   He, Xiangjian ^a   Jin, Jesse S ^c   Luo, Suhuai ^c   Rui, Yong ^d  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

^b INSTITUTE OF AUTOMATION   (China)

^c UNIVERSITY OF NEWCASTLE   (Australia)

^d MICROSOFT RESEARCH   (United States)

Author keywords

Affective content detection; Mid level representation; Multiple modalities

Indexed keywords

CONDITIONAL RANDOM FIELDS(CRFS); CONTENT DETECTION; FUZZY C-MEAN CLUSTERING; HIERARCHICAL METHOD; HIERARCHICAL STRUCTURES; INDEPENDENCE ASSUMPTION; MID-LEVEL REPRESENTATION; MULTIPLE MODALITIES;

HIDDEN MARKOV MODELS; MATHEMATICAL MODELS;

CONTENT BASED RETRIEVAL;

EID: 84876282103     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2012.06.026     Document Type: Article

References (40)

1. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, H. Qian, B. Dom, M. Gorkani, J. Hafiner, D. Lee, D. Petkovic, D. Steele, and P. Yanker Query by image and video content: the {QBIC} System IEEE Computer Special Issue on Content-Based Retrieval 28 9 1995 23 32
2. J. Wei, Z.N. Liu, I. Gertner, A novel motion-based active video indexing method, Proceedings of IEEE International Conference on Multimedia Computing and System, 2 (1999) 460-465.
3. E. Ardizzo, M.L. Cascia, V.D. Gesu, C. Valenti, Content-based indexing of image and video databases by global and shape features, in: Proceedings of 13th International Conference on Patter Recognition, vol. 3, 1996, pp. 140-144.
4. A. Hanjalic Shot-boundary detection: unravelled and resolved IEEE Transactions on Circuits and Systems for Video Technology 12 2 2002 90 105
5. M. Wang, X.-S. Hua, J. Tang, and R. Hong Beyond distance measurement: constructing neighborhood similarity for video annotation IEEE Transactions on Multimedia 11 3 2009 465 476
6. M. Wang, X.-S. Hua, R. Hong, J. Tang, G.-J. Qi, and Y. Song Unified video annotation via multi-graph learning, in: IEE Transactions on Circuits and Systems for Video Technology 19 5 2009 733 746
7. L.Y. Duan, M. Xu, C. Xu, and Q. Tian A unified framework for semantic shot classification in sports video IEEE Transactions on Multimedia 7 6 2005 1066 1083
8. M. Xu, N.C. Maddage, C.-S. Xu, M. Kankanhalli, Q. Tian, Creating audio keywords for event detection in soccer video, Proceedings of International Conference on Multimedia & Expo, 2 (2003) 143-154.
9. S. Zhang, Q. Huang, Q. Tian, S. Jiang, W. Gao, i.MTV: an integrated system for MTV affective analysis, in: Proceedings of ACM International Conference on Multimedia, 2008, 985-986.
10. H.-B. Kang, Affective content detection using HMMs, in: Proceedings of the ACM Multimedia Conference, 2003, pp. 259-262.
11. A. Hanjalic, and L.Q. Xu Affective video content representation and modeling IEEE Transaction on Multimedia 7 1 2005 143 154
12. Z. Rasheed, Y. Sheikh, and M. Shah On the use of computable features for film classification IEEE Transactions on Circuits and Systems for Video Technology 15 1 2005 52 64
13. C. Chan, G.J.F. Jones, Affect-based indexing and retrieval of films, in: Proceedings of the ACM Multimedia Conference, 2005, pp. 427-430.
14. A. Hanjalic Extracting moods from pictures and sounds: towards truly personalized TV IEEE Signal Processing Magazine 23 2 2006 90 100
15. Y.-H. Chen, J.-H. Kuo, W.-T. Chu, J.-L. Wu, Movie emotional event detection based on music mood and video tempo, in: Proceedings of IEEE International Conference on Consumer Electronics, 2006, pp. 151-152.
16. H.L. Wang, and L.F. Cheong Affective understanding in film IEEE Transactions on Circuits and Systems for Video Technology 16 6 2006 689 704
17. S. Arifin, P.Y.K. Cheung, User attention based arousal content modeling, in: Proceedings of the International Conference on Image Processing, 2006, pp. 433-436.
18. S. Arifin, P.Y.K. Cheung, A computation method for video segmentation utilizing the pleasure-arousal-dominance emotional information, in: Proceedings of the ACM Multimedia Conference, 2007, pp. 68-77.
19. G. Irie, K. Hidaka, T.Satou, A. Kojima, T. Yamasaki, K. Aizawa, Latent topic driving model for movie affective scene classification, in: Proceedings of the ACM Multimedia Conference, 2009, 565-568.
20. M. Soleymani, J.J.M. Kierkels, G. Chanel, T. Pun, A bayesian framework for video affective representation, in: Proceedings of the International Conference on Affective Computing and Intelligent interaction, 2009.
21. M. Bradley, Emotions: essays on emotion theory, in: Lawrence Erlbaum, Hillsdale, NJ, Stephanie H.M. van Goozen, N.E. van de Poll, J. A. Sergeant, 1994, 97-134.
22. P. J. Lang, The network model of emotion: motivational connections, Perspectives on Anger and Emotion, Advances in Social Cognition, Psychology Press, in: Robert S. Wyer, Thomas K. Srull, (Eds.), 1993, pp. 109-133.
23. J.A. Russell, and A. Mehrabian Evidence for A Three-factor theory of emotions Journal of Research in Personality 11 3 1977 273 294
24. Z. Kasap, M. Benmoussa, P. Chaudhuri, and N. Magnenat-Thalmann Making them remember-emotional virtual characters with memory IEEE Computer Graphics and Applications 29 2 2009 20 29
25. T.S. Huang, M.A. Hasegawa-Johnson, S.M. Chu, Z. Zeng, and H. Tang Sensitive talking heads IEEE Signal Processing Magazine 26 4 2009 67 72
26. N. Sebe, M.S. Lew, Y. Sun, I. Cohen, T. Gevers, and T.S. Huang Authentic facial expression analysis Image and Vision Computing 25 12 2007 1856 1863
27. C. Shan, S. Gong, and P.W. McOwan Facial expression recognition based on local binary patterns: a comprehensive study Image and Vision Computing 27 6 2009 803 816
28. A. Khalili, C. Wu, H. Aghajan, Autonomous learning of user's preference of music and light services in smart home, in: Proceedings of the behavior monitoring and interpretation workshop at German AI Conference, 2009.
29. S. Zhang, Q. Tian, Q. Huang, W. Gao, S. Li, Utilizing affective analysis for efficient movie browsing, in: Proceedings of IEEE International Conference on Image Processing, 2009, 1853-1856.
30. J.C. Bezdek Pattern Recognition with Fuzzy Objective Function Algoritms 1981 Kluwer Academic Publishers Norwell, MA, USA
31. M. Xu, J.S. Jin, S. Luo, L. Duan, Hierarchical movie affective content analysis based on arousal and valence features, in: Proceedings of the ACM Multimedia Conference, 2008, pp. 677-680.
32. Nash Information Services, LLC, The Numbers - Movie Box Office Data, Film Stars, Idle Speculation , 2009.
33. P. Ekman Universals and cultural differences in the judgments of facial expressions of emotion Journal of Personality and Social Psychology 54 4 1987 712 717
34. P. Valdez, and A. Mehrabian Effects of color on emotions Journal of Experimental Psychology: General 123 4 1994 394 409
35. C. Plantinga, and G.M. Smith Passionate Views: Film Cognition, and Emotion 1999 The Johns Hopkins University Press
36. M. Xu, S. Luo, J.S. Jin, Affective content detection by using timing features and fuzzy clustering, in: Proceedings of the Advances in Multimedia Information Processing - PCM 2008, Lecture Notes in Computer Science, vol. 5353, 2008, pp. 685-692.
37. R.W. Picard Affective Computing 2000 The MIT Press Cambridge, MA
38. Z. Zeng, J. Tu, M. Liu, T.S. Huang, B. Pianfetti, D. Roth, and S. Levinson Audio-visual affective recognition, IEE Transaction on Multimedia 9 2007 424 428
39. E. Chong, and S. Zak An Introduction to Optimization 1996 Wiley New York
40. S.-F. Chang, D. Zhong, and R. Kumar Real-time content-based adaptive streaming of sports video Proceedings of the IEEE Workshop Content-Based Access to Video/Image Library 2001 pp. 139-146