Analysing user physiological responses for affective video summarisation

Displays

Volumn 30, Issue 2, 2009, Pages 59-70

  Money, Arthur G ^a   Agius, Harry ^a  

^a BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

Affective response; Personalisation; Physiological response; Video content semantics; Video summarisation

Indexed keywords

CONTENT BASED RETRIEVAL; INFORMATION THEORY; SEMANTICS; VIDEO STREAMING; VISUAL COMMUNICATION;

AFFECTIVE RESPONSE; PERSONALISATION; PHYSIOLOGICAL RESPONSE; VIDEO CONTENT SEMANTICS; VIDEO SUMMARISATION;

PHYSIOLOGICAL MODELS;

EID: 62149090672     PISSN: 01419382     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.displa.2008.12.003     Document Type: Article

References (67)

1. Agius H., Crockford C., and Money A.G. Geographic video content. In: Furht B. (Ed). Encyclopedia of Multimedia (1993), Springer, New York, NY, USA 48-65
2. K. Aizawa, D. Tancharoen, S. Kawasaki, T. Yamasaki, Efficient retrieval of life log based on context and content, in: Proc. 1st ACM Workshop on Continuous Archival and Retrieval of Personal Experiences (CARPE '04), New York, NY, USA, 15 October, 2004, pp. 22-31.
3. Allanson J., and Fairclough S.H. A research agenda for physiological computing. Interacting with Computers 16 5 (2004) 857-878
4. N. Babaguchi, Y. Kawai, T. Kitahashi, Generation of personalized abstract of sports video, in: Proc. IEEE International Conference on Multimedia and Expo (ICME '01), Tokyo, Japan, 22-25 August, 2001, pp. 800-803.
5. M. Barbieri, L. Agnihotri, N. Dimitrova, Video summarization: methods and landscape, in: J.R. Smith, S. Panchanathan, T. Zhang (Eds.), Internet Multimedia Management Systems IV, Proceedings of SPIE, SPIE, Bellingham, WA, USA, 2003, pp. 1-13.
6. Boiten F.A., Frijda N.H., and Wientjes C.J.E. Emotions and respiratory patterns: review and critical analysis. International Journal of Psychophysiology 17 (1994) 103-128
7. Bradley M., Greenwald M.K., and Hamm A.O. Affective picture processing. In: Birbaumer N., and Öhman A. (Eds). The Structure of Emotion (1993), Hogrefe & Huber Publishers, Toronto 48-65
8. Brown W.A., Corriveau D.P., and Monti P.M. Anger arousal by a motion picture: a methodological note. American Journal of Psychiatry 134 (1977) 930-931
9. Cacioppo J.T., Berntson G.G., Klein D.J., and Poehlmann K.M. The psychophysiology of emotion across the lifespan. Annual Review of Gerontology and Geriatrics 17 (1997) 27-74
10. Carlson N.R. Psychology of Behaviour. seventh ed. (2001), Allyn and Bacon, Boston, MA, USA
11. Cernekova Z., Pitas I., and Nikou C. Information theory-based shot cut/fade detection and video summarization. IEEE Transactions on Circuits and Systems for Video Technology 16 1 (2006) 82-91
12. Coicca G., and Schettini R. An innovative algorithm for key frame extraction in video summarization. Journal of Real-Time Image Processing 1 1 (2006) 69-88
13. Colombo C., Del Bimbo A., and Pala P. Retrieval of commercials by their semantic content: the semiotic perspective. Multimedia Tools and Applications 13 (2001) 73-91
14. S. Coppola, Lost in translation: momentum pictures, 2003.
15. G. de Silva, T. Yamasaki, K. Aizawa, Evaluation of video summarization for a large number of cameras in ubiquitous home, in: Proc. 13th ACM International Conference on Multimedia, Singapore, 6-11 November, 2005, pp. 820-828.
16. Detenber B.H., Simons R.F., and Roll B.G. 'em!: the effects of picture motion on emotional responses. Journal of Broadcasting & Electronic Media 42 1 (1998) 113-127
17. Dimitrova N. Context and memory in multimedia content analysis. IEEE Multimedia 11 3 (2004) 7-11
18. Ekman P., Levenson R.W., and Friesen W.V. Autonomic nervous system activity distinguished between emotion. Science 221 (1983) 1208-1210
19. M. Fletcher, Tea for Three, in Only Fools and Horses, Series 5: BBC Television, 1986.
20. Frazier T.W., Strauss M.E., and Steinhauer S.R. Respiratory sinus arrhythmia as an index of emotional response in young adults. Psychophysiology 41 1 (2004) 75-83
21. Fridja N. The Emotions (1986), Cambridge University Press, Cambridge
22. W. Friedkin, The Exorcist: Warner Bros., 1973.
23. M. Furini, V. Ghini, An audio-video summarisation scheme based on audio and video analysis, in: Proc. IEEE Consumer Communications and Networking Conference (CCNC '06), vol. 2, Las Vegas, NV, USA, 8-10 January, 2006, pp. 1209-1213.
24. T. Gates, Confidence man, in: Lost, Season 1, Channel 4 Television Corporation, 2004.
25. Gomez P., and Danuser B. Affective and physiological responses to environmental noises and music. International Journal of Psychophysiology 53 2 (2004) 93-103
26. Gomez P., Stahel W., and Danuser B. Respiratory responses during affective picture viewing. Biological Psychology 67 3 (2004) 359-373
27. Greenwald M.K., Cook E.W., and Lang P.J. Affective judgement and psychophysiological response: dimensional covariation in the evaluation of pictorial stimuli. Journal of Pyschophysiology 3 (1989) 51-64
28. Gross J.J., and Levenson R.W. Emotion elicitation using films. Cognition and Emotion 9 1 (1995) 87-108
29. Hagemann D., Naumann E., Maier S., Becker G., Lürken A., and Bartussek D. The assessment of affective reactivity using films: validity. Reliability, and Sex Differences, Personality and Individual Differences 26 (1999) 627-639
30. Hanjalic A. Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Transactions on Multimedia 7 6 (2005) 1114-1122
31. A. Hanjalic, L. Xu, User-oriented affective video content analysis, in: Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL 2001), Kauai, HI, 14 December, 2001, pp. 50-57.
32. Hanjalic A., and Xu L.Q. Affective video content representation and modeling. IEEE Transactions on Multimedia 7 1 (2005) 143-154
33. J.A. Healey, Wearable and automotive systems for affect recognition from physiology, Ph.D., Department of Electrical Engineering and Computer Science, MIT, Cambridge, MA, USA, 2000, pp. 158.
34. IMDb, Top 250 Films of All Time, http://www.imdb.com/chart/top, 2005.
35. A. Jaimes, T. Echigo, M. Teraguchi, F. Satoh, Learning personalized video highlights from detailed MPEG-7 metadata, in: Proc. IEEE International Conference on Image Processing (ICIP 2002), vol. 1, New York, NY, USA, 22-25 September, 2002, pp. 133-136.
36. Kramer A.F. Physiological metrics of mental workload: a review of recent progress. In: Damos D.L. (Ed). Multiple-Task-Performance (1991), Taylor & Francis, London 329-360
37. Lang A., Bolls P., Potter R., and Kawahara K. The effects of production pacing and arousing content on the information processing of television messages. Journal of Broadcasting and Electronic Media 43 4 (1999) 451-476
38. Lew M.S., Sebe N., Djeraba C., and Jain R. Content-based multimedia information retrieval: state of the art and challenges, ACM transactions on multimedia computing. Communications and Applications 2 1 (2006) 1-19
39. Li Y., Lee S., Yeh C., and Kuo C. Semantic retrieval of multimedia. IEEE Signal Processing Magazine 23 2 (2006) 79-89
40. R. L. Mandryk, K. M. Inkpen, Physiological indicators for the evaluation of co-located collaborative play, in: Proc. ACM Conference on Computer Supported Cooperative Work (CSCW '04), Chicago, IL, USA, 6-10 November, 2004, pp. 102-111.
41. McIntyre G., and Göcke R. The composite sensing of affect. In: Peter C., and Beale R. (Eds). Affect and Emotion In Human-Computer Interaction. LNCS (2007), Springer, Heidelberg, Germany (vol. 4868)
42. Money A.G., and Agius H. Video summarisation: A conceptual framework and survey of the state of the art. Journal of Visual Communication and Image Representation 19 2 (2008) 121-143
43. Nasoz F., Alvarez K., Lisetti C.L., and Finkelstein N. Emotion recognition from physiological signals for presence technologies. International Journal of Cognition 6 1 (2003) 1-32
44. Palomba D., and Stegagno L. Physiology, perceived emotion and memory: responding to film sequences. In: Birbaumer N., and Ohman A. (Eds). The Structure of Emotion: Psychophysiological, Cognitive, and Clinical Aspects (1993), Hogrefe & Huber, Toronto 158-168
45. Philippot P., Chapelle C., and Blairy S. Respiratory feedback in the generation of emotion. Cognition and Emotion 16 (2002) 605-627
46. Picard R.W. Affective Computing (1997), MIT Press, Cambridge, MA
47. Picard R.W., Vyzas E., and Healey J. Toward machine emotional intelligence: analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 10 (2001) 1175-1191
48. Piferi R.L., Kline K.A., Younger J., and Lawler K.A. An alternative approach for achieving cardiovascular baseline: viewing an aquatic video. International Journal of Psychophysiology 37 (2000) 207-217
49. Power M., and Dalgliesh T. Cognition and Emotion: From Order to Disorder (1998), Psychology Press, Guildford, Surrey
50. Y. Rui, S.X. Zhou, T.S. Huang, Efficient access to video content in a unified framework, in: Proc. IEEE International Conference on Multimedia Computing and Systems (ICMCS '99), vol. 2, Florence, Italy, 7-11 June, 1999, pp. 735-740.
51. Scheirer J., Fernandez P., Klein J., and Picard R.J. Frustrating the user on purpose: a step toward building an affective computer. Interacting with Computers 14 (2002) 93-118
52. X. Shao, C. Xa, M.S. Kankanhalli, A new approach to automatic music video summarization, in: Proc. IEEE International Conference on Image Processing (ICIP 2004) vol. 1, Barcelona, Spain, 24-27 October, 2004, pp. 625-628.
53. Simon H.A. Comments. In: Sydnor C., and Fiske S.T. (Eds). Affect and Cognition (1982), Lawrence Erlbaum Associates, Hillsdale, NJ 333-342
54. Simons R.F., Detenber B.H., Reiss J.E., and Shults C.W. Image motion and context: a between- and within-subject comparison. Psychophysiology 37 (2000) 706-710
55. Smeulders A.W.M., Worring M., Santini S., Gupta A., and Jain R. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 12 (2000) 1349-1380
56. Steinbeis N., Koelsch S., and Sloboda J.A. The role of harmonic expectancy violations in musical emotions: evidence from subjective, physiological, and neural responses. Journal of Cognitive Neuroscience 18 8 (2006) 1380-1393
57. Suziki J., Hiroshi N., and Hori T. Level of interest in video clips modulates event-related potentials to auditory probes. International Journal of Psychophysiology 55 1 (2004) 35-43
58. Y. Takahashi, N. Nitta, N. Babaguchi, Video summarization for large sports video archives, in: Proc. IEEE International Conference on Multimedia and Expo (ICME 2005), Amsterdam, The Netherlands, 6-8 July, 2005, pp. 1170-1173.
59. The Wachowski Brothers, The Matrix: Warner Bros., 1999.
60. B.L. Tseng, J.R. Smith, Hierarchical video summarization based on context clustering, in: J. R. Smith, S. Panchanathan, T. Zhang (Eds.), Internet Multimedia Management Systems IV, Proceedings of SPIE, SPIE, Bellingham, WA, USA, 2003, pp. 14-25.
61. VanDiest I., Winters W., Devriese S., Vercamst E., Han J.N., Van de Woestijne K.P., and Van den Bergh O. Hyperventilation beyond fight/flight: respiratory responses during emotional imagery. Psychophysiology 38 6 (2001) 961-968
62. van Reekum C.M., and Johnstone T. Psychophysiological responses to appraisal dimensions in a computer game. Cognition and Emotion 18 5 (2004) 663-688
63. H. Wang, H. Prendinger, T. Igarashi, Communicating emotions in online chat using physiological sensors and animated text, in: Proc. ACM Conference on Human Factors in Computing Systems (CHI '04), Vienna, Austria, 24-29 April, 2004, pp. 1171-1174.
64. Ward R.D., and Marsden P.H. Physiological responses to different Web page designs. International Journal of Human Computer Studies 59 1-2 (2003) 199-212
65. Winton W.M., Putnam L.E., and Krauss R.M. Facial and autonomic manifestations of the dimensional structure of emotion. Journal of Experimental Social Psychology 20 (1984) 195-216
66. C. Xu, J. Wang, K. Wan, Y. Li, L. Duan, Live sports detection based on broadcast video and Web-casting text, in: Proc. 14th ACM International Conference on Multimedia, Santa Barbara, CA, 23-27 October, 2006, pp. 221-230.
67. J. Zimmerman, N. Dimitrova, L. Agnihotri, A. Janevski, L. Nikolovska, Interface design for MyInfo: a personal news demonstrator combining Web and TV content, in: Proc. IFIP TC13 International Conference on Human-Computer Interaction (INTERACT), Zurich, Switzerland, 1-5 September, 2003.