Human-centered attention models for video summarization

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010

Volumn , Issue , 2010, Pages

  Li, Kaiming ^a,b   Zhang, Tuo ^a   Hu, Xintao ^a   Zhu, Dajiang ^a   Chen, Hanbo ^b   Jiang, Xi ^b   Deng, Fan ^a   Guo, Lei ^b   Faraco, Carlos ^c   Zhang, Degang ^a   Han, Junwei ^a   Hua, Xian Sheng ^d   Liu, Tianming ^b  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b University of Georgia ^*  (United States)

^c UNIVERSITY OF GEORGIA   (United States)

^d MICROSOFT RESEARCH ASIA   (China)

Author keywords

abstraction; attention models; brain imaging; summarization

Indexed keywords

ABSTRACTION; ATTENTION MODEL; ATTENTION MODELS; BRAIN IMAGING; BRAIN IMAGING TECHNIQUES; BRAIN RESPONSE; END USERS; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN BRAIN; HUMAN SUBJECTS; MULTIMEDIA CONTENTS; MULTIMEDIA STREAM; SUMMARIZATION; USER ATTENTION; USER EXPERIENCE; VIDEO BROWSING; VIDEO SUMMARIZATION; WORKING MEMORY SYSTEMS;

ABSTRACTING; BRAIN; INTERACTIVE COMPUTER SYSTEMS; LEARNING SYSTEMS; MAGNETIC RESONANCE IMAGING; RESONANCE; USER INTERFACES; VIDEO RECORDING;

BRAIN MODELS;

EID: 78650965620     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1891903.1891938     Document Type: Conference Paper

References (35)

1. Proceedings of the international workshop on TRECVID video summarization 2007, Augsburg, Bavaria, Germany.
2. T. Liu, H. Zhang, and F.Qi, "Perceptual Frame Dropping in Adaptive Video Streaming", IEEE ISCAS, Arizona, 2002.
3. Y.F. Ma, H.J. Zhang, "A Model of Motion Attention for Video Skimming", In Proceeding of IEEE ICIP, Rochester, New York, Sept. 22-25, 2002.
4. Y.F. Ma, H.J. Zhang, "Contrast-based Image Attention Analysis by Using Fussy Growing", In Proceeding of the 11th ACM International Conference on Multimedia, Berkeley, CA, USA, Nov. 2-8, 2003.
5. Y.F. Ma, L. Lu, H.J. Zhang, M.J. Li, "A User Attention Model for Video Summarization", In Proceeding of the 10th ACM International Conference on Multimedia, Juanles-Pins, France, Dec. 1-6, 2002.
6. A. F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and trecvid. In MIR '06: Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pages 321-330, 2006.
7. Logothetis NK. What we can do and what we cannot do with fMRI. Nature. 2008; 453(7197):869-78.
8. T. M. Mitchell, S. V. Shinkareva, A. Carlson, K.M. Chang, V. L. Malave, R. A. Mason, and M. A. Just, Predicting Human Brain Activity Associated with the Meanings of Nouns, Science, 320, 1191.
9. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., and Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. Science 303, 1634-1640. (Pubitemid 38338315)
10. David C. Van Essen, and Donna L. Dierker, Surface-Based and Probabilistic Atlases of Primate Cerebral Cortex, Neuron. 56, 2007.
11. Yufei MA, Xian-Sheng HUA, Lie LU, Hong-Jiang ZHANG. A Generic Framework of User Attention Model and Its Application in Video Summarization. IEEE Transactions on Multimedia Journal. 2005.
12. Jun Wang, et al., Brain State Decoding for Rapid Image Retrieval, ACM Multimedia 2009.
13. N. Bigdely-Shamlo, A. Vankov, R. Ramirez, and S. Makeig. Brain Activity-Based Image Classification from Rapid Serial Visual Presentation. IEEE Trans. on NSRE, 16(5):432-441, 2008.
14. A. Gerson, et al.. Cortically coupled computer vision for rapid image search. IEEE Trans. on NSRE, 14(2), 2006.
15. A. Kapoor, P. Shenoy, and D. Tan. Combining brain computer interfaces with vision for object categorization. In Proc. CVPR, 2008.
16. A. Gerson, L. Parra, and P. Sajda. Cortically coupled computer vision for rapid image search. IEEE Trans. on NSRE, 14(2):174-179, 2006.
17. http://www.fmrib.ox.ac.uk/fsl/
18. T. Syeda-Mahmood, D. Ponceleon, "Learning Video Browsing Behavior and Its Application in the Generation of Video Previews", In Proceedings of the 9th ACM International Conference on Multimedia, Ottawa, Canada, pp. 119-128, Canada, Sept.30-Oct.5, 2001.
19. H. Sundaram, L. Xie, S.F. Chang, "A Utility Framework for the Automatic Generation of Audio-visual Skims", In Proceedings of the 10th ACM International Conference on Multimedia, pp.189-198, France, Dec. 1- 6, 2002.
20. L.W. He, E. Sanocki, A. Gupta, et al., "Auto-summarization of Audio-Video Presentations", In Proceedings of the 7th ACM international conference on Multimedia, pp.489-498, Orlando, Florida, US, 1999.
21. M.G. Christel, M.A. Smith, C.R. Taylor, et al., "Evolving Video Skims into Useful Multimedia Abstractions", In Proceedings of the SIGCHI conference on Human factors in computing systems, 171-178, California, US, 1998.
22. D.B. Walther, E. Caddigan, L. Fei-Fei, D.M. Beck. Natural scene categories revealed in distributed patterns of activity in the human brain. Journal of Neuroscience, August 26, 2009, 29(34):10573-10581.
23. Uri Hasson, et al., Reliability of cortical activity during natural stimulation, Trends in Cognitive Sciences, vol. 14, Issue 1, January 2010, Pages 40-48.
24. Ying Li, et al., Video Content Analysis Using Multimodal Information: For Movie Content Extraction, Indexing and Representation, 2003, Springer Netherlands.
25. J. D. Meier, et al., Complex Organization of Human Primary Motor Cortex: A High-Resolution fMRI Study, J Neurophysiol 100: 1800-1812, 2008.
26. Fernández G, et al., Language mapping in less than 15 minutes: real-time functional MRI during routine clinical investigation. Neuroimage. 2001 Sep;14(3):585-94.
27. C.C. Faraco, D. Smith, J. Langley, Q. Zhao and L.S. Miller, Mapping the Working Memory Network using the OSPAN Task, NeuroImage. 47, Supplement 1, 2009, Page S105.
28. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. (July 2001). "Neurophysiological investigation of the basis of the fMRI signal." Nature 412 (6843): 150-7.
29. Harel, J., Koch C., and Perona, P., Graph-based visual saliency, in NIPS (2006).
30. Dudai, Y (2008) "Enslaving Central Executives: Toward A Brain Theory of Cinema". Projections 2(2): 21-42.
31. Baddeley A (2003). "Working memory: looking back and looking forward". Nature Reviews. Neuroscience 4 (10): 829-39.
32. Dudai, Y., and M. Carruthers. 2005. "The Janus Face of Mnemosyne. Memory: Some Systems in the Brain May Be Better Equipped to Handle the Future Than the Past." Nature 434 (7033): 567.
33. Baddeley, A. 2007. Working Memory, Thought, and Action. Oxford: Oxford University Press.
34. K.J. Friston, A.P. Holmes, K.J. Worsley, J.B. Poline, C. Frith, and R.S.J. Frackowiak. Statistical Parametric Maps in Functional Imaging: A General Linear Approach. Human Brain Mapping, 2:189-210, 1995.
35. Silvia Pfeiffer,Stephan Fischer and Wolfgang Effelsberg, Automatic Audio Content Analysis, Proc. ACM Multimedia 96, pp. 21-30, Bosten, MA, 1996.