Spatiotemporal saliency for video classification

Signal Processing: Image Communication

Volumn 24, Issue 7, 2009, Pages 557-571

  Rapantzikos, Konstantinos ^a   Tsapatsoulis, Nicolas ^b   Avrithis, Yannis ^a   Kollias, Stefanos ^a  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b CYPRUS UNIVERSITY OF TECHNOLOGY   (Cyprus)

Author keywords

Spatiotemporal visual saliency; Video classification

Indexed keywords

CLASSIFICATION PERFORMANCE; COMPUTER VISION APPLICATIONS; CONSPICUITY; HETEROGENEOUS FEATURES; HUMAN VISUAL ATTENTION; OPTIMIZATION PROCESS; SALIENCY DETECTION; SALIENCY MEASURE; SALIENT REGIONS; SPATIOTEMPORAL SALIENCY; SPATIOTEMPORAL VISUAL SALIENCY; SPATIOTEMPORAL VOLUME; VIDEO CLASSIFICATION; VIDEO SEQUENCES; VISUAL INFORMATION;

COMPUTER VISION; VIDEO RECORDING; VISUAL COMMUNICATION;

COMPUTER APPLICATIONS;

EID: 67849088557     PISSN: 09235965     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.image.2009.03.002     Document Type: Article

References (52)

1. A. Monnet, A. Mittal, N. Paragios, V. Ramesh, Background modelling and subtraction of dynamic scenes, IEEE Int. Conf. Comput. Vision (ICCV'03), vol. 2, 2003, pp. 1305-1313.
2. Torralba A. Contextual influences on saliency. In: Itti L., Rees G., and Tsotsos J. (Eds). Neurobiology of Attention (2005), Academic Press/Elsevier, New York 586-593
3. Torralba A. Contextual priming for object detection. Int. J. Comput. Vision 53 2 (2003) 169-191
4. Treisman A.M., and Gelade G. A feature integration theory of attention. Cognitive Psychol. 12 1 (1980) 97-136
5. Tikhonov A.N., and Arsenin V.Y. Solution of Ill-Posed Problems (1977), W. H. Winston, Washington DC
6. B.A. Draper, A. Lionelle, Evaluation of selective attention under similarity transforms, International Workshop on attention and performance in computer vision (WAPCV'03), April 2003.
7. Koch C., and Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4 4 (1985) 219-227
8. P. Dollar, V. Rabaud, G. Cottrell, S. Belongie, Behavior recognition via sparse spatio-temporal features, in: Second Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65-72.
9. Heeger D.J. Optical flow using spatiotemporal filters. Int. J. Comput. Vision (1988) 279-302
10. Walther D., and Koch C. Modeling attention to salient proto-objects. Neural Networks 19 (2006) 1395-1407
11. Hering E. Outlines of a Theory of the Light Sense (1964), Harvard Univ. Press, Cambridge, Mass.
12. Sahouria E., and Zakhor A. Content analysis of video using principal components. IEEE Trans. Circuits Syst. Video Technol. 9 8 (1999)
13. Kandel E.R., Schwartz J.H., and Jessell T.M. Essentials of Neural Science and Behavior (1995), Appleton & Lange, Stamford, Connecticut
14. Stentiford F. An Attention Based Similarity Measure With Application to Content-Based Information Retrieval (2003), SPIE Electronic Imaging, Santa Clara
15. Li F.-F., VanRullen R., Koch C., and Perona P. Rapid natural scene categorization in the near absence of attention. Proc. Natl. Acad. Sci. 99 (2002) 8378-8383
16. Hamker F.H. A dynamic model of how feature cues guide spatial attention. Vision Res. 44 (2004) 501-521
17. H. Okamoto, Y. Yasugi, N. Babaguchi, T. Kitahashi, Video clustering using spatio-temporal image with fixed length, ICME'02, 2002, pp. 2002-2008.
18. H. Schneiderman, T. Kanade, Probabilistic modeling of local appearance and spatial relationships for object recognition. IEEE Conf. Comput. Vision Pattern Recognition, July 1998, pp. 45-51.
19. Zhang H.J., Kankanhalli A., and Smoliar S.W. Automatic partitioning of full-motion video. Multimedia Syst. 1 (1993) 10-28
20. Laptev I., and Lindeberg T. Space-time interest points. Proc. ICCV'03 (2003), Nice, France 432-443
21. Tsotsos J.K., Culhane S.M., Winky Y.K.W., Yuzhong L., Davis N., and Nuflo F. Modelling visual attention via selective tuning. Artif. Intell. 78 1 (1995) 507-545
22. K. Rapantzikos, N. Tsapatsoulis, Enhancing the robustness of skin-based face detection schemes through a visual attention architecture, in: Proceedings of the IEEE International Conference on Image Processing (ICIP), September 2005.
23. K. Rapantzikos, N. Tsapatsoulis, On the implementation of visual attention architectures, Tales of the Dissapearing Computer, Santorini, June 2003.
24. K. Rapantzikos, N. Tsapatsoulis, Y. Avrithis, spatiotemporal visual attention architecture for video analysis, in: Proceedings of IEEE International Workshop On Multimedia Signal Processing (MMSP'04), 2004.
25. Rapantzikos K., Tsapatsoulis N., Avrithis Y., and Kollias S. A bottom-up spatiotemporal visual attention model for video analysis. IET Image Process. 1 2 (2007) 237-248
26. K. Rapantzikos, Y. Avrithis, An enhanced spatiotemporal visual attention model for sports video analysis, International Workshop on Content-based Multimedia indexing (CBMI'05), June 2005.
27. K. Rapantzikos, M. Zervakis, Robust optical flow estimation in MPEG sequences, IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2005.
28. K.G. Derpanis, J.M. Gryn, Three-Dimensional nth derivative of gaussian separable steerable filters, ICIP'05, vol. 3, 2005, pp. 553-556.
29. Itti L. Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans. Image Process. 13 10 (2004) 1304-1318
30. Itti L., and Koch C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40 (2000) 1489-1506
31. Itti L., Koch C., and Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20 11 (1998) 1254-1259
32. L. Itti, P. Baldi, A Principled Approach to Detecting Surprising Events in Video, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2005, pp. 631-637.
33. Zelnik-Manor L., and Irani M. Statistical analysis of dynamic actions. IEEE Trans. Patt. Anal. Mach. Intell. 28 9 (2006) 1530-1535
34. Riedmiller M. Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithms. Int. J. Comput. Stand Interfaces, Special Issue Neural Networks 16 (1994) 265-278
35. Riesenhuber M., and Poggio T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2 11 (1999) 1019-1025
36. Ristivojević M., and Konrad J. Space-time image sequence analysis: object tunnels and occlusion volumes. IEEE Trans. Image Process. 15 (2006) 364-376
37. Black M.J., and Anandan P. The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. CVIU 63 1 (1996) 75-104
38. Oliver N.M., Rosario B., and Pentland A.P. A Bayesian computer vision system for modelling human interactions. IEEE Trans. Pattern Anal. Mach. Intell. 22 8 (2000)
39. O. Boiman, M. Irani, Detecting irregularities in images and in video, IEEE International Conference on Computer Vision (ICCV), October 2005.
40. Nobuyuki O. A threshold selection method from gray level histograms. IEEE Trans. Syst., Man Cybern. SMC-9 1 (1979)
41. P. Groves, P. Bajcsy, Methodology for hyperspectral band and classification model selection, IEEE Workshop on Advances in Techniques for Analysis of Remotely sensed data, October 2003, pp. 120-128.
42. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification (2001), Wiley, New York
43. S. Corchs, G. Ciocca, R. Schettini, Video summarization using a neurodynamical model of visual attention, Multimedia Signal Processing (MMSP'04), October 2004, pp. 71-74.
44. Avraham T., and Lindenbaum M. Attention-based dynamic visual search using inner-scene similarity: algorithms and bounds. IEEE Trans. Pattern Anal. Mach. Intell. 28 2 (2006) 251-264
45. T.J. Williams, B.A. Draper, An Evaluation of Motion in Artificial Selective Attention, IEEE Conf. Comput. Vision Pattern Recognition (CVPRW'05), June 2005, pp. 85.
46. U. Rutishauer, D. Walther, C. koch, P. Perona, Is bottom-up attention useful for object recognition?, CVPR'04, July 2004, pp. 37-44.
47. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
48. Freeman W.T., and Adelson E.H. The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13 9 (1991) 891-906
49. Y. Ke, R. Sukthankar, M. Hebert, Efficient visual event detection using volumetric features, Int. Conf. Comput. Vision, vol. 1, October 2005, pp. 166-17.
50. Y.-F. Ma, L. Lu, H.-J. Zhang, M. Li, A user attention model for video summarization", ACM Multimedia Conf., 2002, pp.533-542.
51. Ma Y.-F., Hua X.-S., Lu L., and Zhang H.-J. A generic framework of user attention model and its application in video summarization. IEEE Trans. Multimedia 7 5 (2005) 907-919
52. K. Rapantzikos, Y. Avrithis, S. Kollias, Dense saliency-based spatiotemporal feature points for action recognition, IEEE Conf. Comput. Vision Pattern Recognition, 2009, accepted.