Recent advances in content-based video analysis

International Journal of Image and Graphics

Volumn 1, Issue 3, 2001, Pages 445-468

  Ngo, Chong Wah ^a   Pong, Ting Chuen ^a   Zhang, Hong Jiang ^b  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

Semantic Analysis; Video Abstraction; Video Parsing; Video Retrieval

Indexed keywords

IMAGE SEGMENTATION; VIDEO RECORDING;

COMPUTATIONAL APPROACH; CONTENT-BASED VIDEO ANALYSIS; MOTION CHARACTERIZATION; SEMANTIC ANALYSIS; VIDEO ABSTRACTION; VIDEO PARSING; VIDEO RETRIEVAL; VIDEO-CONTENT ANALYSIS;

SEMANTICS;

EID: 33751547574     PISSN: 02194678     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0219467801000268     Document Type: Article

References (100)

1. M. Flickner, H. Sawhney,W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkhani, J. Hafner, D. Lee, D. Petkovic, D. Steele, and P. Yanker, \Query by image and video content: The QBIC system," IEEE Comput. 28(9), 23 (1995).
2. M. K. Mandal, F. Idris, and S. Panchanathan, \A critical evaluation of image and video indexing techniques in the compressed domain," Image and Vision Comput. 17, 513 (1999).
3. C. W. Ngo, T. C. Pong, and R. T. Chin, \A survey of video parsing and image indexing techniques in compressed domain," in Symposium on Image and Speech and Signal Processing and Robotics, Vol. 1, 1998, pp. 231{236.
4. N. V. Patel and I. K. Sethi, \Compressed video processing for cut detection," IEE Proc. Vision and Image and Signal Processing 143, 315 (1996).
5. Y. P. Tan, D. D. Saur, S. R. Kulkarni, and P. J. Ramadge, \Rapid estimation of camera motion from compressed video with application to video annotation," IEEE Trans. Circuits Syst. Video Technology 10(1), 133 (2000).
6. B. L. Yeo and B. Liu, \On the extraction of dc sequence from MPEG compressed video," IEEE Trans. Circuits Syst. Video Technology 5(6), 533 (1995).
7. Y. Zhong, H. J. Zhang, and A. K. Jain, \Automatic caption localization in compressed video," IEEE Trans. Pattern Analysis and Machine Intelligence 22(4), 385 (2000).
8. Y. Rui, T. S. Huang, S. Mehrotra, and M. Ortega, \A relevance feedback architecture for content-based multimedia information retrieval systems," in IEEE Workshop on Content-Based Access of Image and Video Libraries, 1997, pp. 82{89.
9. J. Huang, Z. Liu, and Y. Wang, \Integration of audio and visual information for content-based video segmentation," in Int. Conf. on Image Processing, Vol. 3, 1998, pp. 526{529.
10. Q. Huang, A. Puri, and Z. Liu, \Multimedia search and retrieval: New concepts and system implementation and application," IEEE Trans. Circuits Syst. Video Technology 10(5), 679 (2000).
11. M. R. Naphade and T. S. Huang, \Semantic video indexing using a probabilistic framework," in Int. Conf. on Pattern Recognition, Vol. 3, 2000, pp. 83{88.
12. Y. Rui, A. Gupta, and A. Acero, \Automatically extracting highlights for TV baseball programs," in ACM Multimedia, 2000.
13. G. Ahanger and T. D. C. Little, \A survey of technologies for parsing and indexing digital video," J. Visual Commun. Image Representation 7(1), 28 (1996).
14. P. Aigrain, H. J. Zhang, and D. Petkovic, \Content-based representation and retrieval of visual media: A state of the art review," Multimedia Tools Appl. 3, 179 (1996).
15. R. Brunnelli, O. Mich, and C. M. Modena, \A survey on the automatic indexing of video data," J. Visual Commun. Image Representation 10, 78 (1999).
16. S. F. Chang, T. S. Huang, Q. Huang, A. Puri, and B. Shahraray, \Multimedia search and retrieval," in Advances in Multimedia: Systems, Standards, and Networks, eds. B. Furht and T. Chen (Marcel Dekker, New York, 1999).
17. P. Correia and F. Pereira, \The role of analysis in content-based video coding and indexing," Signal Processing 66, 125 (1998).
18. F. Idris and S. Panchanathan, \Review of image and video indexing techniques," J. Visual Commun. Image Representation 8(2), 146 (1997).
19. H. J. Zhang, \Content-based video browsing and retreival," in Handbook of Multimedia Computing, ed. B. Furht (CRC Press, 1999), pp. 255{280.
20. B. S. Manjunath, T. S. Huang, A. M. Tekalp, and H. J. Zhang, \Introduction to the special issue on image and video processing for digital libraries," IEEE Trans. Image Processing 9(1), 1 (2000).
21. F. Pereira, S. F. Chang, R. Koenen, A. Puri, and O. Avaro, \Introduction to the special issue on object-based video coding and description," IEEE Trans. Circuits Syst. Video Technology 9(8), 1144 (1999).
22. W. Wolf, \Video libraries," Multimedia Systems 7(5), 349 (1999).
23. C. W. Ngo, \Analysis of spatio-temporal slice for video content representation," Ph.D. Thesis, Hong Kong University of Science &Technology, 2000.
24. H. J. Zhang, A. Kankanhalli, and S. W. Smoliar, \Automatic partitioning of fullmotion video," in ACM Multimedia System, Vol. 1, 1993, pp. 10{28.
25. R. Zabih, J. Miller, and K. Mai, \A feature-based algorithm for detecting and classifying scene breaks," in ACM Multimedia, Nov. 1995, pp. 189{200.
26. P. Bouthemy, M. Gelgon, and F. Ganansia, \A unified approach to shot change detection and camera motion characterization," IEEE Trans. Circuits Syst. Video Technology 9(7), 1030 (1999).
27. N. Vasconcelos and A. Lippman, \Statistical models of video structure for content analysis and characterization," IEEE Trans. Image Processing 9(1), 3 (2000).
28. L. F. Cheong, \Scene-based shot change detection and comparative evaluation," J. Comput. Vision and Image Understanding 79(2), 224 (2000).
29. U. Gargi, R. Kasturi, and S. H. Strayer, \Performance characterization of video-shotchange detection methods," IEEE Trans. Circuits Syst. Video Technology 10(1), 1 (2000).
30. V. Kobla, D. Dementhon, and D. Doermann, \Special effect edit detection using video trails: A comparison with existing techniques," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 1999, pp. 302{313.
31. R. Lienhart, \Comparison of automatic shot boundary detection algorithm," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 1999, pp. 290{301.
32. M. S. Drew, S. N. Li, and X. Zhong, \Video dissolve and wipe detection via spatio-temporal images of chromatic histogram differences," in Int. Conf. on Image Processing, Vol. 3, 2000, pp. 929{932.
33. W. A. C. Fernando, C. N. Canagarajah, and D. R. Bull, \Wipe scene change detection in video sequences," in Int. Conf. on Image Processing, Vol. 3, 1999, pp. 294{298.
34. W. A. C. Fernando, C. N. Canagarajah, and D. R. Bull, \Fade and dissolve detection in uncompressed and compressed video sequences," in Int. Conf. on Image Processing, Vol. 3, 1999, pp. 299{303.
35. R. A. Joyce and B. Liu, \Temporal segmentation of video using frame and histogram space," in Int. Conf. on Image Processing, 2000.
36. H. Kim, S.-J. Park, J. Lee, W. M. Kim, and S. M. Song, \Processing of partial video data for detection of wipes," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 2000, pp. 280{289.
37. Z. N. Li, Z. Tauber, and M. S. Drew, \Spatio-temporal joint probability images for video segmentation," in Int. Conf. on Image Processing, Vol. 2, 2000, pp. 295{298.
38. H. B. Lu, Y. J. Zhang, and Y. R. Yao, \Robust gradual scene change detection," in Int. Conf. on Image Processing, Vol. 3, 1999, pp. 304{308.
39. J. Nang, S. Hong, and Y. Ihm, \An efficient video segmentation scheme for MPEG video stream using macroblock information," in ACM Multimedia, 1999, pp. 23{26.
40. C. W. Ngo, T. C. Pong, and R. T. Chin, \Detection of gradual transitions through temporal slice analysis," in Computer Vision and Pattern Recognition, Vol. 1, 1999, pp. 36{41.
41. C. W. Ngo, T. C. Pong, and R. T. Chin, \A robust wipe detection algorithm," in Asian Conference on Computer Vision, Vol. 1, 2000, pp. 246{251.
42. S. B. Jun, K. Yoon, and H. Y. Lee, \Dissolve transition detection algorithm using spatio-temporal distribution of MPEG macro-block types," in ACM Multimedia, 2000.
43. S. W. Lee, Y. M. Kim, and S. W. Choi, \Fast scene change detection using direct feature extraction from MPEG compressed videos," IEEE Trans. Multimedia 2(4), 240 (2000).
44. W. Xiong and C. M. Lee, \Efficient scene change detection and camera motion annotation for video classification," J. Comput. Vision and Image Understanding 17(2), 161 (1998).
45. K. A. Hua and J. H. Oh, \Detecting video shot boundaries up to 16 times faster," in ACM Multimedia, 2000.
46. M. S. Drew, J. Wei, and Z. N. Li, \Illumination-invariant image retrieval and video segmentation," Pattern Recognition 32(8) (1999).
47. C. W. Ngo, T. C. Pong, H. J. Zhang, and R. T. Chin, \Motion characterization by temporal slice analysis," in Computer Vision and Pattern Recognition, Vol. 2, 2000, pp. 768{773.
48. M. Irani and P. Anandan, \Video indexing based on mosaic representations," Proc. IEEE 86, 905 (1998).
49. D. Zhong and S. F. Chang, \An integrated approach for content-based video object segmentation and retrieval," IEEE Trans. Circuits Syst. Video Technology 9(8), 1259 (1999).
50. S. F. Chang, W. Chen, H. J. Meng, H. Sundaram, and D. Zhong, \A fully automatic content-based video search engine supporting multi-object spatio-temporal queries," IEEE Trans. Circuits Syst. Video Technology 8(5), 602 (1998).
51. H. T. Ngyuyen, M. Worring, and A. Dev, \Detection of moving objects in video using a robust motion similarity measure," IEEE Trans. Image Processing 9(1), 137 (2000).
52. H. S. Sawhney and S. Ayer, \Compact representations of videos through dominant and multiple motion estimation," IEEE Trans. Pattern Analysis and Machine Intelligence 18(8), 814 (1996).
53. H. S. Chang, S. S. Sull, and S. U. Lee, \Efficient video indexing scheme for contentbased retrieval," IEEE Trans. Circuits Syst. Video Technology 9(8) (1999).
54. C.-Y. Chang, A. A. Maciejewski, and V. Balakrishnan, \Eigendecomposition-based analysis of video images," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 1999, pp. 186{191.
55. D. DeMenthon, V. Kobla, and D. Doermann, \Video summarization by curve simplification," in ACM Multimedia, 1998, pp. 211{218.
56. A. M. Ferman and A. M. Tekalp, \Efficient filtering and clustering methods for temporal video segmentation and visual summarization," J. Visual Commun. Image Representation 9(4), 336 (1998).
57. M. S. Drew and J. Au, \Video keyframe production by efficient clustering of compressed chromaticity signatures," in ACM Multimedia, 2000, pp. 365{368.
58. Y. Gong and X. Liu, \Video summarization using singular value decomposition," in Computer Vision and Pattern Recognition, Vol. 2, 2000, pp. 174{180.
59. A. Hanjalic and H. J. Zhang, \An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis," IEEE Trans. Circuits Syst. Video Technology 9(8), 1280 (1999).
60. S. Uchihashi and J. Foote, \Summarizing video using a shot importance measure and a frame-packing algorithm," in Int. Conf. on Acoustics and Speech and Signal Processing, Vol. 6, 1999, pp. 3041{3044.
61. S. Uchihashi, J. Foote, A. Girgensohn, and J. Boreczky, \Video manga: Generating semantically meaningful video summaries," in ACM Multimedia, 1999, pp. 383{392.
62. L. Zhao, W. Qi, S. Z. Li, S. Q. Yang, and H. J. Zhang, \Key-frame extraction and shot retrieval using nearest feature line (NFL)," in International Workshop on Multimedia Information Retrieval, 2000.
63. C. W. Ngo, T. C. Pong, H. J. Zhang, and R. T. Chin, \Motion-based video representation for scene change detection," in Int. Conf. Pattern Recognition, Vol. 1, 2000, pp. 827{830.
64. H. J. Zhang, J. Y. A. Wang, and Y. Altunbasak, \Content-based video retrieval and compression: A unified solution," in Int. Conf. on Image Processing, 1997, pp. 13{16.
65. J. Y. A. Wang and E. H. Adelson, \Representing moving images with layers," IEEE Trans. Image Processing 3(5), 625 (1994).
66. B. Erol and F. Kossentini, \Automatic key video object plane selection using the shape information in the MPEG-4 compressed domain," IEEE Trans. Multimedia 2(2), 129 (2000).
67. B. Gunsel, A. M. Tekalp, and P. J. L. Beek, \Content-based access to video objects: Temporal segmentation, visual summarization and feature extraction," Signal Processing 66, 261 (1998).
68. C. Kim and J. N. Hwang, \An integrated scheme for object-based video abstraction," in ACM Multimedia, 2000.
69. D. W. Jacobs, D. Weinshall, and Y. Gdalyahu, \Classification with nonmetric distances: Image retrieval and class representation," IEEE Trans. Pattern Analysis and Machine Intelligence 22(6), 583 (2000).
70. S. Santini and R. Jain, \Similarity matching," IEEE Trans. Pattern Analysis and Machine Intelligence, 1999.
71. J. M. Corridoni and A. Del. Bimbo, \Structured representation and automatic indexing of movie information content," Pattern Recognition 31(12), 2027 (1998).
72. A. Hanjalic, R. L. Lagendijk, and J. Biemond, \Automated high-level movie segmentation for advanced video retrieval systems," IEEE Trans. Circuits Syst. Video Technology 9(5), 580 (1999).
73. H. Jiang, T. Lim, and H. J. Zhang, \Video segmentation with the assistance of audio content analysis," in Int. Conf. on Multimedia and Expo, 2000.
74. T. Lin and H. J. Zhang, \Automatic video scene extraction by shot grouping," in Int. Conf. on Pattern Recognition, Vol. 4, 2000, pp. 39{42.
75. Y. Rui, T. S. Huang, and S. Mehrotra, \Exploring video structure beyong the shots," in Proc. IEEE Conf. on Multimedia Computing and Systems, 1998, pp. 237{240.
76. Y. Rui, T. S. Huang, and S. Mehrotra, \Constructing table-of-content for videos," Multimedia Syst. 7(5), 359 (1999).
77. E. Sahouria and A. Zakhor, \Content analysis of video using principle components," IEEE Trans. Circuits Syst. Video Technology 9(8), 1290 (1999).
78. H. Sundaram and S. F. Chang, \Determining computable scenes in films and their structure using audio-visual memory models," in ACM Multimedia, 2000.
79. M. M. Yeung and B. L. Yeo, \Video visualization for compact presentation and fast browsing of pictorial content," IEEE Trans. Circuits Syst. Video Technology 7(5), 771 (1997).
80. M. J. Swain and D. H. Ballard, \Color indexing," Int. J. Comput. Vision 7(1), 11 (1991).
81. Y. Wu, Y. Zhuang, and Y. Pan, \Content-based video similarity model," in ACM Multimedia, 2000.
82. A. K. Jain, A. Vailaya, and W. Xiong, \Query by video clip," Multimedia Syst. 7(5), 368 (1999).
83. Y. P. Tan, D. D. Saur, S. R. Kulkarni, and P. J. Ramadge, \A framework for measuring video similarity and its application to video query by example," in IEEE Int. Conf. on Image Processing, 1999.
84. P. Bouthemy and R. Fablet, \Motion characterization from temporal co-occurrences of local motion-based measures for video indexing," in Int. Conf. on Pattern Recognition, 1998, pp. 905{908.
85. R. Fablet, P. Bouthemy, and P. Perez, \Statistical motion-based video indexing and retrieval," in Int. Conf. on Content-based Multimedia Info. Access, 2000, pp. 602{619.
86. J. Lee and B. W. Dickinson, \Hierarchical video indexing and retrieval for subbandcoded video," IEEE Trans. Circuits Syst. Video Technology 10(5), 824 (2000).
87. Z. Lei and Y. T. Lin, \3D shape inferencing and modeling for video retrieval," J. Visual Commun. Image Representation 11(1), 41 (2000).
88. R. Nelson and R. Polana, \Qualitative recognition of motion using temporal texture," CVGIP: Image Understanding 56(1), 78 (1992).
89. M. O. Szummer, \Temporal texture modeling," Ph.D. Thesis, MIT, 1995.
90. F. Liu and R. W. Picard, \Finding periodicity in space and time," in Int. Conf. on Computer Vision, 1998, pp. 376{383.
91. S. Daffgtas, W. A. Khatib, A. Ghafoor, and R. L. Kashyap, \Models for motion-based video indexing and retreival," IEEE Trans. on Image Processing 9(1), 88 (2000).
92. W. Zhou, A. Vellaikal, and C.-C. Jay Kuo, \Rule-based video classification system for basketball video indexing," in International Workshop on Multimedia Information Retrieval, 2000.
93. B. Adams, C. Dorai, and S. Venkatesh, \Study of shot length and motion as contributing factors to movie tempo," in ACM Multimedia, 2000.
94. N. Babaguchi, Y. Kawai, Y. Yasugi, and T. Kitahashi, \Linking live and replay scenes in broadcasted sport video," in International Workshop on Multimedia Information Retrieval, 2000.
95. C. Colombo, A. D. Bimbo, and P. Pala, \Semantics in visual information retrieval," IEEE Multimedia 6(3) (1999).
96. A. Hanjalic, R. L. Lagendijk, and J. Biemond, \Semi-automatic news analysis and indexing and classification system based on topics preselection," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 1999, pp. 86{97.
97. Q. Huang, Z. Liu, and A. Rosenberg, \Automatic semantic structure reconstruction and representation generation for broadcast news," in SPIE Proc. Storage and Retrieval for Image and Video Database VII, 1999, pp. 50{62.
98. E. Stringa and C. S. Regazzoni, \Real time video-shot detection for scene surveillance applications," IEEE Trans. Image Processing 9(1) (2000).
99. S. F. Chang, W. Chen, and H. Sundaram, \Semantic visual templates: Linking visual features to semantics," in Int. Conf. on Image Processing, 1998.
100. H. Li, D. Doermann, and O. Kia, \Automatic text detection and tracking in digital video," IEEE Trans. Image Processing 9(1), 147 (2000).