A survey of content-based video retrieval

Journal of Computer Science

Volumn 4, Issue 6, 2008, Pages 474-486

  Geetha, P ^a   Narayanan, Vasumathi ^b  

^a SATHYABAMA UNIVERSITY   (India)

^b St Martins Engg College   (India)

Author keywords

CBVR; Cut; Edge detection; Feature extraction; Key frame; Object motion; RGB; Shot Segmentation; Similarity search; SOM; Threshold

Indexed keywords

EID: 56149124617     PISSN: 15493636     EISSN: None     Source Type: Journal    
DOI: 10.3844/jcssp.2008.474.486     Document Type: Article

References (80)

1. Bing Han, Xinbo Gao, Hongbing Ji, 2005. A shot boundary detection method for news video based on rough-fuzzy sets. Int. J. Inform. Technol., 11: 101-111. www.icis.ntu.edu.sg/scs-ijit/117/117_11.pdf
2. Gao, X. and X. Tang, 2002. Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing. IEEE Trans. Circuits Syst. Video Technol., 12: 765-776. Doi: 10.1109/ TCSVT.2002.800510
3. Gao, X. and X. Tang, 2000. Automatic parsing of news video based on cluster analysis. In: Proceedings of 2000 Asia Pacific Conference on Multimedia Technology and Applications, Kaohsiung, Taiwai, China, Dec. 17-19, pp: 17-19. https://dspace.lib.cuhk.edu.hk/handle/2006/5923
4. Han Bing, Gao Xin-bo, Ji Hong-bing, 2003. An efficient algorithm of gradual transition for shot boundary segmentation. 3rd International Symposium on Multispectral Image Processing and Pattern recognition (MIPPR'03), Beijing, 9: 956-961. http://see.xidian.edu.cn/faculty/hbji/ eng.htm
5. Seung-Hoon Han, Kuk-Jin Yoon, and In So Kweon, 2000. A new technique for shot detection and key frames selection in histogram space. In: 12th Workshop on Image Processing and Image Understanding, pp: 475-479. http://rcv.kaist.ac.kr/publication/file/domestic_conference/ 18_SeungHoonHan_IPIU2000.pdf
6. Zhang, H.J., A. Kankanhalli and S.W. Smoliar, 1993. Automatic partitioning of full motion video. ACM Multimedia Syst., 1: 10-28. Doi: 10. 1007/BF01210504, http://portal.acm.org/citation.cfm?id=173856.173858
7. Hampapur, A., R. Jain and T.E. Weymouth, 1995. Production model based digital video segmentation. Multimedia Tools Appl., 1: 9-46. Doi: 10.1007/BF01261224
8. Bilge, G. and A.M. Tekalp, 1998. Content based video abstraction. In: Proceedings of International Conference on Image Proceeding, Oct. 4-7, 3: 128-132, Doi: 10.1109/ICIP.1998.727150
9. Song, S.M., T.H. Kwon, W.M. Kim, H. Kim and B.D. Rhee, 1998. On detection of gradual scene changes for parsing of video data. Proc. SPIE Storage Retrieval Image Video Database, 3312: 404-413. DOI:10.1117/ 12.298449
10. John, S., Boreczky and D. Lynn, 1998. A hidden markov model framework for video segmentation using audio an image features. In: Proceedings of IEEE International conference on Acoustics, Speech and Signal Processing, May 12-15, 6: 3741-3744. Doi: 10.1109/ICASSP.1998.679697
11. Arman, F., A. Hsu and M.Y. Chiu, 1994. Image processing on encoded video sequences. ACM Multimedia Syst., 1: 211-219. Doi: 10.1007/BF01268945
12. Jain, R., 1988. Dynamic vision. In: 9th International Conference on Pattern Recognition, Nov. 14-17, 1: 226-235. Doi: 10.1109/ICPR.1988.28212
13. Zabih, R., J. Miller and K. Mai, 1995. A feature-based algorithm for detecting and classifying scene breaks. In: Proceedings of the 3rd ACM International Conference on Multimedia, pp: 189-200. http://doi.acm.org/ 10.1145/217279.215266
14. Zhang, H.J., A. Kankanhalli and S.W. Smoliar, 2001. Automatic partitioning of full-motion video. Readings Multimedia Comput. Network.ing, 321-339. http://portal.acm.org/citation.cfm?id=383905
15. Shan Li, Moon-Chuen Lee, 2005. An improved sliding window method for shot change detection. Proceeding of the 7th IASTED International Conference Signal and Image Processing, Aug. 15-17, Honolulu, Hawaii, USA, pp: 464-468. http://appsrv.cse.cuhk.edu.hk/~sli/publication/ sip05.pdf
16. Gargi, U., R. Kasturi and S.H. Strayer, 2000. Performance characterization of video-shot-change detection methods. IEEE Trans. Circuits Syst. Video Technol., 10: 1-13, 2000. Doi: 10.1109/76.825852
17. Yeo, B.L. and B. Liu, 1995. Rapid scene analysis on compressed video. IEEE Trans. Circuits Syst. Video Technol., 5: 533-544, 1995. Doi: 10.1109/76.475896
18. O'Toole, C., A. Smeaton, N. Murphy and S. Marlow, 1999. Evaluation of automatic shot boundary detection on a large video suite. In: 2nd U.K. Conference Image Retrieval: The Challenge of Image Retrieval, Feb. 25-26, Newcastle, U.K., pp: 1-12. http://doras.dcu.ie/346/
19. Cabedo Sushil, Xavier Ubiergo and K. Bhattacharjee, 1998. Shot detection tools in digital video. In: Proceedings of Non-linear Model Based Image Analysis 1998, Springer Verlag, Glasgow, pp. 121-126, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.2365
20. Cernekova, Z., N. Nikolaidis and I. Pitas, 2006. Temporal video segmentation by graph partitioning. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, May 14-19, 2: 209-212, Doi: 10.1109/ICASSP.2006.1660316
21. Cernekova, Z., C. Nikou and I. Pitas, "Shot detection in video sequences using entropy-based metrics," In: Proceedings of IEEE International Conference on Image Processing, vol: 3, pp: 421-424, 24-28 June 2002. Doi: 10.1109/ICIP.20 02.1038995
22. Sarah V Porter, 2004. Video segmentation and indexing using motion estimation. Ph.D Thesis, University of Bristol, http://www.cs.bris.ac.uk/ Publications/pubs̄ter.jsp?id=2000202
23. Hauptmann, A.G., R. Jin and T.D. Ng, 2003. Video retrieval using speech and image information. In: SPIE Proceedings Series on Storage and Retrieval for Multimedia Databases, Jan. 20-24, 5021: 148-159. http://cat.inist.fr/?aModele=afficheN&cpsidt=15204368
24. Hafner, J., H.S. Sawhney, W. Equitz, M. Flickner and W. Niblack, 1995. Efficient color histogram indexing for quadratic form distance. IEEE Trans. Pattern Anal. Machine Intel., 17: 729-736. Doi: 10.1109/34.391417
25. Sato, T., T. Kanade, E.K Hughes and M.A. Smith, 1998. Video OCR for digital news archive. In: Proceedings of IEEE International Workshop on Content-Based Access of Image and Video Databases, Jan. 3rd, pp: 52-60, Doi: http://doi.ieeecomputersociety.org/10.1109/CAIVD.1998.646033
26. C. Colombo, A. Del Bimbo and P. Pala, 1999. Semantics in visual information retrieval. IEEE Multimedia, 6: 38-53. Doi: 10.1109/93.790610
27. Smeulders, A.W.M., M. Worring, S. Santini, A. Gupta and R. Jain, 2000. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Machine Intel., 22: 1349-1380. Doi: 10.1109/34.895972
28. Chong-Wah Ngo, Ting-Chuen Pong and Hong-Jiang Zhang, 2003. Motion analysis and segmentation through spatio-temporal slices processing. IEEE Trans. Image Process., 12: 341-355. Doi: 10.1109/TIP.2003.809020
29. Fablet, R., P. Bouthemy and P. Perez, 2000. Statistical motion-based video indexing and retrieval. In: Proceedings of International Conference on Content-based Multimedia Information Access, pp: 602-619. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.25.7728
30. Fan, J., A.K. Elmagarmid, X. Zhu, W.G. Aref and L. Wu, 2004. Classview: Hierarchical video shot classification, indexing and accessing. IEEE Trans. Multimedia, 6: 70-86. Doi: 10.1109/TMM.2003.819583
31. Hwanjo, Yu, Jiawei Han and Kevin Chen-Chuan Chang, 2004. PEBL: Web page classification without negative examples. IEEE Trans. Knowledge Data Eng., 16: 70-81. Doi: 10.1109/TKDE.2004. 1264823
32. Cao, Y., W. Tavanapong, K. Kim and J. Oh, 2003. Audio assisted scene segmentation for story browsing. Proc. Int. Conf. Image Video Retrieval, 2728: 446-455. Doi: 10.1007/3-540-45113-7_44
33. Tavanapong, W. and Junyu Zhou, 2004. Shot clustering techniques for story browsing. IEEE Trans. Multimedia, 6: 517-527. Doi: 10.1109/ TMM.2004.830810
34. Solomon Kullback, 1997. Information Theory and Statistics. 1st Edn., Courier Dover Publications, USA., ISBN 0486696847, 978-0486696843
35. Zhong, D., H. Zhang and S.F. Chang, 1996. Clustering methods for video browsing and annotation. In: Procedding of SPIE Storage and Retrieval for still Image and Video Database IV, 2670: 239-246. http://adsabs.harvard.edu/abs/1996SPIE.2670..239Z
36. Anil K. Jain, Richard C. Dubes, "Algorithms for clustering data", Published by Prentice Hall College Div, March 1988, ISBN: 013022278X, 978-0130222787
37. Zhang, H.J. and D. Zhong, "A scheme for visual feature based image indexing". In: Proceeding of SPIE Conference on Storage and Retrieval for Image and Video Databases III, vol. 2420, pp: 36-46, February 1995. http://citeseerx.ist.psu.edu/ showciting;jsessionid=EA38C101B5FA1D8EAA9F9F268750EE28?cid=1529801.
38. Ankush, M., "An overview of multimedia content-based retrieval strategies". Informatica, vol: 30, No: 3, pp: 347-356, October 2006, http://www.informatica.si/PDF/30-3/ 09_Mittal_An%20Overview%20of%20Multimedia%20Content.pdf.
39. Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Qian Huang, Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P., "Query by image and video content: The QBIC system," IEEE Computer, vol. 28, no. 9, pp: 23-32, Sep 1995. Doi: 10.1109/2.410146.
40. Chen, J.Y., C. Taskiran, E.J. Delp and C.A. Bouman, "ViBE: A new paradigm for video database browsing and search". In Proceedings of IEEE Workshop on Content-Based Access of Image and Video Libraries, pp: 96-100, 21 June 1998. 10.1109/IVL.1998.694510.
41. J. R. Smith and S.F. Chang, "VisualSEEk: A Fully Automated Content-Based Image Query System," In proceedings of fourth ACM International Conference on Multimedia, Boston, MA, pp: 87-98, November 1996. http://portal.acm.org/citation.cfm?id=244130.244151.
42. Chang, S.F., W. Chen, H.J. Meng, H. Sundaram and D. Zhong, "VideoQ: an automated content based video search system using visual cues". In Proceedings of 5th ACM International conference on Multimedia, pp: 313-324, November 9-13, 1997, http://portal.acm.org/ citation.cfm?id=266382.
43. Pentland, A., R.W. Picard and S. Sclaroff, 1996. Photobook: Content-based manipulation of image databases. Proc. IJCV, 18: 233-254. Doi: 10.1007/BF00123143
44. Picard, R.W. and T.P. Minka, 1995. Vision texture for annotation. Proc. ACM J. Multimedia Syst., 3: 3-14. http://portal.acm.org/ citation.cfm?id=203180
45. Ogle, V.E. and M. Stonebraker, 1995. Chabot: Retrieval from a relational database of images. Proc. IEEE Comput., 28: 40-48. Doi: 10.1109/2.410150
46. Mehrotra, S., Yong Rui, M. Ortega-Binderberger and T.S. Huang, 1997. Supporting content-based queries over images in MARS. In: Proceedings of IEEE International Conference on Multimedia Computing and Systems, Jun 3-6, Ottawa, Ont., Canada, pp: 632-633. Doi: 10.1109/MMCS.1997.609791
47. Bach, J.R., C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. Jain and C. Shu, 1996. Virage image search engine: An open framework for image image management. Proc. SPIE Conf. Storage Retrieval Still Image Video Databases, 2670: 76-87. Doi: 10.1117/12.234785
48. Ardizzone, E., La Cascia, M., "Automatic Video Database Indexing and Retrieval," Journal on Multimedia Tools and Applications, vol: 4, no. 1, pp. 29-56, January 1997, Doi 10.1023/A: 1009630331620
49. Ramin Zabih, Justin Miller, Kevin Mai, "Video Browsing Using Edges and Motion," in proceedings on IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp: 439-446, 18-20 June 1996, Doi: 10.1109/CVPR.1996.517109
50. Shearer, K., Bunke, H., Venkatesh, S., "Video indexing and similarity retrieval by largest common sub graph detection using decision trees", In proceedings of Pattern Recognition, vol: 34, No. 5, pp. 1075-1091, May 2001. Doi: 10.1016/S0031-3203(00)00048-0
51. Guttman, A., "R-trees: a dynamic index structure for spatial searching",in proceedings of ACM SIGMOD International Conference on Management of Data, vol:1, no: 2, pp:47-57, 1984, http://doi.acm.org/ 10.1145/602259.602266
52. White, D.A.; Jain, R., "Similarity indexing with the SS-tree", in Proceedings of the Twelfth International Conference on Data Engineering, pp: 516-523, 26 Feb-March 1 1996, Doi: 10.1109/ ICDE.1996.492202
53. Ju, S.X., Black, M.J., Minnerman, S. and Kimber D., " Analysis of Gesture and Action in Technical Talks for Video Indexing" In proceedings of IEEE computer society conference on Computer Vision and Pattern Recognition, pp: 595-601, 17-19 Jun 1997. http://portal.acm.org/ citation.cfm?id=79418 9.794452.
54. Krishnapuram, R., S. Medasani, S.H. Jung, Y. Choi and R. Balasubramaniam, "Content-based image retrieval based on a fuzzy approach." IEEE Trans. Knowledge and Data engineering, vol: 16, no: 10, pp: 1185-1199, October 2004, Doi: 10.1109/TKDE.2004.53.
55. B. E. Rogowitz, T. Frese, J. Smith, C. A. Bouman, and E. Kalin, "Percpetual image similarity experiments," in IS&T/SPIE Conf. on Human Vision and Electronic Imaging III, 1998. http://serv1.ist.psu.edu:8080/viewdoc/summary?doi=10.1.1.89.5054.
56. T.V. Papathomas, T.E. Conway, I.J. Cox, J. Ghosn, M.L. Miller, T.P. Minka, P.N. Yianilos, "Psychophysical studies of the performance of an image database retrieval system", in Proc. SPIE Conf. on Human Vision and Electronic Imaging, vol: 3299 pp. 591-602, July 1998, http://citeseerx.ist.psu.edu/ showciting;jsessionid=2C93B1E50B192C14F6ABBF5335C8B100?cid=61126
57. Mufit Ferman, A. Murat Tekalp, A., "Probabilistic analysis and extraction of video content", In Proceedings of International Conference on Image Processing, Vol: 2, pp: 91-95, 1999, Doi: 10.1109/ ICIP.1999.822861.
58. Naphade, M.R., Kristjansson, T., Frey, B., Huang, T.S., "Probabilistic multimedia objects (MULTIJECTS): a novel approach to video indexing and retrieval in multimedia systems", in Proceedings of International Conference on Image Processing, vol.3, pp: 536-540, October 1998. http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.53.8930
59. Vasconcelos, N. Lippman, A., "Towards semantically meaningful feature spaces for the characterization of video content ", in Proceedings of International Conference on Image Processing, Vol: 1, pp: 25-28, 26-29 October 1997, Doi: 10.1109/ICIP.1997.647375.
60. Hanjalic, A., 2005. "Adaptive extraction of highlights from a sport video based on excitement modeling." IEEE Trans. Multimedia, 7: 1114-1122, Doi: 10.1109/TMM.2005.858397
61. Rasheed, Z. Sheikh, Y. Shah, M., "On the use of computable features for film classification", IEEE Transactions on Circuits and Systems for Video Technology, Vol: 15, no: 1, pp: 52-64, Jan. 2005, Doi: 10.1109/ TCSVT.2004.839993(410) 1.
62. Gorkani, M.M.; Picard, R.W., "Texture orientation for sorting photos at a glance", in Proceedings of the 12th IAPR International Conference on Computer Vision & Image Processing., Pattern Recognition, Vol. 1, pp: 459-464, 9-13 October 1994, Doi: 10.1109/ ICPR.1994.576325.
63. David A. Forsyth, Jitendra Malik, Margaret M. Fleck, Hayit Greenspan, Thomas K. Leung, Serge Belongie, Chad Carson, Chris Bregler, "Finding Pictures of Objects in Large Collections of Images," Proceedings of the International Workshop on Object Representation in Computer Vision II, p.335-360, April 13-14, 1996. http://portal.acm.org/ citation.cfm?id=648641.
64. Torralba, A.B. and A. Oliva, "Semantic organization of scenes using discriminant structural templates". IEEE International Conference on Computer Vision, vol. 2, pp. 1253-1258, 1999. Doi: 10.1109/ ICCV.1999.790424.
65. Aditya Vailaya, Anil Jain, "Reject Option for VQ-Based Bayesian Classification," in proceedings of 15th International Conference on Pattern Recognition, Vol. 2, pp: 48-51, 2000, Doi:10. 1109/ ICPR.2000.906016
66. C.-W. Ngo, H.-J. Zhang, and T.-C. Pong, "Recent advances in content based video analysis," International Journal of Image and Graphics, vol. 1, no. 3, pp. 445-468, 2001. http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.17.9593.
67. Yi Wu, Y. Zhuang, and Y. Pan:, "Content-Based Video Similarity Model," In Proc. of the 8th ACM Int. Multimedia Conf. on Multimedia, USA, pp. 465-467, 2000, http://portal.acm.org/citation.cfm?id=376380.
68. Anil Jain, Aditya Vailaya, Wei Xiong., "Query by video clip," In Proceedings of Fourteenth International Conference on Pattern Recognition, vol.1. pp: 909-911, 16-20 Aug 1998, Doi: 10.1109/ ICPR.1998.711299.
69. Tan, Y.P. [Yap-Peng], Kulkarni, S.R.[Sanjeev R.], Ramadge, P.J.[Peter J.], "A Framework for Measuring Video Similarity and Its Application to Video Query By Example", in proc. IEEE International conference on image processing, vol.2, pp: 106-110, 1999, Doi: 10.1109/ ICIP.1999.822864
70. Zhong, D. [Di], Chang, S.F.[Shih-Fu], "An integrated approach for content-based video object segmentation and retrieval", IEEE transactions on circuits and video technology, vol.9, no. 8, pp. 1259-1268, December 1999, Doi: 10.1109/76.809160.
71. Li Zhao, Wei Qi, Stan Z. Li, S.Q.Yang, H.J. Zhang, "Key-frame Extraction and Shot Retrieval Using Nearest Feature Line (NFL)", in Proceedings of ACM international workshop on Multimedia, pp: 217-220, November 2000, http://portal.acm.org/citation.cfm?id=357942
72. Dagntas, S., Al-Khatib, W., Ghafoor, A., Kashyap, R.L., "Models for Motion-Based Video Indexing and Retrieval", IEEE Transactions on image processing, vol.9, No. 1, pp. 88-101, January 2000, Doi: 10.1109/ 83.817601
73. C. V. Jawahar, BalaKrishna Chennupati, Balamanohar Paluri and Nataraj Jammalamadaka, "Video Retrieval Based on Textual Queries", in Proceedings of the Thirteenth International Conference on Advanced Computing and Communications, Coimbatore, December 2005. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.61.9857.
74. Meshesha, M., C.V.Jawahar and A. Balasubramanian, "Searching in Document Images", in Proc. of the 4th Indian. Conference on Computer Vision, Graphics and Image. Processing (ICVGIP) pp: 622-627, 2004. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.61.8420.
75. Rath, T.M. and R. Manmatha, "Word image matching using dynamic time warping", In: Conference on Computer Vision and Pattern Recognition, pp: 521-527, 2003. ISBN: 0-7695-1900-8.
76. Yan, R., Jin, R., Hauptmann, A., "Multimedia search with pseudo-relevance feedback". In Proc. Int. Conf. on Image and Video Retrieval, pp: 238-247, 2003, http://cat.inist.fr/ ?aModele=afficheN&cpsidt=15567430.
77. R. Yan, A. Hauptmann, and R. Jin, "Negative Pseudo-Relevance Feedback in Content-based Video Retrieval", in proceedings of 11th ACM International Conference on Multimedia, pp. 343-346, 2003. http://portal.acm.org/citation.cfm?id=957013.957087.
78. Yan, R., Jin, R., Hauptmann, A., "Multimedia Search with Pseudo-Relevance Feedback," AAAI Spring Symposium Series on Intelligent Multimedia Knowledge Management, Palo Alto, CA, March 24-26, 2003. http://www.informedia.cs.cmu.edu/pubs/abstract.asp?id=162.
79. Yan, R., A. Hauptmann and R. Jin, "Pseudo-Relevance Feedback for Multimedia Retrieval", In Kluwer International series in video computing, 2003.www.visionbib.com/bibliography/applicat803.html.
80. Wu, L., C. Faloutsos, K.P. Sycara and T.R. Payne, "Multimedia queries by example and relevance feedback", in proc. IEEE Data Engineering Bulletin 24(3), 2001, http://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.7.8289.