Survey of compressed-domain features used in audio-visual indexing and analysis

Journal of Visual Communication and Image Representation

Volumn 14, Issue 2, 2003, Pages 150-183

  Wang, Hualu ^a   Divakaran, Ajay ^b   Vetro, Anthony ^b   Chang, Shih Fu ^a   Sun, Huifang ^b  

^a Columbia University ^*  (United States)

^b MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

Author keywords

Audio indexing; Compressed domain; Compression; Features; MPEG 1; MPEG 2; MPEG 7; Multimedia; Video indexing

Indexed keywords

COLOR IMAGE PROCESSING; COMPUTATIONAL COMPLEXITY; DECODING; SIGNAL ENCODING; VIDEO SIGNAL PROCESSING;

AUDIO INDEXING; VIDEO INDEXING;

IMAGE COMPRESSION;

EID: 0038136880     PISSN: 10473203     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1047-3203(03)00019-1     Document Type: Review

References (74)

1. Akutsu, A., Tonomura, Y., Hashimoto, H., Ohba, Y., 1992. Video indexing using motion vectors. In: Proc. SPIE Conf. on Visual Communications and Image Processing, SPIE vol. 1818, pp. 1522-1530.
2. Ardizzone, E., La Cascia, M., Avanzato, A., Bruna, A., 1999. Video indexing using MPEG motion compensation vectors. In: Proc. IEEE Internat. Conf. on Multimedia Computing and Systems, 1999.
3. Ardizzone, E., La Cascia, M., Molinelli, E., 1996. Motion and color based video indexing and retrieval. In: Proc. Internat. Conf. on Pattern Recognition.
4. Arman, F., Hsu, A., Chiu, M.-Y., 1993. Image processing on compressed data for large video databases. In: Proc. ACM Multimedia 93, Anaheim, CA, pp. 267-272.
5. Bao, O.K..-C, Lay, J.A., Guan, L., 2000. Compressed domain video parsing using energy histograms of the lower frequency DCT coefficients. Proc. SPIE Conference on Storage and Retrieval for Multimedia Database 2000, San Jose, CA, January, pp. 293-300.
6. Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R., 1992. Hierarchical model-based motion estimation. In: Proc. 2nd European Conf. on Computer Vision.
7. Boccignone, G., De Santo, M., Percannella, G., 2000. An algorithm for video cut detection in MPEG sequences. In: Proc. SPIE Conf. on Storage and Retrieval of Media Databases 2000, San Jose, CA, January, pp. 523-530.
8. Boreczky, J.S., Rowe, L.A., 1996. Comparison of video shot boundary detection techniques. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases IV, January, pp. 170-179.
9. Chang, S.-F., Messerschmit, D.G., 1995. Manipulation and compositing of MC-DCT compressed video, IEEE J. Selected Areas Commun. Special Issue on Intelligent Signal Processing, January, pp. 1-11.
10. Chen, J.-Y., Taskiran, C., Delp, E.J., Bouman, C.A., 1998. ViBE: a new paradigm for video database browsing and search. In: Proc. IEEE Workshop on Content-Based Access of Image and Video Databases.
11. Davis, J.W., 1998. Recognizing movement using motion histograms. Technical Report No. 487, MIT Media Laboratory Perceptual Computing Section, April.
12. Dimitrova, N., Golshani, F., 1995. Motion recovery for video content analysis. ACM Trans. Information Systems vol. 13, No. 4, October, pp. 408-439.
13. Divakaran, A., Ito, H., Sun, H., Poon, T., 1999. Scene change detection and feature extraction for MPEG4 sequences. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 545-551.
14. Divakaran, A., Sun, H., 2000. A descriptor for spatial distribution of motion activity for compressed video. In: Proc. SPIE Conf. on Storage and Retrieval for Media Databases 2000, San Jose, CA, January, pp. 392-398.
15. Dolby Laboratories, Dolby AC-3: Multi-channel perceptual coding. Available from http://www.dolby.com/tech/multipc.html, 1997.
16. Eng, H.-L., Ma, K.-K., 1999. Motion trajectory extraction based on macroblock motion vectors for video indexing, In: Proc. IEEE Internat. Conf. on Image Processing, Kobe, Japan, October.
17. Faloutsos, C., Lin, K., 1995. FastMap: a fast algorithm for indexing, data mining, and visualization of traditional and multimedia databases. In: Proc. ACM SIGMOD Conf., pp. 163-174.
18. Feng, J., Lo, K.T., Mehrpour, H., 1996. Scene change detection algorithm for MPEG video sequence. In: Proc. IEEE Internat. Conf. on Image Processing, Lausanne, Switzerland.
19. Fernando, W.A.C., Canagarajah, C.N., Bull, D.R., 1999. Video segmentation and classification for content based storage and retrieval using motion vectors. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 687-698.
20. Ford, R.M., 1999. A quantitative comparison of shot boundary detection metrics. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 666-676.
21. Haskell, B.G., Puri, A., Netravali, A.N., 1997. Digital Video: An Introduction to MPEG-2. Chapman & Hall, London.
22. He, L., Sanocki, E., Gupta, A., Grudin, J., 1999. Auto-summarization of audio-video presentations. In: Proc. ACM Multimedia 99, Orlando, FL, October, pp. 489-498.
23. Ho, Y.S., Gersho, A., 1989. Classified transform coding of images using vector quantization. In: Proc. ICASSP 89, pp. 1890-1893.
24. Huang, J., Liu, Z., Wang, Y., 1998. Integration of audio and visual information for content-based video segmentation. In: Proc. IEEE Internat. Conf. on Image Processing.
25. ISO/IEC IS 11172-2, MPEG-1 Video, 1993.
26. ISO/IEC IS 11172-3, MPEG-1 Audio, 1993.
27. ISO/IEC IS 13818-2, MPEG-2 Video, 1996.
28. ISO/IEC IS 13818-3, MPEG-2 Audio BC, 1996.
29. ISO/IEC IS 13818-7, MPEG-2 Audio AAC, 1997.
30. Kobla, V., DeMenthon, D., Doermann, D., 1999. Detection of slow-motion replay sequences for identifying sports videos. In: Proc. IEEE Workshop on Multimedia Signal Processing.
31. Kobla, V., DeMenthon, D., Doermann, D., 1999. Special effect edit detection using VideoTrails: a comparison with existing techniques. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII.
32. Kobla, V., Doermann, D., Faloutsos, C., 1997. VideoTrails: representing and visualizing structure in video sequences, In: Proc. ACM Multimedia 97, pp. 335-346.
33. Kobla, V., Doermann, D., Lin, K.-I., 1996. Archiving, indexing, and retrieval of video in the compressed domain. In: Proc. SPIE Conf. on Multimedia Storage and Archiving Systems, SPIE vol. 2916, pp. 78-89.
34. Kobla, V., Doermann, D., Lin, K.-I., Faloutsous, C., 1997. Compressed domain video indexing techniques using DCT and motion vector information in MPEG video. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases V, SPIE vol. 3022, pp. 200-211.
35. Lienhart, R., 1999. Comparison of automatic shout boundary detection algorithms. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 290-301.
36. Meng, J., Chang, S.-F., 1996. CVEPS - a compressed video editing and parsing system. In: Proc. ACM Multimedia 96.
37. Meng, J., Juan, Y., Chang, S.-F., 1995. Scene change detection in an MPEG compressed video sequence. In: IS & T/SPIE Symposium Proceedings, vol. 2419, San Jose, CA, February.
38. Nang, J., Hong, S., Ihm, Y., 1999. An efficient video segmentations scheme for MPEG video stream using macroblock information. In: Proc. ACM Multimedia 99, Orlando, FL, October, pp. 23-26.
39. Naphade, M.R., Kristjansson, T., Frey, B.J., Huang, T.S., 1998. Probabilistic multimedia objects (Multijects): a novel approach to video indexing and retrieval in multimedia systems. In: Proc. IEEE Internat. Conf. on Image Processing, vol. 5, Chicago, IL, October.
40. Naphade, M.R., Huang, T.S., 2000. A probabilistic framework for semantic indexing and retrieval in video. In: Proc. IEEE Internat. Conf. on Multimedia and Expo, vol. 1, New York, NY, July-August.
41. Ngo, C.W., Pong, T.C., Zhang, H.J., Chin, R.T., 2000. Motion characterization by temporal slice analysis. In: Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 2, Hilton Head Island, SC, June 13-15.
42. Ngo, C.W., Pong, T.C., Zhang, H.J., 2001. On clustering video shots. In: Proc. ACM Multimedia 2001, Ottawa, Ont., Canada, September 30-October 5, pp. 51-60.
43. Patel, N.V., Sethi, I.K., 1995. Audio characterization for video indexing. In: Proc. SPIE Conference on Storage & Retrieval for Image and Video Databases IV, San Jose, CA, February.
44. Patel, N.V., Sethi, I.K., 1997. Video shot detection and characterization for video databases. Patter Recognition 30 (4), 583-592.
45. Patti, A.J., Sezan, M.I., Tekalp, A.M., 1997. Robust methods for high quality stills from interlaced video in the presence of dominant motion. IEEE Trans. Circuits Systems Video Technol. 7 (2), 328-342.
46. Rabiner, L., Juang, B.-H., 1993. Fundamentals of Speech Recognition. Prentice-Hall, Englewood Cliffs, NJ.
47. Saunders, J., 1996. Real-time discrimination of broadcast speech/music. In: Proc. ICASSP 96, pp. 993996.
48. Saur, D.D., Tan, Y.-P., Kulkarni, S.R., Ramadge, P.J., 1997. Automated analysis and annotation of basketball video. In: Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases V, SPIE vol. 3022, pp. 176-187.
49. Scheirer, E., Slaney, M., 1997. Construction and evaluation of a robust multi-feature speech/music discriminator. In: Proc. ICASSP 97.
50. Shen, B., Sethi, I.K., 1996a. Convolution-based edge detection for image/video in block DCT domain. J. Visual Commun. Image Representation 7 (4), 411-423.
51. Shen, B., Sethi, I.K., 1996. Direct feature extraction from compressed images. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Database IV, January.
52. Shen, K., Delp, E., 1995. A fast algorithm for video parsing using MPEG compressed sequences. In: Proc. IEEE Internat. Conf. on Image Processing, vol. 2, pp. 252-255.
53. Song, B.C., Ra, J.B., 1999. Fast edge map extraction from MPEG compressed video data for video parsing. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 710-721.
54. Srinivasan, S., Petkovic, D., Ponceleon, D., 1999. Towards robust features for classifying audio in the CueVideo system. In: Proc. ACM Multimedia 99, Orlando, FL, October, pp. 393-400.
55. Sundaram, H., Chang, S.-F., 2000. Audio scene segmentation using multiple models, features and time scales. In: Proc. ICASSP 2000, Istanbul, Turkey, June.
56. Sundaram, H., Chang, S.-F., 2000. Determining computable scenes in films and their structures using audio-visual memory models. In: Proc. ACM Multimedia 2000, Los Angeles, CA, October-November.
57. Tan, Y.-P., Kulkarni, S.R., Ramadge, P.J., 1999. A framework for measuring video similarity and its application to video query by example. In: Proc. IEEE Internat. Conf. on Image Processing, Kobe, Japan, October.
58. Tan, Y.-P., Kulkarni, S.R., Ramadge, P.J., 1995. A new method for camera motion parameter estimation. In: Proc. IEEE Internat. Conf. on Image Processing, vol. 1, pp. 406-409.
59. Tan, Y.-P., Saur, D.D., Kulkarni, S.R., Ramadge, P.J., 1999. Rapid estimation of camera motion from compressed video with application to video annotation. In: IEEE Trans. on Circuits and Systems for Video Technology.
60. Taniguchi, Y., Akutsu, A., Tonomura, Y., 1997. PanoramaExcerpts: extracting and packing panoramas for video browsing. In: Proc. ACM Multimedia 97, pp. 427-436.
61. Tse, Y.T., Baker, R.L., 1991. Camera zoom/pan estimation and compensation fore video compression. In: Proc. SPIE Conf. on Image Processing Algorithms and Techniques II, Boston, MA, pp. 468-479.
62. Wang, Y., Huang, J., Liu, Z., Chen, T., 1997. Multimedia content classification using motion and audio information. In: Proc. ISCAS 97, vol. 2, Hong Kong, June, pp. 1488-1491.
63. Wolf, W., 1996. Key frame selection by motion analysis. In: Proc. ICASSP 96, vol. II, pp. 1228-1231.
64. Won, C.S., Park, D.K., Na, I.Y., Yoo, S.-J., 1999. Efficient color feature extraction in compressed video. In: Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases VII, San Jose, CA, January, pp. 677-686.
65. Yeo, B.-L., Liu, B., 1995a. On the extraction of DC sequence from MPEG video. In: Proc. IEEE Internat. Conf. on Image Processing, vol. 2, pp. 260-263.
66. Yeo, B.-L., Liu, B., 1995b. Rapid scene analysis on compressed videos. IEEE Trans. Circuits Systems Video Technol.fs 5 (6), 533-544.
67. Yeo, B.-L., Liu, B., 1995c. Unified approach to temporal segmentation of Motion JPEG and MPEG video. In: Proc. Internat. Conf. on Multimedia Computing and Systems, pp. 2-13.
68. Yoshida, T., Sakai, Y., 1995. Reliability metric of motion vectors and its application to motion estimation. In: Proc. SPIE Conference on Visual Communications and Image Processing 95, pp. 799-809.
69. Zabih, R., Miller, J., Mai, K., 1995. A feature-based algorithm for detecting and classifying scene breaks. In: Proc. ACM Multimedia 95, San Francisco, CA, November, pp. 189-200.
70. Zhang, H.J., Low, C.Y., Smoliar, S.W., 1995. Video parsing and browsing using compressed data. Multimedia Tools Appl. 1 (1), 89-111.
71. Zhang, H.J., Kankanhalli, A., Smoliar, S.W., 1993. Automatic partitioning of full-motion video. Multimedia Syst. J. 1 (1), 10-28.
72. Zhang, T., Kuo, C.-C.J., 1999a. Heuristic approach for generic audio data segmentation and annotation. In: Proc. ACM Multimedia 99, Orlando, FL, October, pp. 67-76.
73. Zhang, T., Kuo, C.-C.J., 1999b. Hierarchical classification of audio data for archiving and retrieving. In: Proc. ICASSP 99, vol. 6, pp. 3001-3004.
74. Zhong, Y., Zhang, H.J., Jain, A.K., 1999. Automatic caption localization in compressed video. In: Proc. IEEE Internat. Conf. on Image Processing, Kobe, Japan, October.