Search the audio, browse the video - A generic paradigm for video collections

Eurasip Journal on Applied Signal Processing

Volumn 2003, Issue 2, 2003, Pages 209-222

  Amir, Arnon ^a   Srinivasan, Savitha ^a   Efrat, Alon ^b  

^a IBM ALMADEN RESEARCH CENTER   (United States)

^b UNIVERSITY OF ARIZONA   (United States)

Author keywords

Automatic video indexing; Phonetic speech retrieval; Video and speech retrieval; Video browsing

Indexed keywords

IMAGE PROCESSING; SPEECH ANALYSIS; SPEECH RECOGNITION; WEB BROWSERS;

PHONETIC SPEECH RETRIEVAL;

ONLINE SEARCHING;

EID: 0037301246     PISSN: 11108657     EISSN: None     Source Type: Journal    
DOI: 10.1155/S111086570321012X     Document Type: Review

References (64)

1. R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval, Addison Wesley, Reading, Mass, USA, 1999.
2. X. S. Zhou and T. S. Huang, "Unifying keywords and visual contents in image retrieval," IEEE Multimedia, vol. 9, no. 2, pp. 23-33, 2002.
3. P. G. B. Enser, "Query analysis in a visual information retrieval context," Journal of Document and Text Management, vol. 1, no. 1, pp. 25-52, 1993.
4. L. H. Armitage and P. G. B. Enser, "Analysis of user need in image archives," Journal of Information Science, vol. 23, no. 4, pp. 287-299, 1997.
5. S. Shatford, "Analyzing the subject of a picture: A theoretical approach," Cataloging and Classification Quarterly, vol. 6, no. 3, pp. 39-62, 1986.
6. A. Hamrapur, A. Gupta, B. Horowitz, et al., "Virage video engine," in Storage and REtrieval for Image and Video Databases V, vol. 3656 of Proceedings of SPIE, pp. 188-197, San Jose, Calif, USA, February 1997.
7. A. Gupta and R. Jain, "Visual information retrieval," Communications of the ACM, vol. 40, no. 5, pp. 70-79,1997.
8. S.-F. Chang, W. Chen, and H. Sundaram, "VideoQ: a fully automated video retrieval system using motion sketches," in Proc. 4th IEEE Workshop on Applications of Computer Vision, pp. 270-271, Princeton, NJ, USA, October 1998.
9. S.-F. Chang, A. Eleftheriadis, and R. McClintock, "Next generation content representation, creation and searching for new media applications in education," Proceedings of the IEEE, vol. 86, no. 5, pp. 884-904, May 1998, Special Issue on Multimedia Signal Processing, Part One.
10. J. R. Smith and S.-F. Chang, "Querying by color regions using the visualseek content-based visual query system," in Intelligent Multimedia Information Retrieval. IJCAI, pp. 159-173, MIT press/AAAI press, 1997.
11. Deng Yining and B. S. Manjunath, "NeTra-V: toward an object-based video representation," IEEE Trans. Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 616-627,1998.
12. J. J. Little and Z. Gu, "Video retrieval by spatial and temporal structure of trajectories," in Proc. SPIE Storage and REtrieval for Media Databases, San Jose, Calif, USA, 2001.
13. S. W. Smoliar, J. D. Baker, T. Nakayama, and L. Wilcox, "Multimedia search: A authoring perspective," in Proc. 1st International Workshop on Image Databases and Multimedia Search (IAPR-1996), pp. 1-8, Amsterdam, The Netherlands, August 1996.
14. P. Over and R. Taban, "The TREC-2001 video track framework," in Proc. 10th Text REtrieval Conference (TREC-10), Gaithersburg, Md, USA, November 2001.
15. M. Flickher, H. Sawhney, W. Niblack, et al., "Query by image and video content: The QBIC system," IEEE Computer, vol. 28, no. 9, pp. 23-32, 1995.
16. J. Garofolo, C. Auzanne, and E. Voorhees, "The TREC spoken document retrieval track: A success story," in Proc. 8th Text REtrieval Conference (TREC-8), E. Voorhees and D. Harman, Eds., vol. 500-246 of NIST Special Publication, pp. 107-130, Gaithersburg, Md, USA, November 1999.
17. P. Aigrain, H. J. Zhang, and D. Petkovic, "Content-based representation and retrieval of visual media: A state-of-the-art review," Multimedia Took and Applications, vol. 3, no. 3, pp. 179-202, 1996.
18. J. R. Bach, C. Fuller, A. Gupta, et al., "Virage image search engine: An open framework for image management," in IS&T/SPIE Conference on Storage and REtrieval for Still Image and Video Databases IV, vol. 2670, pp. 76-87, San Jose, Calif, USA, February 1996.
19. G. J. F. Jones, J. T. Foote, K. S. Jones, and S. J. Young, "Retrieving spoken documents by combining multiple index sources," in Proc. Annual International ACM SIGIR Conference on Research and Development in Information REtrieval, pp. 30-38, Zurich, Switzerland, August 1996.
20. H. J. Zhang, P. Aigrain, and D. Petkovic, "Content-based representation and retrieval of visual media: A state-of-the-art review," Multimedia Tools and Applications, vol. 3, no. 3, pp. 179-202,1996.
21. S.-F. Chang, W. Chen, H. J. Meng, H. Sundaram, and D. Zhong, "VideoQ: An automated content based video search system using visual cues," in Proc. ACM Multimedia, pp. 313-324, ACM Press, Seattle, Wash, USA, November 1997.
22. J. K. Aggarwal and Q. Cai, "Human motion analysis: A review," Computer Vision and Image Understanding, vol. 73, no. 3, pp. 428-440,1999.
23. H. Wactlar, M. Christel, Y. Gong, and A. Hauptmann, "Lessons learned from building a terabyte digital video library," IEEE Computer, vol. 32, no. 2, pp. 66-73, 1999.
24. A. Del Bimbo, Visual Information Retrieval, Morgan Kaufmann, San Francisco, Calif, USA, 1999.
25. A. Yoshitaka and T. Ichikawa, "A survey on content-based retrieval for multimedia databases," IEEE Trans. Knowledge and Data Engineering, vol. 11, no. 1, pp. 81-93,1999.
26. H. Greenspan, G. Dvir, and Y. Rubner, "Region correspondence for image matching via EMD flow," in IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL-2000), pp. 27-31, Tel Aviv, Israel, June 2000.
27. M. Petkovic and W. Jonker, "Content-based video retrieval by integrating spatio-temporal and stochastic recognition of events," in Proc. IEEE International Workshop on Detection and Recognition of Events in Video (EVENT '01), Vancouver, BC, Canada, July 2001.
28. Y. Gong, L. T. Sin, C. H. Chuan, H. J. Zhang, and M. Sakauchi, "Automatic parsing of TV soccer programs," in Proc. IEEE International Conference on Multimedia Computing and Systems, pp. 167-174, Washington, DC, May 1995.
29. D. M. Gavrila, "The visual analysis of human movement: A survey," Computer Vision and Image Understanding, vol. 73, no. 1, pp. 82-98, 1999.
30. R. Qian, N. Hearing, and I. Sezan, "A computational approach to semantic event detection," in Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 200-206, Fort Collins, Colo, USA, June 1999.
31. M. R. Naphade, T. Kristjansson, B. Frey, and T. S. Huang, "Probabilistic multimedia objects (multijects): A novel approach to video indexing and retrieval in multimedia systems," in Proc. 5th IEEE International Conference on Image Processing, vol. 3, pp. 536-540, Chicago, Ill, USA, October 1998.
32. P. Salembier and F. Marqués, "Region-based representations of image and video: segmentation tools for multimedia services," IEEE Trans. Circuits and Systems for Video Technology, vol. 9, no. 8, pp. 1147-1169, 1999.
33. X. He, W. Y. Ma, O. King, M. Li, and H. J. Zhang, "Learning and inferring a semantic space from users relevance feedback for image retrieval," in Proc. ACM Multimedia, France, December 2002.
34. A. Hauptmann and M. Witbrock, "Informedia: News-on-demand multimedia information acquisition and retrieval," in Intelligent Multimedia Information REtrieval, chapter 10, pp. 215-240, MIT Press, Cambridge, Mass, USA, 1997.
35. S. Srinivasan and D. Petkovic, "Phonetic confusion matrix based spoken document retrieval," in Proc. 23rd Annual International ACM SIGIR Conference on Research and Development in Information REtrieval, pp. 81-87, Athens, Greece, July 2000.
36. M. G. Christel, M. A. Smith, C. R. Taylor, and D. B. Winkler, "Evolving video skims into useful multimedia abstractions," in Proc. ACM CHI '98 Conference on Human Factors in Computing Systems, pp. 171-178, Los Angeles, Calif, USA, April 1998.
37. G. J. F. Jones, J. T. Foote, K. S. Jones, and S. J. Young, "Video mail retrieval: the effect of word spotting accuracy on precision," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, vol. 1, pp. 309-312, Detroit, Mich, USA, May 1995.
38. E. M. Voorhees and D. Harman, "Overview of the sixth text retrieval conference (TREC-6)," Information Processing and Management, vol. 36, no. 1, pp. 3-35,2000.
39. L. He, A. Gupta, S. A. White, and J. Grudin, "Corporate deployment of on-demand video: Usage, benefits, and lessons," Tech. Rep. MSR-TR-98-62, Microsoft Research, Redmond, Wash, USA, November 1998.
40. N. Omoigui, L. He, A. Gupta, J. Grudin, and E. Sanocki, "User benefits of non-linear time compression," in Proc. ACM CHI '99 Conference on Human Factors in Computing Systems, pp. 136-143, Pittsburgh, Pa, USA, May 1999.
41. F. C. Li, A. Gupta, E. Sanocki, L. W. He, and Y. Rui, "Browsing digital video," in Proc. ACM CHI '00 Conference on Human Factors in Computing Systems, pp. 169-176, The Hague, The Netherlands, April 2000.
42. S. Srinivasan, D. Ponceleon, A. Amir, and D. Petkovic, "What is in that video anyway? In search of better browsing," in Proc. IEEE International Conference on Multimedia Computing and Systems, pp. 388-392, Florence, Italy, June 1999.
43. A. Amir, D. Ponceleon, D. Blanchard, B. Petkovic, S. Srinivasan, and G. Cohen, "Using audio time scale modification for video browsing," in Proc. 33rd Hawaii International Conference on System Sciences (HICSS-33), vol. 3, Maui, Hawaii, USA, January 2000.
44. D. Malah, "Time-domain algorithms for harmonic bandwidth reduction and time scaling of speech signals," IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 27, no. 2, pp. 121-133,1979.
45. S. Srinivasan, D. Ponceleon, A. Amir, B. Blanchard, and D. Petkovic, Web Engineering: Managing Diversity and Complexity in Web Application Development, vol. 2016 of Lecture Notes in Computer Science, Springer-Verlag, Berlin, Germany, April 2001.
46. A. Amir, G. Ashour, and S. Srinivasan, "Towards automatic real time preparation of on-line video proceedings for conference talks and presentations," in Video Use in Office and Education, Proc. 34th Hawaii International Conference on System Sciences (HICSS-34), Maui, Hawaii, USA, January 2001.
47. S. Chen, E. Eide, M. J. F. Gales, R. A. Gopinath, D. Kanevsky, and P. Olsen, "Automatic transcription of broadcast news," Speech Communication, vol. 37, no. 1-2, pp, 69-87,2001.
48. S. E. Johnson, P. Jourlin, G. L. Moore, K. S. Jones, and P. C. Woodland, "Spoken document retrieval for TREC-7 at Cambridge University," in Proc. 7th Text REtrieval Conference (TREC-7), vol. 500-242 of NIST Special Publication, pp. 191-200, Gaithersburg, Md, USA, November 1998.
49. A. Singhal, J. Coi, D. Hindle, D. Lewis, and F. Pereira, "AT&T at TREC-7," in Proc. 7th Text REtrieval Conference (TREC-7), vol. 500-242 of NIST Special Publication, Gaithersburg, Md, USA, 1998.
50. S. Srinivasan, D. Petkovic, D. Ponceleon, and M. Viswanathan, "Query expansion for imperfect speech: applications in distributed learning," in Proc. IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL '00), Hilton Head Island, SC, USA, June 2000.
51. S. E. Robertson, A. Walker, K. S. Jones, M. M. Hancock-Beaulieu, and M. Gatford, "Okapi at TREC-3," in Proc. 3rd Text REtrieval Conference (TREC-3), vol. 500-225 of NIST Special Publication, pp. 109-126, Gaithersburg, MD, USA, April 1995.
52. R. Byrd and Y. Ravin, "Identifying and extracting relations in text," in Proc. 4th International Conference on Applications of Natural Language to Information Systems (NLDB '99), pp. 149-154, Klagenfurt, Austria, June 1999.
53. J. S. Justeson and S. M. Katz, "Technical terminology: some linguistic properties and an algorithm for Identification in text," Natural Language Engineering, vol. 1, no. 1, pp. 9-27, 1995.
54. K. Ng and V. Zue, "Phonetic recognition for spoken document retrieval," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 325-328, Seattle, Wash, USA, May 1998.
55. M. Wechsler, E. Munteanu, and P. Schäuble, "New techniques for open vocabulary spoken document retrieval," in Proc. Annual International ACM SIGIR Conference on Research and Development in Information REtrieval, pp. 20-27, Melbourne, Australia, 1998.
56. M. Witbrock and A. Hauptmann, "Using words and phonetic strings for efficient information retrieval from imperfectly transcribed spoken documents," in Proc. 2nd ACM International Conference on Digital Libraries, pp. 30-35, Philadelphia, Pa, USA, July 1997.
57. P. Lawrence, "Hanging on the metaphone," Computer Language, vol. 7, no. 12, pp. 39-43,1990.
58. D. Gusfield, Algorithms on Strings, Trees, and Sequences: Computer Science and Computational Biology, Cambridge University Press, Cambridge, UK, 1997.
59. J. Allan, J. Carbonell, G. Doddington, J. P. Yamron, and Y. Yang, "Topic detection and tracking pilot study: final report," in Proc. DARPA Broadcast News Transcription and Understanding Workshop, pp. 194-218, Lansdowne, Va, USA, February 1998.
60. S. Dharanipragada, M. Franz, and S. Roukos, "Audio-indexing for broadcast news," in Proc. 7th Text REtrieval Conference (TREC-7), vol. 500-242 of NIST Special Publication, pp. 63-67, Gaithersburg, Md, USA, 1999.
61. M. A. Siegler, M. J. Witbrock, S. T. Slattery, K. Seymore, R. E. Jones, and A. Hauptmann, "Experiments in spoken document retrieval at CMU," in Proc. 7th Text REtrieval Conference (TREC-7), vol. 500-242 of MS Special Publication, pp. 264-270, Gaithersburg, Md, USA, November 1998.
62. NIST Spoken Natural Language Processing Group, "Hub-4 broadcast news evaluation English test material," LDC catalog LDC2000S86, LDC, University of Pennsylvania, Philadelphia, Pa, USA, 1998.
63. D. A. Van Leeuwen, W. Kraaij, and R. Ekkelenkamp, "Prediction of keyword spotting performance based on phonemic contents," in Proc. ESCA ETRW Workshop: Accessing Information in Spoken Audio, University of Cambridge, Cambridge, UK, 1999.
64. A. Amir, A. Efrat, and S. Srinivasan, "Advances in phonetic word spotting," in Proc. 10th International Conference on Information and Knowledge Management (ACM CIKM), pp. 580-582, Atlanta, Ga, USA, November 2001.