A multimodal scheme for program segmentation and representation in broadcast video streams

IEEE Transactions on Multimedia

Volumn 10, Issue 3, 2008, Pages 393-408

  Wang, Jinqiao ^b   Duan, Lingyu ^c   Liu, Qingshan ^a,b   Lu, Hanqing ^a,b   Jin, Jesse S ^a,d  

^a IEEE

^b INSTITUTE OF AUTOMATION   (China)

^c INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^d UNIVERSITY OF NEWCASTLE   (Australia)

Author keywords

Broadcast video; Latent semantic analysis; Multimodal fusion; TV program segmentation

Indexed keywords

DATABASE SYSTEMS; IMAGE SEGMENTATION; SEMANTICS; USER INTERFACES; VIDEO RECORDING; WEB BROWSERS;

BROADCAST VIDEO; LATENT SEMANTIC ANALYSIS; MULTIMODAL FUSION; TV PROGRAM SEGMENTATION;

BROADCASTING;

EID: 41549165353     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2008.917362     Document Type: Conference Paper

References (48)

1. J. Masthoff and R. Lukin, "Preface: Workshop future TV: Adaptive instruction in your living room," in Proc. Future. TV: Adaptive Instruction In Your Living Room, 2002.
2. Tivo [Online]. Available: http://www.tivo.com.
3. L. Ardissono, A. Kobsa, and M. M. editors, Personalized Digital Television. Norwell, MA: Kluwer, 2004.
4. M. M. Yeung and B. L. Yeo, "Time-constrained clustering for segmentation of video into story units," in Proc. ICPR'96, 1996, vol. 3, pp. 375-380.
5. C. W. Ngo, T. C. Pong, and H. J. Zhang, "On clustering and retrieval of video shots through temporal slices analysis," IEEE Trans. Multimedia, vol. 4, pp. 446-458, Dec. 2002.
6. H. Okamoto, Y. Yasugi, N. Babaguchi, and T. Kitahashi, "Video clustering using spatio-temporal image with fixed length," in Proc. ICME'02, Aug. 2002, vol. 1, pp. 53-56.
7. D. Q. Zhang, C. Y. Lin, S. F. Chang, and J. R. Smith, "Semantic video clustering across sources using bipartite spectral clustering," in Proc. ICME '04, Jun. 2004, pp. 117-120.
8. Z. Rasheed and M. Shah, "Detection and representation of scene in videos," IEEE Trans. Multimedia, vol. 7, pp. 1097-1105, Dec. 2005.
9. Y. Zhai and M. Shah, "A general framework for temporal video scene segmentation," in Proc. ICCV05 ., Beijing, China, Oct. 2005, pp. 15-21.
10. J. Huang, Z. Liu, and Y. Wang, "Joint scene classification and segmentation based on hidden Markov model," IEEE Trans. Multimedia, vol. 7, pp. 538-550, Jun. 2005.
11. J. Allan, J. Carbonell, G. Doddington, J. Yamron, and Y. Yang, "Topic detection and tracking pilot study final report," in Proc. DARPA Broadcast News Transcription and Understanding Workshop, 1998, pp. 194-218.
12. J. Gauvain, L. Lamel, and G. Adda, "The limsi broadcast news transcription system," Speech Commun., pp. 89-108, 2002.
13. S. Fischer, R. Lienhart, and W. Effelsberg, "Automatic recognition of film genres," in Proc. ACM MM'95, 1995.
14. A. G. Hauptmann and M. J. Witbrock, "Story segmentation and detection of commercials in broadcast news video," in Proc. Advances in Digital Libraries Conf., Santa Barbara, CA, Apr. 1998.
15. J. H. Z. Liu and W. Yao, "Integration of audio and visual information for content-based video segmentation," in Proc. ICIP 98, Oct. 1998, vol. 3, pp. 526-529.
16. M. Roach, J. Mason, and M. Pawlewski, "Video genre classification using dynamics," in Proc. ICCASP 01, Salt Lake City, UT, May 2001, vol. 3, pp. 7-11.
17. R. Jasinschi and J. Louie, "Automatic TV program, genre classification based on audio patterns," in Proc. Euromicro Conf., Sep. 2001, pp. 370-375.
18. B. T. Truong and C. Dorai, "Automatic genre identification for content-based video categorization," in Proc. ICPR'00, 2000, vol. 4, pp. 230-233.
19. G. W. L. Agnihotri and N. Dimitrova, "TV program, classification based on face and text processing," in Proc. ICME'00, 2000, vol. 3, pp. 1345-1348.
20. T.-H. Kim, W.-H. Lee, and D.-S. Jeong, "Automatic video genre identification method in mpeg compressed domain," in Proc. ITC-CSCC'02, Jul. 2002, pp. 1527-1530.
21. J. Wang, L. Duan, H. Lu, and J. S. Jin, "A semantic image category for structuring TV broadcast video streams," in Proc. PCM'06, 2006, pp. 279-286.
22. Opencredit [Online]. Available: http://en.wikipedia.org/wiki/ Opening_credits
23. Closecredit [Online]. Available: http://en.wikipedia.org/wiki/ Closing_cred.its
24. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
25. J. P. Campbell, Jr., "Speaker recognition: A tutorial," Proc. IEEE, vol. 85, no. 9, pp. 1437-1642, Sep. 1997.
26. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, "Indexing by latent semantic analysis," J. Soc. Inform. Sci., vol. 41, no. 6, pp. 391-407, 1990.
27. L.-Y. Duan, J. Wang, Y. Zheng, J. S. Jin, H. Lu, and C. Xu, "Segmentation, categorization, and identification of commercials from TV streams using multimodal analysis," in Proc. ACM MM'06, 2006, pp. 202-210.
28. T.-S. Chua, S.-F. Chang, L. Chaisorn, and W. Hsu, "Story boundary detection in large broadcast news video archives: Techniques, experience and trends," in Proc. ACM MM'04, 2004, pp. 656-659.
29. P. Rennen, "Streamesage unsupervised asr-based topic segmentation," in Proc. TRECVID Workshop, 2003.
30. J. Reynar, "An automatic method of finding topic boundaries," in Proc. 32nd Annu. Meeting of the Associationfor Computational Linguistics, 1994.
31. W. Hsu, S.-F. Chang, C.-W. Huang, L. Kennedy, C.-Y. Lin, and G. Iyengar, "Discovery and fusion of salient multi-modal features towards news story segmentation," in Proc. IS&T/SPIE Symp. Electronic Imaging: Science and Technology, Jan. 2004.
32. A. Hanjalic, R. Lagendijk, and J. Biemond, "Automatically segmenting movies into logical story units," in Proc. 3rd Int. Conf. VISUAL '99, Jun. 1999.
33. H. Sundaram, "Segmentation, Structure Detection, and Summarization of Multimedia Sequences," Ph.D. dissertation, Columbia Univ., New York, 2002.
34. B. Adams, C. Dorai, and S. Venkatesh, "Automatic extraction of expressive elements from motion pictures: Tempo," IEEE Trans. Multimedia, vol. 4, pp. 472-481, Dec. 2002.
35. C. G. Snoek, M. Worring, J. C. v. Gemert, J.-M. Geusebroek, and A. W. Smeulders, "The challenge problem, for automated detection of 101 semantic concepts in multimedia," in Proc. ACM MM'06, 2006.
36. C. G. Snoek, M. Worring, J.-M. Geusebroek, D. C. Koelma, and F. J. Seinstra, "The mediamill trecvid 2004 semantic video search engine," in Proc. TRECVID Workshop'04, 2004.
37. Y. Wu, E. Y. Chang, K. C.-C. Chang, and J. R. Smith, "Optimal multimodal fusion for multimedia data analysis," in Proc. ACM MM'04, 2004.
38. C. C. Vogt and G. W. Cottrell, "Fusion via a linear combination of scores," Inf. Retriev., Oct. 1999.
39. Trecvid [Online]. Available: http://www-nlpir.nist.gov/projects/t01v
40. J. Wang, L. Duan, H. Lu, J. S. Jin, and C. Xu, "A mid-level scene change representation via audiovisual alignment," in Proc. ICASSP '06, May 2006.
41. D. Hawking, T. Upstill, and N. C. Toward, "Toward better weighting of anchors," in Proc. ACM SIGIR'04, Jun. 2004, pp. 25-29.
42. J. Canny, "A computational approach to edge detection," IEEE Trans. Pattern Anal. Machine Intell., vol. 8, no. 6, pp. 679-698, 1986.
43. B. Manjunath and W. Ma, "Texture features for browsing and retrieval of image data," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, no. 8, pp. 837-842, 1996.
44. L. Lu, H. Jiang, and H. Zhang, "Robust audio classification and segmentation method," in Proc. ACM MM'01, 2001, pp. 203-211.
45. W. Hsu, L. Kennedy, C.-W. Huang, S.-F. Chang, C.-Y. Lin, and G. Iyengar, "News video story segmentation using fusion of multi-level multi-modal features in trecvid 2003," in Proc. ICASSP '04, May 2004.
46. J. E. Gentle, "Singular value factorization," Numer. Linear Algebra Applicat. Statist., 1998.
47. M. Franz, J. S. McCarley, and S. Roukos, "Segmentation and detection at IBM: Hybrid statistical models and two-tiered clustering broadcast news domain," in. Proc. TDT-3 Workshop'00, 2000.
48. A. Y. Ng, M. I. Jordan, and Y. Weiss, "On spectral clustering: Analysis and an algorithm," Adv. Neural Inform. Process. Syst., 2001.