Temporal video segmentation to scenes using high-level audiovisual features

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 21, Issue 8, 2011, Pages 1163-1177

  Sidiropoulos, Panagiotis ^a,b   Mezaris, Vasileios ^a   Kompatsiaris, Ioannis ^a   Meinedo, Hugo ^c   Bugalho, Miguel ^b,d   Trancoso, Isabel ^b,d  

^a INFORMATION TECHNOLOGIES INSTITUTE   (Greece)

^b UNIVERSITY OF SURREY   (United Kingdom)

^c INESC   (Portugal)

^d INSTITUTO SUPERIOR TÉCNICO   (Portugal)

Author keywords

Audio events; scene transition graph; scenes; video segmentation; visual concepts

Indexed keywords

AUDIO EVENTS; SCENES; TRANSITION GRAPHS; VIDEO SEGMENTATION; VISUAL CONCEPT;

MOTION PICTURES; SEMANTICS;

IMAGE SEGMENTATION;

EID: 79961197386     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2011.2138830     Document Type: Article

References (40)

1. G. Boccignone, A. Chianese, V. Moscato, and A. Picariello, "Foveated shot detection for video segmentation," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 11, pp. 365-377, Nov. 2005. (Pubitemid 40384658)
2. A. Hanjalic, R. L. Lagendijk, and J. Biemond, "Automated highlevel movie segmentation for advanced video-retrieval systems," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 4, pp. 580-588, Jun. 1999.
3. R. Jain, "Eventweb: Developing a human-centered computing system," IEEE Comput., vol. 41, no. 2, pp. 42-50, Feb. 2008.
4. A. F. Smeaton, P. Over, and A. R. Doherty, "Video shot boundary detection: Seven years of trecvid activity," Comput. Vision Image Understand., vol. 114, no. 4, pp. 411-418, Apr. 2010.
5. M. Yeung, B.-L. Yeo, and B. Liu, "Segmentation of video by clustering and graph analysis," Comput. Vision Image Understand., vol. 71, no. 1, pp. 94-109, 1998. (Pubitemid 128420523)
6. W. Tavanapong and J. Zhou, "Shot clustering techniques for story browsing," IEEE Trans. Multimedia, vol. 6, no. 4, pp. 517-527, Aug. 2004.
7. Z. Rasheed and M. Shah, "Detection and representation of scenes in videos," IEEE Trans. Multimedia, vol. 7, no. 6, pp. 1097-1105, Dec. 2005. (Pubitemid 41789121)
8. Y. Zhai and M. Shah, "Video scene segmentation using Markov chain Monte Carlo," IEEE Trans. Multimedia, vol. 8, no. 4, pp. 686-697, Aug. 2006. (Pubitemid 44144154)
9. C.-W. Ngo, Y.-F. Ma, and H.-J. Zhang, "Video summarization and scene detection by graph modeling," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 2, pp. 296-305, Feb. 2005.
10. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Patt. Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
11. Y. Zhao, T. Wang, P. Wang, W. Hu, Y. Du, Y. Zhang, and G. Xu, "Scene segmentation and categorization using NCuts," in Proc. CVPR, 2007, pp. 1-7.
12. V. T. Chasanis, A. C. Likas, and N. P. Galatsanos, "Scene detection in videos using shot clustering and sequence alignment," IEEE Trans. Multimedia, vol. 11, no. 1, pp. 89-100, Jan. 2009.
13. Y. Cao, W. Tavanapong, K. Kim, and J. Oh, "Audio-assisted scene segmentation for story browsing," in Proc. ACM CIVR, 2003, pp. 446-455.
14. Y. Zhu and D. Zhou, "Scene change detection based on audio and video content analysis," in Proc. 5th Int. Conf. Computat. Intell. Multimedia Applicat., 2003, pp. 229-234.
15. H. Sundaram and S.-F. Chang, "Video scene segmentation using video and audio features," in Proc. IEEE ICME, Jul.-Aug. 2000, pp. 1145-1148. (Pubitemid 33059143)
16. S. Rho and E. Hwang, "Video scene determination using audiovisual data analysis," in Proc. 24th Int. Conf. Distribut. Comput. Syst. Workshops, 2004, pp. 124-129.
17. S.-C. Chen, M.-L. Shyu, W. Liao, and C. Zhang, "Scene change detection by audio and video clues," in Proc. IEEE ICME, Nov. 2002, pp. 365-368.
18. A. Chianese, V. Moscato, A. Penta, and A. Picariello, "Scene detection using visual and audio attention," in Proc. Ambi-Sys Workshop Ambient Media Delivery Interact. Television, 2008, pp. 4030-4033.
19. S. Pfeiffer, R. Lienhart, and W. Efflsberg, "Scene determination based on video and audio features," Multimedia Tools Applicat., vol. 15, no. 1, pp. 59-81, 2001. (Pubitemid 32897567)
20. A. Velivelli, C.-W. Ngo, and T. S. Huang, "Detection of documentary scene changes by audio-visual fusion," in Proc. ACM CIVR, 2004, pp. 227-237.
21. N. Nitanda, M. Haseyama, and H. Kitajima, "Audio signal segmentation and classification for scene-cut detection," in Proc. IEEE Int. Symp. Circuits Syst., vol. 4. May 2005, pp. 4030-4033.
22. V. Parshin, A. Paradzinets, and L. Chen, "Multimodal data fusion for video scene segmentation," in Proc. Int. Conf. Vis. Inform. Inform. Syst., 2005, pp. 279-289.
23. N. Goela, K. Wilson, F. Niu, and A. Divakaran, "An SVM framework for genre-independent scene change detection," in Proc. IEEE ICME, Jul. 2007, pp. 532-535.
24. K. Wilson and A. Divakaran, "Discriminative genre-independent audiovisual scene change detection," Proc. SPIE Conf. Multimedia Content Access: Algorithms Syst. III, vol. 7255, p. 725502, Jan. 2009.
25. P. Sidiropoulos, V. Mezaris, I. Kompatsiaris, H. Meinedo, and I. Trancoso, "Multi-modal scene segmentation using scene transition graphs," in Proc. ACM Multimedia, 2009, pp. 665-668.
26. P. Sidiropoulos, V. Mezaris, I. Kompatsiaris, H. Meinedo, M. Bugalho, and I. Trancoso, "On the use of audio events for improving video scene segmentation," in Proc. WIAMIS, Apr. 2010, pp. 1-4.
27. J. Smith, M. Naphade, and A. Natsev, "Multimedia semantic indexing using model vectors," in Proc. IEEE ICME, Jul. 2003, pp. 445-448.
28. N. Rasiwasia, P. Moreno, and N. Vasconcelos, "Bridging the gap: Query by semantic example," IEEE Trans. Multimedia, vol. 9, no. 5, pp. 923-938, Aug. 2007. (Pubitemid 47171250)
29. C. Snoek, M. Worring, J. van Gemert, J.-M. Geusebroek, and A. Smeulders, "The challenge problem for automated detection of 101 semantic concepts in multimedia," in Proc. ACM Multimedia, Oct. 2006, pp. 421-430. (Pubitemid 47113558)
30. B. Manjunath, J.-R. Ohm, V. Vasudevan, and A. Yamada, "Color and texture descriptors," IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 6, pp. 703-715, Jun. 2001. (Pubitemid 32616532)
31. V. Mezaris, P. Sidiropoulos, A. Dimou, and I. Kompatsiaris, "On the use of visual soft semantics for video temporal decomposition to scenes," in Proc. IEEE ICSC, Sep. 2010, pp. 141-148.
32. R. Amaral, H. Meinedo, D. Caseiro, I. Trancoso, and J. Neto, "A prototype system for selective dissemination of broadcast news in European portuguese," EURASIP J. Adv. Signal Process., vol. 2007, no. 37507, pp. 1-11, 2007.
33. H. Meinedo, "Audio pre-processing and speech recognition for broadcast news," Ph.D. thesis, IST, Tech. Univ. Lisbon, Lisbon, Portugal, Mar. 2008.
34. M. Bugalho, J. Portelo, I. Trancoso, T. Pellegrini, and A. Abad, "Detecting audio events for semantic video search," in Proc. Interspeech, Sep. 2009, pp. 1151-1154.
35. I. Trancoso, T. Pellegrini, J. Portelo, H. Meinedo, M. Bugalho, A. Abad, and J. Neto, "Audio contributions to semantic video search," in Proc. IEEE ICME, Jul. 2009, pp. 630-633.
36. G. Chavez, M. Cord, S. Philip-Foliguet, F. Precioso, and A. Araujo, "Robust scene cut detection by supervised learning," in Proc. EUSIPCO, Sep. 2006.
37. E. Tsamoura, V. Mezaris, and I. Kompatsiaris, "Gradual transition detection using color coherence and other criteria in a video shot metasegmentation framework," in Proc. IEEE ICIP-MIR, Oct. 2008, pp. 45-48.
38. J. Vendrig and M. Worring, "Systematic evaluation of logical story unit segmentation," IEEE Trans. Multimedia, vol. 4, no. 4, pp. 492-499, Dec. 2002.
39. J. Yang and A. Hauptmann, "(Un)reliability of video concept detection," in Proc. ACM CIVR, Jul. 2008, pp. 85-94.
40. J. Uijlings, A. Smeulders, and R. Scha, "Real-time bag of words, approximately," in Proc. ACM CIVR, 2009.