On the correlation of automatic audio and visual segmentations of music videos

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 17, Issue 3, 2007, Pages 347-355

  Gillet, Olivier ^b   Essid, Slim ^b   Richard, Gael ^a,b  

^a IEEE   (France)

^b INSTITUT TELECOM   (France)

Author keywords

Audio segmentation; Cross modal queries; Information retrieval; Multimedia indexing; Multimodal processing; Music videos; Novelty detection

Indexed keywords

AUDIO SEGMENTATION; AUTOMATIC STRUCTURING; CROSS-MODAL QUERIES; KERNEL METHODS; MULTIMEDIA INDEXING; MULTIMODAL PROCESSING; MUSIC VIDEOS; NOVELTY DETECTION;

ALGORITHMS; DATABASE SYSTEMS; INFORMATION RETRIEVAL; MULTIMEDIA SYSTEMS; SEMANTICS;

IMAGE SEGMENTATION;

EID: 33847794586     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2007.890831     Document Type: Article

References (41)

1. L. Xie, L. Kennedy, S.-F. Chang, A. Divakaran, H. Sun, and C.-Y. Lin, "Discovering meaningful multimedia patterns with audio-visual concepts and associated text," in Proc. IEEE Int. Conf. Image Process., Oct. 2004, vol. 4, pp. 2383-2386.
2. J. Huang, Z. Liu, and Y. Wang, "Classification of TV programs based on audio information using hidden Markov model," in IEEE Workshop Multimedia Signal Process., 1998, pp. 27-32.
3. A. Albiol, L. Torres, and E. Delp, "Combining audio and video for video sequence indexing applications," in Proc. IEEE Int. Conf. Multimedia Expo, 2002, pp. 353-356.
4. M. Gondry (director), The Work of Director Michel Gondry. DVD, Palm Pictures, 2003.
5. S. Jonze (director), The Work of Director Spike Jonze. DVD, Palm Pictures, 2003.
6. EAF Music Visual Niches - Extraordinary Music Videos, DVD, 2002.
7. L. Agnihotri, N. Dimitrova, J. Kender, and J. Zimmerman, "Music videos miner," in Proc. 11th ACM Int. Conf. Multimedia, 2003, pp. 442-443.
8. L. Agnihotri, N. Dimitrova, and J. R. Kender, "Design and evaluation of a music video summarization system," in Proc. 2004 IEEE Int. Conf. Multimedia Expo, Jun. 2004, pp. 1943-1946.
9. X. Shao, C. Xu, and M. S. Kankanhalli, "A new approach to automatic music video summarization," in Proc. Int. Conf. Imag. Proc., Oct. 2004, pp. 625-628.
10. O. Gillet and G. Richard, "Comparing audio and video segmentations for music videos indexing," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., May 2006.
11. J. Foote, M. Cooper, and A. Girgensohn, "Creating music videos using automatic media analysis," in Proc. 10th ACM Int. Conf. Multimedia, 2002, pp. 553-560.
12. P. Mulhem, M. S. Kankanhalli, J. Yi, and H. Hassan, "Pivot vector space approach for audio-video mixing," IEEE Multimedia, vol. 10, no. 2, pp. 28-40, Apr./Jun. 2003.
13. R. Yang and M. S. Brown, "Music database query with video by synesthesia observation," in Proc. 2004 IEEE Int. Conf. Multimedia Expo, Jun. 2004, pp. 305-308.
14. M. Nayak, S. H. Srinivasan, and M. S. Kankanhalli, "Music synthesis for home videos: An analogy based approach," in Proc. IEEE Pacific-Rim Conf. Multimedia, Dec. 2003.
15. J. Bello, L. Daudet, S. Abdallah, C. Duxbury, M. Davies, and M. Sandler, "A tutorial on onset detection in music signals," IEEE Trans. Speech Audio Process., vol. 13, no. 9, pp. 1035-1047, Sep. 2005.
16. S. Jeannin and A. Divakaran, "MPEG-7 visual motion descriptors," IEEE Trans. Circuits Syst. Video Technol., vol. 11, pp. 720-724, Nov. 2001.
17. G. Peeters, A. L. Burthe, and X. Rodet, "Toward automatic music audio summary generation from signal analysis," in Proc. 3rd Int. Conf. Music Information Retrieval, 2002, pp. 86-92.
18. M. Cooper and J. Foote, "Automatic music summarization via similarity analysis," in Proc. Int. Symp. Music Inf. Retrieval, 2002, pp. 81-85.
19. P. Over, T. Ianeva, W. Kraaij, and A. F. Smeaton, "TRECVID 2005 - An overview," National Institute of Standards and Technology (NIST), 2006 [Online]. Available: http://www-nlpir.nist.gov/projects/tvpubs/tv5.papers/ tv5overview.pdf
20. E. Scheirer, "Tempo and beat analysis of acoustic musical signals," J. Acoust. Soc. Amer., vol. 103, no. 1, pp. 588-601, 1998.
21. A. Klapuri, "Sound onset detection by applying psychoacoustic knowledge," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1999, pp. 3089-3092.
22. M. Alonso, G. Richard, and B. David, "Extracting note onsets from musical recordings," in Proc. IEEE Int. Conf. Multimedia Expo, 2005, CD-ROM.
23. J. Laroche, "Efficient tempo and beat tracking in audio recordings," J. Audio Eng. Soc., vol. 51, no. 4, pp. 226-233, Apr. 2004.
24. G. Peeters, "A large set of audio features for sound description (similarity and classification) in the CUIDADO project," IRCAM, 2004.
25. M. A. Bartsch and G. H. Wakefield, "To catch a chorus: Using chromabased representations for audio thumbnailing," in Proc. 2001 IEEE Workshop Appl. Signal Process. Audio Acoust., 2001, pp. 15-18.
26. S. Essid, G. Richard, and B. David, "Musical instrument recognition based on class pairwise feature selection," in Proc. 5th Int. Conf. Music Information Retrieval, Oct. 2004, pp. 560-567.
27. L. R. Rabiner, Fundamentals of Speech Processing. Englewood Cliffs, NJ: Prentice-Hall, 1993.
28. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, no. 1-2, pp. 273-324, 1997.
29. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning," Artif. Intell., vol. 97, no. 1-2, pp. 245-271, 1997.
30. P. Mitra, C. A. Murthy, and S. K. Pal, "Unsupervised feature selection using feature similarity," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 301-312, Mar. 2002.
31. R. Duda, P. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2001.
32. S. Essid, "Classification automatique des signaux audiofrequences: Reconnaissance des instruments de musique," Ph.D. dissertation, Université Pierre et Marie Curie, Paris, France, 2005.
33. B. Zhou and J. Hansen, "Unsupervised audio stream segmentation and clustering via the Bayesian information criterion," in Proc. Int. Conf. Spoken Language Process., 2000, pp. 714-717.
34. S. S. Chen and P. S. Gopalakrishnan, "Speaker, environment and channel change detection and clustering via the Bayesian information criterion," in Proc. DARPA Broadcast News Transcrip. Understand. Workshop, Feb. 1998, pp. 602-610.
35. B. Shölkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
36. G. Loosli, S. G. Lee, and S. Canu, "Context changes detection by one-class SVMs," in Proc. Workshop Mach. Learning for User Modeling, 2005, pp. 27-34.
37. F. Desobry, M. Davy, and C. Doncarli, "An online Kernel Change Detection algorithm," IEEE Trans. Signal Process., vol. 53, no. 8, pp. 2961-2974, Aug. 2005.
38. S. Zhou and R. Chellappa, "From sample similarity to ensemble similarity: Probabilistic distance measures in reproducing Kernel Hubert Space," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 917-929, Jun. 2006.
39. O. Gillet and G. Richard, "Automatic transcription of drum sequences using audiovisual features," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process., Mar. 2005, pp. 205-208.
40. A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, and G. Volpe, "Multimodal analysis of expressive gesture in music and dance performances," in Proc. 5th Int. Gesture Workshop, Apr. 2003, pp. 20-39.
41. J. B. Kruskal, "An overview of sequence comparison," in Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison, D. Sankoff and J. B. Kruskal, Eds. Reading, MA: Addison-Wesley, 1983, pp. 1-44.