Audio-visual atoms for generic video concept classification

ACM Transactions on Multimedia Computing, Communications and Applications

Volumn 6, Issue 3, 2010, Pages

  Jiang, Wei ^a   Cotton, Courtenay ^a   Chang, Shih Fu ^a   Ellis, Dan ^a   Loui, Alexander C ^b  

^a Columbia University ^*  (United States)

^b EASTMAN KODAK COMPANY   (United States)

Author keywords

Audio Visual Atom; Audio visual codebook; Joint audio visual analysis; Semantic concept detection

Indexed keywords

AUDIO-VISUAL; AUDIO-VISUAL ATOM; AUDIOVISUAL ANALYSIS; CODEBOOKS; CONCEPT CLASSIFICATION; CONCEPT DETECTION; ENERGY ONSET; HIERARCHICAL ALGORITHM; SEMANTIC CONCEPT DETECTION; TIME FREQUENCY ANALYSIS; VISUAL FEATURE;

FEATURE EXTRACTION; SEMANTICS;

ATOMS;

EID: 77956370559     PISSN: 15516857     EISSN: 15516865     Source Type: Journal    
DOI: 10.1145/1823746.1823748     Document Type: Conference Paper

References (33)

1. ANEMUELLER, J., BACH, J., CAPUTO, B., ET AL. 2008. Biologically motivated audio-visual cue integration for object categorization. In Proceedings of the International Conference on Computational Science.
2. BARZELAY, Z. AND SCHECHNER, Y. 2007. Harmony in motion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
3. BEAL, M. AND JOJIC, N. 2003. A graphical model for audiovisual object tracking. IEEE Trans. Patt. Anal. Mach. Intell. 25, 7, 828-836.
4. BIRCHFELD, S. 2007. Kit: An implementation of the Kanade-Lucas-Tomasi feature tracker. http://vision. stanford.eduj/~birch.
5. CHANG, S., ELLIS, D., JIANG, W., ET AL. 2007. Large-scale multimodal semantic concept detection for consumer video. In Proceedings of the ACM SIGMM Workshop on Multimedia Information Retrieval.
6. CHEN, Y. AND WANG, J. 2004. Image categorization by learning and reasoning with regions. J. Mach. Learn. Resear. 5, 913-939.
7. CHU, S. AND NARAYANAN, S. 2008. Environmental sound recognition using mp-based features. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 1-4.
8. CRISTANI, M., MANUELE, B., AND MURINO, V. 2007. Audio-visual event recognition m surveillance video sequences. IEEE Trans. Multimedia 9, 2, 257-267.
9. DEMENTHON, D. AND DOERMANN, D. 2003. Video retrieval using spatial-temporal descriptors. In Proceedings of ACM Multimedia, 508-517.
10. DENG, Y. AND MANJUNATH, B. 2001. Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Patt. Anal. Mach. Intell. 23, 8, 800-810.
11. FRIEDMAN, J., HASTIE, T., AND TIBSHIRANI, R. 2000. Additive logistic regression: a statistical view of boosting. Ann. Stat. 28, 22, 337-407.
12. GALMAR, E. AND HUET, B. 2007. Analysis of vector space model and spatiotemporal segmentation for video indexing and retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval.
13. GRAUMAN, K. AND DARREL, T. 2005. The pyramid match kernel: Discriminative classification with sets of image features. In Proceedings of the IEEE International Conference on Computer Vision. 1458-1465.
14. IWANO, K., YOSHINAGA, T., TAMURA, S., AND FURUI, S. 2007. Audiovisual speech recognition using lip information extracted from side-face images. EURASIP J. Audio Speech Music Process. 1, 4-4.
15. JEPSON, A., FLEET, D., AND EL-MARAGHI, T. 2003. Robust online appearance models for visual tracking. IEEE Trans. Patt. Anal. Mach. Intell. 25, 10, 1296-1311.
16. JIANG, W. 2010. Advanced techniques for semantic concept detection in unconstrained videos. Ph. D. thesis, Columbia University.
17. KAUCIC, R., DALTON, B., AND BLAKE, A. 1996. Real-time lip tracking for audio-visual speech recognition applications. In Proceedings of the 2nd European Conference on Computer Vision. 376-387.
18. KRSTULOYIC, S. AND GRIGONYAL, R. 2006. MPTK Matching Pursuit made tractable. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. 496-499.
19. LOUI, A., LUO, J., AND CHANG, S. 2007. Kodak's consumer video benchmark data set: concept definition and annotation. In Proceedings of the ACM SIGMM Workshop on Multimedia Information Retrieval. 245-254.
20. LOWE, D. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91-110.
21. LUCAS, B. AND KANADE, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the Imaging Understanding Workshop. 121-130.
22. MALLAT, S. AND ZHANG, Z. 1993. Matching pursuits with time-frequency dictionaries. IEEE Trans. Sign. Process. 41, 12, 3397-3415.
23. MARON, O. AND LOZANO-PEREZ, T. 1998. A framework for multiple-instance learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 570-576.
24. NIEBLES, J., HAN, B., FERENCZ, A., AND LI, F. 2008. Extracting moving people from internet videos. Int. J. Comput. Vision 79, 3, 299-318.
25. OGLE, J. AND ELLIS, D. 2007. Fingerprinting to identify repeated sound events in long-duration personal audio recordings. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. 233-236.
26. PETITCOLAS, F. 2003. Mpeg for matlab. http://www.petitcolas.net/fabien/ software.mpeg.
27. SHI, J. AND TOMASI, C. 1994. Good features to track. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 593-600.
28. SMEATON, A. F., OVER, P., AND KRAAIJ, W. 2006. Evaluation campaigns and TRECVid. In Proceedings of the ACM SIGMM Workshop on Multimedia Information Retrieval. 321-330.
29. STAUFFER, C. AND GRIMSON, W. 2002. Learning patterns of activity using real-time tracking. IEEE Trans. Patt. Anal. Mach. Intell. 22, 8, 747-757.
30. TIEU, K. AND VIOLA, P. 2000. Boosting Image retrieval. Int. J. Comput. Vision 56, 1-2, 228-235.
31. YANAGAWA, A., HSU, W., AND CHANG, S. 2006. Brief descriptions of visual features for baseline TRECVID concept detectors. Columbia University ADVENT Tech. rep. 219-2006-5.
32. YANG, C., DONG, M., AND HUA, J. 2006. Region-based image annotation using asymmetrical support vector machine-based multiple-instance learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2057-2063.
33. ZHOU, H., YUAN, Y., AND SHI, C. 2009. Object tracking using SIFT features and Mean Shift. Comput. Vis. Image Understand. 113, 93, 345-352.