Audio-visual grouplet: Temporal audio-visual interactions for general video concept classification

MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

Volumn , Issue , 2011, Pages 123-132

  Jiang, Wei ^a   Loui, Alexander C ^a  

^a EASTMAN KODAK COMPANY   (United States)

Author keywords

Audio visual grouplet; Foreground background separation; Temporal audio visual interaction; Video concept classification

Indexed keywords

AUDIO-VISUAL; AUDIO-VISUAL CORRELATIONS; AUDIO-VISUAL INTERACTIONS; AUDIOVISUAL ANALYSIS; BUILDING ELEMENT; CODE-WORDS; COLUMBIA; CONCEPT CLASSIFICATION; CONSUMER VIDEOS; FOREGROUND/BACKGROUND; PERFORMANCE IMPROVEMENTS; SEMANTIC CONCEPT; STATE-OF-THE-ART APPROACH; TEMPORAL CORRELATIONS; VIDEO CONTENTS; VISUAL CHANNELS;

SEMANTICS; SEPARATION;

BUILDING CODES;

EID: 84455170074     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2072298.2072316     Document Type: Conference Paper

References (39)

1. T. Aach and A. Kaup. Bayesian algorithms for adaptive change detection in image sequences using Markov random fields. Signal Processing: Image Communication, vol. 7, pages 147-160, 1995.
2. A. Akutsu and Y. Tonomura. Video tomography: An efficient method for camerawork extraction and motion analysis. ACM Multimedia, 1994.
3. Z. Barzelay and Y. Schechner. Harmony in motion. In Proc. IEEE CVPR, pages 1-8, 2007.
4. M.J. Beal, N. Jojic, and H. Attias. A graphical model for audiovisual object tracking. IEEE Trans. PAMI, 25(7):828-836, 2003.
5. S. Chang, D. Ellis, W. Jiang, K. Lee, A. Yanagawa, A. Loui, and J. Luo. Large-scale multimodal semantic concept detection for consumer video. In Proc. ACM MIR, pages 255-264, 2007.
6. C. Cotton, D. Ellis, and A. Loui. Soundtrack classification by transient events. In IEEE ICASSP, Prague, Czech Republic, 2011.
7. M. Cristani, B. Manuele, and M. Vittorio. Audiovisual event recognition in surveillance video sequences. IEEE Trans. Multimedia, 9(2):257-267, 2007.
8. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Proc. IEEE CVPR, pages 886-893, 2005.
9. M. Ding, Y. Chen, and S.L. Bressler. Granger causality: Basic theory and applications to neuroscience. In S. Schelter, N. Winterhalder, J. Timmer, Handbook of Time Series Analysis, Wiley, Wienheim, 2006.
10. S. Divvala, D. Hoiem, J. Hays, A. Efros, and M. Hebert. An empirical study of context in object detection. In IEEE CVPR, Miami, Florida, 2009.
11. S.Y. Elhabian, K.M. El-Sayed. Moving object detection in spatial domain using background removal techniques - State-of-art. Recent Patents on Computer Science, 1(1):32-54, 2008.
12. O. Enqvist, K. Josephson, and F. Kahl. Optimal correspondences from pairwise constraints. In IEEE ICCV, Kyoto, Japan, 2009.
13. M. Everingham, L. Van Cool, C.K.I. Williams, J. Winn, and A. Zisserman. The Pascal visual object classes (VOC) challenge. IJCV, 88(2):303-338, 2010.
14. C. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica, 37(3):424-438, 1969.
15. K. Iwano, T. Yoshinaga, S. Tamura, and S. Furui. Audio-visual speech recognition using lip information extracted from side-face images. EURASIP Journal on ASMP, 2007(1):4-12, 2007.
16. W. Jiang, C. Cotton, S. Chang, D. Ellis, and A. Loui. Audio-visual atoms for generic video concept classification. ACM TOMCCAP, 6:1-19, 2010.
17. Y. Jiang, C.W. Ngo, and J. Yang. Towards optimal bag-of-features for object categorization and semantic video retrieval. In ACM CIVR, pages 494-501, 2007.
18. Y. Jiang, G. Ye, S. Chang, D. Ellis, and A. Loui. Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In ACM ICMR, Trento, Italy, 2011.
19. P. Joly and H.K. Kim. Efficient automatic analysis of camera work and microsegmentation of video using spatiotemporal images. Signal Processing: Image Communication, 8(4):295-307, 1996.
20. Y. Ke, R. Sukthankar, and M. Hebert. Event detection in crowded videos. In IEEE ICCV, Brazil, 2007.
21. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In IEEE CVPR, Alaska, 2008.
22. W.H. Lin and A. Hauptmann. News video classification using svm-based multimodal classifiers and combination strategies. In Proc. ACM Multimedia, pages 323-326, 2002.
23. D.G. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91-110, 2004.
24. M. Marszalek, I. Laptev, and C. Schmid. Actions in context. In IEEE CVPR, Miami, Florida, 2009.
25. A. Nedungadi, G. Rangarajan, N. Jain, and M. Ding. Analyzing multiple spike trains with nonparametric granger causality. Journal of Computational Neuroscience, 27(1):55-64, 2009.
26. A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In NIPS, 2001.
27. D. Nistér and H. Stewénius. Scalable recognition with a vocabulary tree. In Proc. IEEE CVPR, pages 2161-2168, 2006.
28. D.T. Pham and J.F. Cardoso. Blind separation of instantaneous mixtures of non stationary sources. IEEE Trans. Signal Processing, 49(9):1837-1848, 2001.
29. K. Prabhakar, S. Oh, P. Wang, G. Abowd, and J.M. Rehg. Temporal causality for the analysis of visual events. In IEEE CVPR, San Francisco, 2010.
30. S.T. Roweis. One microphone source separation. In NIPS, pages 793-799, 2001.
31. M. Sargin, H. Aradhye, P.J. Moreno, and M. Zhao. Audiovisual celebrity recognition in unconstrained web videos. In IEEE ICASSP, Taipei, 2009.
32. A.F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and TRECVid. In Proc. ACM MIR, pages 321-330, 2006.
33. C. Stauffer and E. Grimson. Learning patterns of activity using real-time tracking. IEEE Trans. PAMI, 22(8):747-757, 2000.
34. A. Walden. A unified view of multitaper multivariate spectral estimation. Biometrika, 87(4):767-788, 2000.
35. B. Wang and M.D. Plumbley. Investigating single-channel audio source separation methods based on non-negative matrix factorization. In Proc. ICArn, pages 17-20, 2006.
36. W. Wang, D. Cosker, Y. Hicks, S. Saneit, and J. Chambers. Video assisted speech source separation. In Proc. IEEE ICASSP, pages 425-428, 2005.
37. Y. Wu, C.Y. Lin, E.Y. Chang, and J.R. Smith. Multimodal information fusion for video concept detection. In Proc. IEEE ICIP, pages 2391-2394, 2004.
38. J. Yang, Y. Jiang, A. Hauptmann, and C. Ngo. Evaluating bag-of-visual-words representations in scene classification. ACM MIR, pages 197-206, 2007.
39. B. Yao and L. Fei-Fei. Grouplet: A structured image representation for recognizing human and object interactions. In IEEE CVPR, San Francisco, 2010.