Improving acoustic event detection using generalizable visual features and multi-modality modeling

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2011, Pages 349-352

  Huang, Po Sen ^a   Zhuang, Xiaodan ^a   Hasegawa Johnson, Mark ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

acoustic event detection; coupled hidden Markov models; hidden Markov models; multi stream HMM; optical flow

Indexed keywords

ACOUSTIC EVENT CLASSIFICATION; ACOUSTIC EVENTS; ASYNCHRONY; AUDIO-VISUAL; COUPLED HIDDEN MARKOV MODELS; DATA RESOURCES; DATA SETS; JOINT MODELING; LOCALIZATION INFORMATION; MULTI-MODALITY; MULTI-STREAM; MULTI-STREAM HMM; STATE-SPACE; TIME STAMPS; VIDEO DATA; VIDEO STREAMS; VISUAL FEATURE; VISUAL REPRESENTATIONS;

CLASSIFICATION (OF INFORMATION); DETECTORS; HIDDEN MARKOV MODELS; MULTIMEDIA SYSTEMS; OPTICAL FLOWS; SIGNAL DETECTION; SPEECH COMMUNICATION; SPEECH RECOGNITION;

AUDIO ACOUSTICS;

EID: 80051652444     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2011.5946412     Document Type: Conference Paper

References (17)

1. C. Canton-Ferrer, T. Butko, C. Segura, X. Giro, C. Nadeu, J. Hernando, and J. Casas, "Audiovisual event detection towards scene understanding," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2009, pp. 81-88.
2. C. Clavel, T. Ehrette, and G. Richard, "Events detection for an audio-based surveillance system," in IEEE International Conference on Multimedia and Expo, 2005.
3. F. Beaufays, D. Boies, M. Weintraub, and Q. Zhu, "Using speech/non-speech detection to bias recognition search on noisy data," in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003.
4. A. Temko, "CLEAR 2007 AED evaluation plan," http://isl.ira.uka.de/clear07, 2007.
5. X. Zhuang, X. Zhou, M. A. Hasegawa-Johnson, and T. S. Huang, "Real-world acoustic event detection," Pattern Recognition Letters, vol. 31, no. 12, pp. 1543-1551, 2010.
6. M. R. Naphade, A. Garg, and T. Huang, "Duration-dependent input-output Markov models for audio-visual event detection," in International Conference on Multimedia, 2001.
7. Z. Xiong, R. Radhakrishnan, A. Divakaran, and T. Huang, "Audio-visual event recognition with application in sports video," in Intelligent Multimedia Processing with Soft Computing, ser. Studies in Fuzziness and Soft Computing, 2005, vol. 168, pp. 129-149.
8. S. M. Chu and T. S. Huang, "Audio-visual speech modeling using coupled hidden Markov models," in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002.
9. C. Sanderson and K. K. Paliwal, "Identity verification using speech and face information," Digital Signal Processing, vol. 14, no. 5, pp. 449-480, 2004.
10. C. Zach, T. Pock, and H. Bischof, "A duality based approach for realtime tv-l1 optical flow," in Pattern Recognition (Proc. DAGM), Heidelberg, Germany, 2007, pp. 214-223.
11. N. Ikizler, R. Cinbis, and P. Duygulu, "Human action recognition with line and flow histograms," in 19th International Conference on Pattern Recognition, 2008.
12. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in CVPR, 2006.
13. A. Adjoudani and C. Benoit, "On the integration of auditory and visual parameters in an HMM-based ASR," In D. G. Stork and M. E. Hennecke (Eds.), Speechreading by Humans and Machines. Berlin: Springer-Verlag, pp. 461-471, 1996.
14. P. Teissier, J. Robert-Ribes, and J. L. Schwartz, "Comparingmodels for audiovisual fusion in a noisy-vowel recognition task," IEEE Trans. Speech Audio Processing, vol. vol. 7, pp. 629-642, 1999.
15. K. Chaudhuri, S. M. Kakade, and K. Livescu, "Multi-view clustering via canonical correlation analysis," in In Proc. of ICML09, 2009.
16. S. Nakamura, "Statistical multimodal integration for audio-visual speech processing," IEEE Transactions on Neural Networks, vol. 13, no. 4, 2002.
17. X. Zhuang, X. Zhou, T. S. Huang, and M. Hasegawa-Johnson, "Feature analysis and selection for acoustic event detection," in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2008.