Boosting-based multimodal speaker detection for distributed meeting videos

IEEE Transactions on Multimedia

Volumn 10, Issue 8, 2008, Pages 1541-1551

  Zhang, Cha ^a   Yin, Pei ^b   Rui, Yong ^a,c   Cutler, Ross ^d   Viola, Paul ^a   Sun, Xinding ^d   Pinto, Nelson ^d   Zhang, Zhengyou ^a  

^a MICROSOFT RESEARCH   (United States)

^b Georgia Institute of Technology   (United States)

^c MICROSOFT RESEARCH ASIA   (China)

^d MICROSOFT   (United States)

Author keywords

Audiovisual fusion; Boosting; Speaker detection

Indexed keywords

FEATURE EXTRACTION; MAGNETOPLASMA; SPEECH RECOGNITION; VISUAL COMMUNICATION;

AUDIO AND VISUAL INFORMATIONS; AUDIOVISUAL FUSION; BOOSTING; COMMERCIAL PRODUCTS; DISTRIBUTED MEETINGS; ERROR RATES; FEATURE LEVELS; HIGH EFFICIENCIES; HIGH RESOLUTIONS; MICROSOFT; MULTI MODALS; SOUND SOURCE LOCALIZATIONS; SPEAKER DETECTION; VISUAL FEATURES;

AUDIO ACOUSTICS;

EID: 57849134738     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2008.2007344     Document Type: Article

References (37)

1. C. Busso, S. Hernanz, C. Chu, S. Kwon, S. Lee, P. Georgiou, I. Cohen, and S. Narayanan, "Smart room: Participant and speaker localization and identification", in Proc. IEEE ICASSP, 2005.
2. R. Cutler, Y. Rui, A. Gupta, J. Cadiz, I. Tashev, L. He, A. Colburn, Z. Zhang, Z. Liu, and S. Silverbert, "Distributed meetings: A meeting capture and broadcasting system", in Proc. ACM Conf. Multimedia, 2002.
3. B. Kapralos, M. Jenkin, and E. Milios, "Audio-Visual Localization of Multiple Speakers in a Video Teleconferencing Setting", Tech. Rep. York University, Canada, 2002.
4. B. Yoshimi and G. Pingali, "A multimodal speaker detection and tracking system for teleconferencing", in Proc. ACM Conf. Multimedia, 2002.
5. P. Viola and M. Jones, "Robust real-time face detection", Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, 2004.
6. H. Wang and P. Chu, "Voice source localization for automatic camera pointing system in videoconferencing", in Proc. IEEE ICASSP, 1997.
7. Y. Rui, D. Florencio, W. Lam, and J. Su, "Sound source localization for circular arrays of directional microphones", in Proc. IEEE ICASSP, 2005.
8. A. Adjoudani and C. Benoit, "On the integration of auditory and visual parameters in an HMM-based ASR", in Speechreading by Humans and Machines. Berlin, Germany: Springer, 1996, pp. 461-471.
9. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition", IEEE Trans. Multimedia, vol. 2, no. 3, pp. 141-151, Sep. 2000.
10. W. Hsu, S.-F. Chang, C.-W. Huang, L. Kennedy, C.-Y. Lin, and G. Iyengar, "Discovery and fusion of salient multi-modal features towards news story segmentation", in SPIE Electronic Imaging, 2004.
11. M. Brand, N. Oliver, and A. Pentland, "Coupled hidden Markov models for complex action recognition", in Proc. IEEE CVPR, 1997.
12. M. Naphade, A. Garg, and T. Huang, "Duration dependent input output Markov models for audio-visual event detection", in Proc. IEEEICME, 2001.
13. G. Iyengar and C. Neti, "Speaker change detection using joint audiovisual statistics", in Int. RIAO Conf., 2000.
14. P. Besson and M. Kunt, "Information Theoretic Optimization of Audio Features for Multimodal Speaker Detection", Tech. Rep. Signal Processing Institute, EPFL, Lausanne, Switzerland, 2005.
15. J. Fisher, III, T. Darrell, W. Freeman, and P. Viola, "Learning joint statistical models for audio-visual fusion and segregation", in NIPS, 2000, pp. 772-778.
16. J. Hershey and J. Movellan, "Audio vision: Using audio-visual synchrony to locate sounds", in Advances in Neural Information Processing Systems, 2000.
17. V. Pavlović, A. Garg, J. Rehg, and T. Huang, "Multimodal speaker detection using error feedback dynamic Bayesian networks", in Proc. IEEE CVPR, 2001.
18. D. N. Zotkin, R. Duraiswami, and L. S. Davis, "Logistic regression, adaboost and bregman distances", EURASIP J. Appl. Signal Process., vol. 2002, no. 11, pp. 1154-1164, 2002.
19. J. Vermaak, M. Gangnet, A. Black, and P. Pérez, "Sequential Monte Carlo fusion of sound and vision for speaker tracking", in Proc. IEEE ICCV, 2001.
20. H. Nock, G. Iyengar, and C. Neti, "Speaker localisation using audiovisual synchrony: An empirical study", in Proc. CIVR, 2003.
21. R. Cutler and L. Davis, "Look who's talking: Speaker detection using video and audio correlation", in Proc. IEEE ICME, 2000.
22. M. Beal, H. Attias, and N. Jojic, "Audio-video sensor fusion with probabilistic graphical models", in Proc. ECCV, 2002.
23. Y. Chen and Y. Rui, "Real-time speaker tracking using particle filter sensor fusion", Proc. IEEE, vol. 92, pp. 485-494, 2004.
24. K. Nickel, T. Gehrig, R. Stiefelhagen, and J. McDonough, "A joint particle filter for audio-visual speaker tracking", in ICMI, 2005.
25. C. Zhang, Z. Zhang, and D. Florêncio, "Maximum likelihood sound source localization for multiple directional microphones", in ICASSP, 2007.
26. T. Gustafsson, B. Rao, and M. Trivedi, "Source localization in reverberant environments: Performance bounds and ml estimation", in Proc. ICASSP, 2001.
27. M. Brandstein and H. Silverman, "A robust method for speech signal time-delay estimation in reverberant rooms", in Proc. ICASSP, 1997.
28. J. Friedman, T. Hastie, and R. Tibshirani, "Additive Logistic Regression: A Statistical View of Boosting", Tech. Rep. Stanford Univ., Dept. Statistics, Stanford, CA, 1998.
29. R. Schapire and Y. Singer, "Improved boosting algorithms using confidence-rated predictions." Mach. Learn., vol. 37, no. 3, pp. 297-336, 1999.
30. P. Viola, M. Jones, and D. Snow, "Detecting pedestrians using patterns of motion and appearance", in Proc. IEEE ICCV, 2003.
31. R. Collins, A. Lipton, T. Kanade, H. Fujiyoshi, D. Duggins, Y. Tsin, D. Tolliver, N. Enomoto, and O. Hasegawa, "A System for Video Surveillance and Monitoring", Tech. Rep. Carnegie Mellon Univ., Robotics Inst., Pittsburgh, PA, 2000.
32. C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland, "Pfinder: Realtime tracking of the human body", IEEE Trans. Pattern. Anal. Mach. Intell., vol. 19, no. 7, pp. 780-785, Jul. 1997.
33. S. Goodridge, "Multimedia sensor fusion for intelligent camera control and human computer interaction", Ph.D. dissertation, Dept. Elect. Eng., North Carolina Start Univ., Raleigh, 1997.
34. M. Collins, R. Schapire, and Y. Singer, "Logistic regression, Adaboost and Bregman distances", Mach. Learn., vol. 48, no. 1-3, pp. 253-285, 2002.
35. L. Mason, J. Baxter, P. Bartlett, and M. Frean, "Boosting algorithms as gradient decent", in NIPS, 2000.
36. C. Zhang and P. Viola, "Multiple-instance pruning for learning efficient cascade detectors", in NIPS, 2007.
37. N. Dalai and B. Triggs, "Histograms of oriented gradients for human detection", in Proc. CVPR, 2005.