Exploring co-occurence between speech and body movement for audio-guided video localizagion

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 18, Issue 11, 2008, Pages 1608-1617

  Vajaria, Himanshu ^a   Sarkar, Sudeep ^a   Kasturi, Rangachar ^a  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

Author keywords

Audio visual association; Meeting analysis; Speaker diarization; Speaker localization

Indexed keywords

CLUSTER ANALYSIS; DIFFRACTIVE OPTICAL ELEMENTS; FEATURE EXTRACTION; FLOW OF SOLIDS; VISUAL COMMUNICATION;

ANALYSIS METHODS; AUDIO-VISUAL ASSOCIATION; BLOB MODELS; BODY MOVEMENTS; BOTTOM-UP; ITERATIVE CLUSTERING; LONG TERMS; MEETING ANALYSIS; MIXTURE OF GAUSSIANS; MOTION INFORMATIONS; MUTUAL INFORMATIONS; NONFRONTAL VIEWS; REAL DATUMS; SOURCE LOCALIZATIONS; SPATIAL CLUSTERING; SPEAKER DIARIZATION; SPEAKER LOCALIZATION; STATIONARY CAMERAS; TEMPORAL CLUSTERING; VIDEO FRAMES; VISUAL INFORMATIONS;

SPEECH;

EID: 55149110077     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2008.2005602     Document Type: Article

References (26)

1. Z. Barzelay and Y. Schechner, "Harmony in motion," Comput. Vis. Pattern Recognit., pp. 1-8, 2007.
2. M. Ben, M. Betser, F. Bimbot, and G. Gravier, "Speaker diarization using bottom-up clustering based on a parameter-derived distance between adapted GMMs," in Proc. Int. Conf. Spoken Language Processing, 2004, pp. 2329-2332.
3. C. Busso, S. Hernanz, C.-W. Chu, S.-I. Kwon, S. Lee, P. Georgiou, 1. Cohen, and S. Narayanan, "Smart room: Participant and speaker localization and identification," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, 2005, vol. 2, pp. 1117-1120.
4. S. S. Cheng and H. M. Wang, "A sequential metric-based audio segmentation using Bayesian information criterion," presented at the Eurospeech, 2003.
5. R. Cutler and L. Davis, "Look who's talking: Speaker detection using video and audio correlation," in Proc. IEEE Int. Conf. Multimedia, 2000, pp. 1589-1592.
6. R. Cutler, Y. Rui, A. Gupta, J. Cadiz, I. Tashev, L. He, A. Colburn, Z. Zhang, Z. Liu, and S. Silverberg, "Distributed meetings: A meeting capture and broadcast system," ACM Multimedia, pp. 503-512, 2002.
7. J. Fiscus, N. Radde, J. Garofolo, A. Le, J. Ajot, and C. Laprun, Rich Transcription 2005 Spring Meeting Recognition Evaluation 2007 [Online]. Available: www.nist.gov/speech/publications
8. J. Fisher and T. Darrell, "Probabalistic models and informative subspaces for audiovisual correspondence," in Proc: European Conf. Computer Vision, 2002, vol. 3, pp. 592-603.
9. J. S. Garofolo, C. D. Laprun, M. Michel, V. M. Stanford, and E. Tabassi, "The NIST meeting room pilot corpus," presented at the Int. Conf. Language Resources and Evaluation, 2004.
10. J. Hershey and J. Movellan, "Audio vision: Using audiovisual synchrony to locate sounds," Adv. Neural Inf. Process. Syst., pp. 813-819, 1999.
11. B. Kapralos, M. Jenkin, and E. Milios, "Audio-visual localization of multiple speakers in a video teleconferencing setting," Int. J. Imag. Syst. Technol., vol. 13, pp. 95-105, 2003.
12. E. Kidron and Y. Schechner, "Pixels that sound," Comput. Vis. Pattern Recognit., pp. 88-95, 2005.
13. G. Monaci, O. D. Escoda, and P. Vandergheynst, "Analysis of multimodal sequences using geometric video representations," Signal Process., vol. 86, pp. 3534-3548, 2006.
14. H. J. Nock, G. lyengar, and C. Neti, "Speaker localization using audio-visual synchrony: An empirical study," in Proc. ACM Int. Conf. Multimedia, 2003, pp. 488-499.
15. J. Pardo, X. Anguera, and C. Wooters, "Speaker diarization for multiple-distant-microphone meetings using several sources of information," IEEE Trans. Comput., vol. 56, no. 9, pp. 1212-1224, Sep. 2007.
16. E. Patterson, S. Gurbuz, Z. Tufekci, and J. Gowdy, "Moving talker, speaker-independent feature study and baseline results using the CUAVE multimodal speech corpus," J. Appl. Signal Process., vol. 11, pp. 1189-1201, 2002.
17. J. Regh, K. Murphy, and P. Fieguth, "Vision-based speaker detection using Bayesian networks," Comput. Vis. Pattern Recognit., pp. 110-116, 1999.
18. D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, "Speaker verification using adapted Gaussian mixture models," Digital Signal Process., vol. 10, pp. 19-41, 2000.
19. M. Sargin, O. Aran, A. Karpov, F. Ofli, Y. Yasinnik, S. Wilson, E. Erzin, Y. Yemez, and A. Tekalp, "Combined gesture-speech correlation analysis and speech driven gesture synthesis," in Proc. IEEE Int. Conf. Multimedia, 2006, pp. 893-896.
20. S. Sarkar and P. Soundararajan, "Supervised learning of large perceptual organization: Graph spectral partitioning and learning automata," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 5, pp. 504-525, May 2000.
21. M. Slaney and M. Covell, "Facesync: A linear operator for measuring synchronization of video facial images and audio tracks," Adv. Neural Inf. Process. Syst., vol. 14, pp. 814-820, 2000.
22. R. Stiefelhagen, K. Bernardin, R. Bowers, J. Garofolo, D. Mostefa, and P. Soundararajan, "The CLEAR 2006 evaluation," Springer Lecture Notes Comput. Sci., no. 4122, pp. 1-44, 2006.
23. S. Tranter and D. Reynolds, "An overview of automatic speaker diarization systems," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 5, pp. 1557-1565, May 2006.
24. H. Vajaria, T. Islam, S. Sarkar, R. Sankar, and R. Kasturi, "Audio segmentation and speaker localization in meeting videos," in Proc. Int. Conf. Pattern Recognit., 2006, vol. 2, pp. 1150-1153.
25. L. Valbonesi, R. Ansari, D. McNeill, F. Quek, S. Duncan, K. E. McCullough, and R. Bryll, "Multimodal signal analysis of prosody and hand motion: Temporal correlation of speech and gestures," in Proc. Eur. Signal Processing Conf., 2002, pp. 1330-1345.
26. C. Wooters, J. Fung, B. Peskin, and X. Anguera, "Towards robust speaker segmentation: The ICSI-SRI fall 2004 diarization system," presented at the Rich Transcription Workshop, 2004.