Multimodal Speaker diarization

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 34, Issue 1, 2012, Pages 79-93

  Noulas, Athanasios ^a   Englebienne, Gwenn ^a   Kröse, Ben J A ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

audiovisual fusion; dynamic Bayesian networks; Speaker diarization

Indexed keywords

AUDIO AND VIDEO; AUDIO STREAM; AUDIO-BASED; AUDIO-VISUAL FUSION; BROADCAST VIDEO; DATA SETS; DYNAMIC BAYESIAN NETWORK; DYNAMIC BAYESIAN NETWORKS; EXPECTATION-MAXIMIZATION ALGORITHMS; LABELED TRAINING DATA; MEETING VIDEO; MODALITY ANALYSIS; MODEL PARAMETERS; MULTI-MODAL; MULTIMODAL FRAMEWORKS; PROBABILISTIC FRAMEWORK; RECORDING EQUIPMENT; SPEAKER DIARIZATION; VIDEO STREAMS;

ALGORITHMS; HIDDEN MARKOV MODELS; SPEECH RECOGNITION;

BAYESIAN NETWORKS;

EID: 81855191839     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2011.47     Document Type: Article

References (36)

1. D.A. Reynolds and P.A. Torres-Carrasquillo, "Approaches and Applications of Speaker Diarization," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 953-956, 2010.
2. J.G. Fiscus, J. Ajot, and J.S. Garofolo, "The Rich Transcription 2007 Meeting Recognition Evaluation," Multimodal Technologies for Perception of Humans, pp. 373-389, Springer-Verlag, 2008.
3. R. Bakis, S. Chen, P. Gopalakrishnan, R. Gopinath, L. Polymenakos, and M. Franz, "Transcription of Broadcast News Shows with the IBM Large Vocabulary Speech Recognition System," Proc. Speech Recognition Workshop, pp. 67-72, 1997.
4. J. luc Gauvain, L. Lamel, and G. Adda, "Partitioning and Transcription of Broadcast News Data," Proc. Int'l Conf. Spoken Language Processing, pp. 1335-1338, 1998.
5. J. Pelecanos and S. Sridharan, "Feature Warping for Robust Speaker Verification," Proc. Int'l Speech Comm. Assoc. Workshop Speaker Recognition: A Speaker Oddyssey, 2001.
6. M. Yamaguchi, M. Yamashita, and S. Matsunaga, "Spectral Cross-Correlation Features for Audio Indexing of Broadcast News and Meetings," Proc. Ninth European Conf. Speech Comm. and Technology, pp. 613-616, 2005. (Pubitemid 43908137)
7. P. Delacourt, D. Kryze, and C.J. Wellekens, "DISTBIC: A Speaker-Based Segmentation for Audio Data Indexing," Speech Comm., vol. 32, pp. 111-126, 2000.
8. S.S. Chenn and P. Gopalakrishnan, "Clustering via the Bayesian Information Criterion with Applications in Speech Recognition," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 645-648, 1998.
9. K. Mori and S. Nakagawa, "Speaker Change Detection and Speaker Clustering Using VQ Distortion for Broadcast News Speech Recognition," Systems and Computers in Japan, vol. 34, pp. 413-416, 2001. (Pubitemid 32839275)
10. R. Gangadharaiah, B. Narayanaswamy, and Narayanaswamy, "A Novel Method for Two-Speaker Segmentation," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2004.
11. C. Wooters and M. Huijbregts, "The ICSI RT07s Speaker Diarization System," Multimodal Technologies for Perception of Humans, pp. 509-519, Springer-Verlag, 2008.
12. Z. Barzelay and Y.Y. Schechner, "Harmony in Motion," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2007.
13. E. Kidron, Y.Y. Schechner, and M. Elad, "Pixels That Sound," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 88-95, 2005.
14. J. Hershey and J. Movellan, "Using Audio-Visual Synchrony to Locate Sounds," Advances in Neural Information Processing Systems, vol. 12, pp. 813-819, MIT Press, 1999.
15. T. Darrell, J.W. Fisher III, and P. Viola, "Audiovisual Segmentation and the Cocktail Party Effect," Proc. Int'l Conf. Multimodal Interfaces, pp. 32-40, 2000.
16. H.J. Nock, G. Iyengar, and C. Neti, "Multimodal Processing by Finding Common Cause," Comm. ACM, vol. 47, no. 1, pp. 51-56, 2004.
17. G. Iyengar, H.J. Nock, and C. Neti, "Audio-Visual Synchrony for Detection of Monologues in Video Archives," Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 329-332, 2003.
18. J.W. Fisher III and T. Darrell, "Probabilistic Models and Informative Subspaces for Audiovisual Correspondence," Proc. European Conf. Computer Vision, Part III, pp. 592-603, 2002.
19. R. Cutler, Y. Rui, A. Gupta, J. Cadiz, I. Tashev, L. wei He, A. Colburn, Z.Z.Z. Liu, and S. Silverberg, "Distributed Meetings: A Meeting Capture and Broadcasting System," Proc. 10th ACM Int'l Conf. Multimedia, 2002.
20. Y. Chen and Y. Rui, "Real-Time Speaker Tracking Using Particle Filter Sensor Fusion," Proc. IEEE, vol. 92, no. 3, pp. 485-494, Mar. 2004. (Pubitemid 40890755)
21. N. Checka, K. Wilson, M. Siracusa, and T. Darrell, "Multiple Person and Speaker Activity Tracking with a Particle Filter," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, May 2004.
22. D. Gatica-Perez, G. Lathoud, J.-M. Odobez, and I. McCowan, "Multimodal Multispeaker Probabilistic Tracking in Meetings," IDIAP-RR 66, IDIAP, 2004.
23. M.J. Beal, H. Attias, and N. Jojic, "Audio-Video Sensor Fusion with Probabilistic Graphical Models," Proc. European Conf. Computer Vision, 2002.
24. Z. Ghahramani, M.I. Jordan, and P. Smyth, "Factorial Hidden Markov Models," Machine Learning, MIT Press, 1997.
25. L.R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
26. P. Viola and M. Jones, "Robust Real-Time Object Detection," Proc. Second Int'l Workshop Statistical and Computational Theories of Vision-Modeling, Learning, Computing, and Sampling, 2001.
27. C. Dance, J. Willamowski, L. Fan, C. Bray, and G. Csurka, "Visual Categorization with Bags of Keypoints," Proc. European Conf. Computer Vision Int'l Workshop Statistical Learning in Computer Vision, 2004.
28. T. Darrell, J.W. Fisher III, W.T. Freeman, and P. Viola, "Learning Joint Statistical Models for Audio-Visual Fusion and Segregation," Advances in Neural Information Processing Systems, vol. 13, pp. 772-778, MIT Press, 2000.
29. B. Thomas, "An Essay Towards Solving a Problem in the Doctrine of Chances," Philosophical Trans. Royal Soc., vol. 53, pp. 370-418, 1763.
30. A.K. Noulas, N. Vlassis, and B.J.A. Krö se, "Cross Entropy for Learning in Multi-Modal Streams," Proc. Joint Workshop Multi-Modal Interaction and Related Machine Learning Algorithms, 2007.
31. A.K. Noulas and B.J.A. Krö se, "EM Detection of Common Origin of Multi-Modal Cues," Proc. Int'l Conf. Multimodal Interfaces, pp. 201-208, 2006. (Pubitemid 47113450)
32. A.K. Noulas and B.J.A. Krose, "A Hybrid Generative-Discriminative Approach to Speaker Diarization," Proc. Fifth Int'l Workshop Machine Learning for Multimodal Interaction, pp. 98-109, 2008.
33. A. Noulas, "Audiovisual Fusion for Speaker Diarization," PhD dissertation, Univ. of Amsterdam, 2010.
34. J. Carletta, "Announcing the AMI Meeting Corpus," The ELRA Newsletter, vol. 1, no. 1, pp. 3-5, Jan.-Mar. 2006.
35. D.C. Moore, "The IDIAP Smart Meeting Room," IDIAP, IDIAPCOM 07, 2002.
36. X. Anguera, C. Wooters, and J. Hernando, "Automatic Cluster Complexity and Quantity Selection: Towards Robust Speaker Diarization," Proc. Third Joint Workshop Multimodal Interaction and Related Machine Learning Algorithms, pp. 248-256, 2006.