Speaker segmentation and clustering

Signal Processing

Volumn 88, Issue 5, 2008, Pages 1091-1124

  Kotti, Margarita ^a   Moschou, Vassiliki ^a   Kotropoulos, Constantine ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Diarization; Speaker clustering; Speaker segmentation

Indexed keywords

CLUSTERING ALGORITHMS; PROBABILISTIC LOGICS;

MOVIE ANALYSIS; SPEAKER CLUSTERING; SPEAKER SEGMENTATION;

SPEECH PROCESSING;

EID: 38949122754     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2007.11.017     Document Type: Review

References (103)

1. A. Solomonoff, A. Mielke, M. Schmidt, H. Gish, Clustering speakers by their voices, in: Proceedings of the 1998 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, Seattle, USA, May 1998, pp. 757-760.
2. R. Sinha, S.E. Tranter, M.J.F. Gales, P.C. Woodland, The Cambridge University March 2005 speaker diarisation system, in: Proceedings of the European Conference on Speech Communication and Technology, Lisbon, Portugal, September 2005, pp. 2437-2440.
3. ISO/IEC 15938-4:2001, Multimedia content description interface-part 4: audio, Version 1.0, 2001.
4. Manjunath B.S., Salembier P., Sikora T., and Salembier P. Introduction to MPEG 7: Multimedia Content Description Language (2002), Wiley, West Sussex, England
5. H.G. Kim, T. Sikora, Comparison of MPEG-7 audio spectrum projection features and MFCC applied to speaker recognition, sound classification and audio segmentation, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 5, Montreal, Canada, May 2004, pp. 925-928.
6. H.G. Kim, T. Sikora, Audio spectrum projection based on several basis decomposition algorithms applied to general sound recognition and audio segmentation, in: Proceedings of the 12th European Signal Processing Conference, Vienna, Austria, September 2004, pp. 1047-1050.
7. M. Kotti, E. Benetos, C. Kotropoulos, Automatic speaker change detection with the Bayesian information criterion using MPEG-7 features and a fusion scheme, in: Proceedings of the 2006 IEEE International Symposium on Circuits and Systems, Kos, Greece, May 2006.
8. M. Kotti, L.G.P.M. Martins, E. Benetos, J.S. Cardoso, C. Kotropoulos, Automatic speaker segmentation using multiple features and distance measures: a comparison of three approaches, in: Proceedings of the 2006 IEEE International Conference on Multimedia and Expo, Toronto, Canada, July 2006, pp. 1101-1104.
9. W.H. Tsai, S.S. Cheng, H.M. Wang, Speaker clustering of speech utterances using a voice characteristic reference space, in: Proceedings of the International Conference on Spoken Language Processing, Jeju Island, Korea, October 2004.
10. D. Liu, F. Kubala, Online speaker clustering, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, Montreal, Canada, May 2004, pp. 333-336.
11. S.S. Chen, P.S. Gopalakrishnan, Clustering via the Bayesian information criterion with applications in speech recognition, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, Seattle, USA, May 1998, pp. 645-648.
12. S. Meignier, J.F. Bonastre, S. Igounet, E-HMM approach for learning and adapting sound models for speaker indexing, in: Proceedings of the Odyssey Speaker and Language Recognition Workshop, Crete, Greece, June 2001, pp. 175-180.
13. J. Ajmera, H. Bourlard, I. Lapidot, I. McCowan, Unknown-multiple speaker clustering using HMM, in: Proceedings of the International Conference on Spoken Language Processing, CO, USA, September 2002, pp. 573-576.
14. X. Zhu, C. Barras, S. Meignier, J.-L. Gauvain, Combining speaker identification and BIC for speaker diarization, in: Proceedings of the InterSpeech, Lisbon, Portugal, September 2005, pp. 2441-2444.
15. Meignier S., Moraru D., Fredouille C., Bonastre J.F., and Besacier L. Step-by-step and integrated approaches in broadcast news speaker diarization. Comput. Speech Language 20 2-3 (April-July 2006) 303-330
16. Barras C., Zhu X., Meignier S., and Gauvain J.L. Multistage speaker diarization of broadcast news. IEEE Trans. Audio Speech Language Process. 14 5 (September 2006) 1505-1512
17. Tranter S.E., and Reynolds D.A. An overview of automatic speaker diarization systems. IEEE Trans. Audio Speech Language Process. 14 5 (September 2006) 1557-1565
18. Campbell J.P. Speaker recognition: a tutorial. Proc. IEEE 85 9 (September 1997) 1437-1462
19. V. Wan, W.M. Campbell, Support vector machines for speaker verification and identification, in: Proceedings of the Neural Networks for Signal Processing, vol. 10, Sydney, Australia, December 2000, pp. 775-784.
20. Reynolds D.A., Quatiery T.F., and Dunn R.B. Speaker verification using adapted Gaussian mixture models. Digital Signal Process. 10 (October 2000) 19-41
21. Gales M.J.F., Kim D.Y., Woodland P.C., Chan H.Y., Mrva D., Sinha R., and Tranter S.E. Progress in the CU-HTK broadcast news transcription system. IEEE Trans. Speech Audio Process. 14 5 (September 2006) 1513-1525
22. National Institute of Standards and Technology (NIST)-The Segmentation Task: Find the Story Boundaries 〈http://www.nist.gov/speech/tests/tdt/tdt99/presentations/NIST_segmentation/index.htm〉.
23. The Center for Spoken Language Research of the Colorado University (CSLR) 〈http://cslr.colorado.edu/〉.
24. International Computer Science Institute-Speech Research Group Berkeley 〈http://www.icsi.berkeley.edu/groups/speech/〉.
25. Speech Analysis and Interpretation Laboratory (SAIL) at the University of Southern California 〈http://sail.usc.edu/projectsIntro.php〉.
26. International Speech Technology and Research (STAR) Laboratory at Stanford research institute (SRI) 〈http://www.speech.sri.com/projects/sieve/〉.
27. Microsoft Audio Projects 〈http://research.microsoft.com/users/llu/Audioprojects.aspx〉.
28. The Institut Dalle Molle d'Intelligence Artificielle Perceptive (IDIAP) Research Institute 〈http://www.idiap.ch/speech_processing.php〉.
29. The Laboratoire d'Informatique pour la Mècanique et les Sciences de l'Ingènieur (LIMSI) Spoken Language Processing Group 〈http://www.limsi.fr/TLP〉.
30. The Department of Speech, Music and Hearing of the Royal Institute of Technology (KTH) at Stockholm 〈http://www.speech.kth.se〉.
31. The Chair of Computer Science VI, Computer Science Department, Aachen University 〈http://www-i6.informatik.rwth-aachen.de〉.
32. The Infant Speech Segmentation Project at Berkeley University 〈http://www-gse.berkeley.edu/research/completed/InfantSpeech.html〉.
33. Language Science Research Group, Washington University 〈http://lsrg.cs.wustl.edu〉.
34. The University College of London Psychology Speech Group, speech segmentation issues 〈http://www.speech.psychol.ucl.ac.uk〉.
35. Oppenheim A.V., and Schafer R.W. Digital Signal Processing (1975), Prentice-Hall, Englewood Cliffs, NJ
36. L. Lu, H. Zhang, Speaker change detection and tracking in real-time news broadcast analysis, in: Proceedings of the ACM Multimedia 2002, Juan-les-Pins, France, December 2002, pp. 602-610.
37. Lu L., and Zhang H. Unsupervised speaker segmentation and tracking in real-time audio content analysis. Multimedia Systems 10 4 (April 2005) 332-343
38. A. Tritschler, R. Gopinath, Improved speaker segmentation and segments clustering using the Bayesian information criterion, in: Proceedings of the 6th European Conference on Speech Communication and Technology, Budapest, Hungary, September 1999, pp. 679-682.
39. Delacourt P., and Wellekens C.J. DISTBIC: a speaker-based segmentation for audio data indexing. Speech Comm. 32 (September 2000) 111-126
40. S. Know, S. Narayanan, Speaker change detection using a new weighted distance measure, in: Proceedings of the International Conference on Spoken Language, vol. 4, CO, USA, September 2002, pp. 2537-2540.
41. T. Wu, L. Lu, K. Chen, H. Zhang, UBM-based real-time speaker segmentation for broadcasting news, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, Hong Kong, April 2003, pp. 193-196.
42. S.S. Cheng, H.M. Wang, A sequential metric-based audio segmentation method via the Bayesian information criterion, in: Proceedings of the 8th European Conference on Speech Communication and Technology, Geneva, Switzerland, September 2003, pp. 945-948.
43. Ajmera J., McCowan I., and Bourlard H. Robust speaker change detection. IEEE Signal Process. Lett. 11 8 (August 2004) 649-651
44. H. Kim, D. Elter, T. Sikora, Hybrid speaker-based segmentation system using model-level clustering, in: Proceedings of the 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. I, Philadelphia, USA, March 2005, pp. 745-748.
45. 2 statistic and the Bayesian information criterion. IEEE Trans. Speech Audio Process. 13 4 (July 2005) 467-474
46. Know S., and Narayanan S. Unsupervised speaker indexing using generic models. IEEE Trans. Speech Audio Process. 13 5 (September 2005) 1004-1013
47. Wu C.H., Chiu Y.H., Shia C.J., and Lin C.Y. Automatic segmentation and identification of mixed-language speech using delta-BIC and LSA-based GMMs. IEEE Trans. Audio Speech Language Process. 14 1 (January 2006) 266-276
48. T. Wu, L. Lu, K. Chen, H. Zhang, Universal background models for real-time speaker change detection, in: Proceedings of the 9th International Conference on Multimedia Modeling, Tamshui, Taiwan, January 2003, pp. 135-149.
49. S.E. Tranter, K. Yu, G. Evermann, P.C. Woodland, Generating and evaluating segmentations for automatic speech recognition of conversational telephone speech, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Montreal, Canada, May 2004, pp. 433-477.
50. D. Wang, L. Lu, H.J. Zhang, Speech segmentation without speech recognition, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, Hong Kong, April 2003, pp. 468-471.
51. Wu C.H., and Hsieh C.H. Multiple change-point audio segmentation and classification using an MDL-based Gaussian model. IEEE Trans. Audio Speech Language Process. 14 2 (March 2006) 647-657
52. R. Huang, J.H.L. Hansen, Advances in unsupervised audio segmentation for the broadcast news and ngsw corpora, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Montreal, Canada, May 2004, pp. 741-744.
53. Kim H.-G., Moreau N., and Sikora T. MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval (2005), Wiley, West Sussex, England
54. Deller J.R., Hansen J.H.L., and Proakis J.G. Discrete-Time Processing of Speech Signals (1999), Wiley, IEEE, New York
55. Huang X.D., Acero A., and Hon H.-S. Spoken Language Processing: A Guide to Theory, Algorithm, and System Development (2001), Pearson Education, Prentice-Hall, Upper River Saddle, NJ
56. Sondhi M.M. New methods of pitch extraction. IEEE Trans. Audio Electroacoustics 16 2 (June 1968) 262-266
57. Hess W.J. Pitch and voicing determination. In: Furui S., and Sondhi M.M. (Eds). Advances in Speech Signal Processing (1991), Marcel Dekker Inc., New York
58. Ververidis D., and Kotropoulos C. Emotional speech recognition: resources, features, and methods. Speech Comm. 48 9 (September 2006) 1162-1181
59. B. Li, Y. Li, C. Wang, C. Zhang, A new efficient pitch-tracking algorithm, in: Proceedings of the 2003 IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, vol. 2, Hunan, China, October 2003, pp. 1102-1107.
60. T. Kemp, M. Schmidt, M. Westphal, A. Waibel, Strategies for automatic segmentation of audio data, in: Proceedings of the 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 3, Istanbul, Turkey, June 2000, pp. 1423-1426.
61. M. Collet, D. Charlet, F. Bimbot, A correlation metric for speaker tracking using anchor models, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, Hong Kong, April 2003, pp. 713-716.
62. Pellom B.L., and Hansen J.H.L. Automatic segmentation of speech recorded in unknown noisy channel characteristics. Speech Comm. 25 1-3 (August 1998) 97-116
63. Ajmera J., McCowan I., and Bourland H. Speech/music segmentation using entropy and dynamism features in a HMM classification framework. Speech Comm. 40 3 (May 2003) 351-363
64. N. Mesgarani, S. Shamma, M. Slaney, Speech discrimination based on multiscale spectro-temporal modulations, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, Montreal, Canada, May 2004, pp. 601-604.
65. J.A. Arias, J. Pinquier, R. Andè-Obrecht, Evaluation of classification techniques for audio indexing, in: Proceedings of the 13th European Signal Processing Conference, Antalya, Turkey, September 2005.
66. Harb H., and Chen L. Audio-based description and structuring of videos. Internat. J. Digital Libraries 6 1 (February 2006) 70-81
67. Bimbot F., Magrin-Chagnolleau I., and Mathan L. Second-order statistical measures for text-independent speaker identification. Speech Comm. 17 1-2 (August 1995) 177-192
68. Hansen J.H.L., Huang R., Zhou B., Seadle M., Deller J.R., Gurijala A.R., Kurimo M., and Angkititrakul P. SpeechFind: advances in spoken document retrieval for a national gallery of the spoken word. IEEE Trans. Speech Audio Process. 13 5 (September 2005) 712-730
69. M. Cettolo, M. Vescovi, Efficient audio segmentation algorithms based on the BIC, in: Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 6, Hong Kong, April 2003, pp. 537-540.
70. M. Vescovi, M. Cettolo, R. Rizzi, A DP algorithm for speaker change detection, in: Proceedings of the 8th European Conference on Speech Communication and Technology, Geneva, Switzerland, September 2003, pp. 2997-3000.
71. Q. Jin, K. Laskowski, T. Schultz, A. Waibel, Speaker segmentation and clustering in meetings, in: Proceedings of the NIST Meeting Recognition Workshop, Montreal, Canada, May 2004, pp. 112-117.
72. Cettolo M., Vescovi M., and Rizzi R. Evaluation of BIC-based algorithms for audio segmentation. Comput. Speech Language 19 (April 2005) 1004-1013
73. Campbell N.A. Robust procedures in multivariate analysis I: robust covariance estimation. Appl. Statist. 29 3 (1980) 231-237
74. S. Cheng, H. Wang, Metric SEQDAC: a hybrid approach for audio segmentation, in: Proceedings of the 8th International Conference on Spoken Language Processing, Jeju, Korea, October 2004, pp. 1617-1620.
75. H. Gish, M.H. Siu, R. Rohlicek, Segregation of speakers for speech recognition and speaker identification, in: Proceedings of the 1991 IEEE International Conference on Acoustics, Speech, and Signal Processing, Toronto, Canada, April 1991, pp. 873-876.
76. J. Ajmera, G. Lathoud, I. McCowan, Clustering and segmenting speakers and their locations in meetings, in: Proceedings of the 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, Montreal, Canada, May 2004, pp. 605-608.
77. D.P.W. Ellis, J.C. Liu, Speaker turn segmentation based on between-channel differences, in: Proceedings of the NIST Meeting Recognition Workshop, Montreal, Canada, May 2004, pp. 112-117.
78. J. Alabiso, R. MacIntyre, D. Graff, 1997 English Broadcast News Transcripts (HUB4), Linguistic Data Consortium, Philadelphia, 1998.
79. Garofolo J.S. DARPA TIMIT Acoustic-Phonetic Continuous Speech Corpus (1993), Linguistic Data Consortium, Philadelphia
80. Godfrey J.J., and Holliman E. Switchboard-1 Release 2 (1997), Linguistic Data Consortium, Philadelphia
81. M. Federica, D. Giordani, P. Caletti, Development and evaluation of an Italian broadcast news corpus, in: Proceedings of the 2nd International Conference on Language Resources and Evaluation, Athens, Greece, May-June 2000, pp. 921-924.
82. S. Chen, P. Gopalakrishnan, Speaker, environment and channel change detection and clustering via the Bayesian information criterion, in: Proceedings of the DARPA Broadcast News Transcription Understanding Workshop, Landsdowne, VA, February 1998, pp. 127-132.
83. Wang H.M., Chen B., Kuo J.W., and Cheng S.S. MATBN: a mandarin Chinese broadcast news corpus. Comput. Linguistics Chinese Language Process. 10 2 (June 2005) 219-236
84. Graff D. TDT3 Mandarin Audio (2001), Linguistic Data Consortium, Philadelphia
85. Zhu Y., and Rong X. Unified fusion rules for multisensor multihypothesis network decision systems. IEEE Trans. System Man Cybernet. 33 4 (July 2003) 502-513
86. M. Kotti, E. Benetos, C. Kotropoulos, Computationally efficient and robust BIC-based speaker segmentation, IEEE Trans. Audio Speech Language Process., in revision.
87. The Linguistic Data Consortium 〈http://www.ldc.upenn.edu/〉.
88. Almpanidis G., and Kotropoulos C. Phonemic segmentation using the generalised Gamma distribution and small sample Bayesian information criterion. Speech Comm. 50 1 (January 2008) 38-55
89. W.-H. Tsai, H.-M. Wang, Speaker clustering of unknown utterances based on maximum purity estimation, in: Proceedings of the European Conference on Speech Communication and Technology, Lisbon, Portugal, September 2005.
90. J.-L. Gauvain, L. Lamel, G. Adda, Partitioning and transcription of broadcast news data, in: Proceedings of the International Conference on Spoken Language Processing, Sydney, Australia, December 1998, pp. 1335-1338.
91. Jain A.K., and Dubes R.C. Algorithms for Clustering Data (1988), Prentice-Hall, Englewood Cliffs, NJ
92. J. Ajmera, C. Wooters, A robust speaker clustering algorithm, in: Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, Virgin Islands, November 2003, pp. 411-416.
93. I. Voitovetsky, H. Guterman, A. Cohen, Validity criterion for unsupervised speaker recognition, in: Proceedings of the First Workshop Text, Speech, and Dialogue, Brno, Czech Republic, September 1998, pp. 321-326.
94. I. Voitovetsky, H. Guterman, A. Cohen, Unsupervised speaker classification using self-organizing maps, in: Proceedings of the IEEE Workshop Neural Networks for Signal Processing, Amelia Island, USA, September 1997, pp. 578-587.
95. I. Lapidot, H. Guterman, Resolution limitation in speakers clustering and segmentation problems, in: Proceedings of the 2001: A Speaker Odyssey, The Speaker Recognition Workshop, Chania, Greece, June 18-22, 2001, pp. 169-173.
96. Lapidot I., Guterman H., and Cohen A. Unsupervised speaker recognition based on competition between self-organizing maps. IEEE Trans. Neural Networks 13 4 (July 2002) 877-887
97. 1998 HUB4 Broadcast News Evaluation English Test Material, Linguistic Data Consortium, Philadelphia, 2000.
98. Graff D., and Alabiso J. 1996 English Broadcast News Transcripts (HUB4) (1997), Linguistic Data Consortium, Philadelphia
99. M. Przybocki, A. Martin, 2001 NIST Speaker Recognition Evaluation Corpus, Linguistic Data Consortium, Philadelphia, 2002.
100. H. Jin, F. Kubala, R. Schwartz, Automatic speaker clustering, in: Proceedings of the Speech Recognition Workshop, Chantilly, Virginia, 1997, pp. 108-111.
101. Fiscus J., Garofolo J., Przybocki M., Fisher W., and Pallett D. 1997 English Broadcast News Speech (HUB4) (1998), Linguistic Data Consortium, Philadelphia
102. C. Barras, X. Zhu, S. Meignier, J.-L. Gauvain, Improving speaker diarization, in: Proceedings of the Fall Rich Transcription Workshop (RT-04), Palisades, NY, November 2004 [Online]. Available: 〈http://www.limsi.fr/Individu/barras/publis/rt04f_diarization.pdf〉.
103. Graff D., Alabiso J., Fiscus J., Garofolo J., Fisher W., and Pallett D. 1996 English Broadcast News Dev and Eval (HUB4) (1997), Linguistic Data Consortium, Philadelphia