Theoretical analysis of diversity in an ensemble of automatic speech recognition systems

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 3, 2014, Pages 711-726

  Audhkhasi, Kartik ^a   Zavou, Andreas M ^b   Georgiou, Panayiotis G ^a   Narayanan, Shrikanth S ^a  

^a University of Southern California ^*  (United States)

^b CYPRUS UNIVERSITY OF TECHNOLOGY   (Cyprus)

Author keywords

Ambiguity decomposition; Automatic speech recognition; Discriminative training; Diversity; Ensemble methods; ROVER; System combination

Indexed keywords

AMBIGUITY DECOMPOSITION; AUTOMATIC SPEECH RECOGNITION; DISCRIMINATIVE TRAINING; DIVERSITY; ENSEMBLE METHODS; ROVER; SYSTEM COMBINATION;

ALGORITHMS; ECONOMIC AND SOCIAL EFFECTS;

SPEECH RECOGNITION;

EID: 84898080333     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2303295     Document Type: Article

References (65)

1. K. Audhkhasi, A. M. Zavou, P. G. Georgiou, and S. S. Narayanan, "Empirical link between hypothesis diversity and fusion performance in an ensemble of automatic speech recognition systems," in Proc. Interspeech, 2013.
2. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ, USA: Prentice-Hall, 1993.
3. H. Soltau, G. Saon, B. Kingsbury, H. K. J. Kuo, L. Mangu, D. Povey, and A. Emami, "Advances in Arabic speech transcription at IBM under the DARPA GALE program," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 5, pp. 884-894, Jul. 2009.
4. D. Stallard, F. Choi, C. L. Kao, K. Krstovski, P. Natarajan, R. Prasad, S. Saleem, and K. Subramanian, "The BBN 2007Displayless English/Iraqi Speech-to-Speech Translation System," in Proc. Interspeech, 2007.
5. S. F. Chen, B. Kingsbury, L. Mangu, D. Povey, G. Saon, H. Soltau, and G. Zweig, "Advances in speech transcription at IBM under the DARPA EARS program," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 5, pp. 1596-1608, Sep. 2006.
6. G. Tur, A. Stolcke, L. Voss, J. Dowding, B. Favre, R. Fernandez, M. Frampton, M. Frandsen, C. Frederickson, and M. Graciarena, "The CALO meeting speech recognition and understanding system," Proc. IEEE SLT, pp. 69-72, 2008.
7. J. Cui, X. Cui, B. Ramabhadran, J. Kim, B. Kingsbury, J. Mamou, L. Mangu, M. Picheny, T. N. Sainath, and A. Sethy, "Developing speech recognition systems for corpus indexing under the IARPA BABEL program," in Proc. ICASSP, 2013, pp. 6353-6357.
8. J. Fiscus, "A post processing system to yield reduced word error rates: Recognizer Output Voting Error Reduction (ROVER)," in Proc. ASRU, 1997, pp. 347-354.
9. H. Schwenk, "Using boosting to improve a hybrid HMM/neural network speech recognizer," in Proc. IEEE ICASSP, 1999, vol. 2, pp. 1009-1012.
10. G. Cook and T. Robinson, "Boosting the performance of connectionist large vocabulary speech recognition," in Proc. ICSLP, 1996, vol. 3, pp. 1305-1308.
11. C. Dimitrakakis and S. Bengio, "Boosting HMMs with an application to speech recognition," in Proc. ICASSP, 2004, vol. 5, pp. 618-621.
12. O. Siohan, B. Ramabhadran, and B. Kingsbury, "Constructing ensembles of ASR systems using randomized decision trees," in Proc. ICASSP, 2005, vol. 1, pp. 197-200.
13. J. Xue and Y. Zhao, "Random forests of phonetic decision trees for acoustic modeling in conversational speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 3, pp. 519-528, Mar. 2008.
14. G. Saon and H. Soltau, "Boosting systems for large vocabulary continuous speech recognition," Speech Commun., vol. 54, no. 2, pp. 212-218, 2012.
15. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996.
16. Y. Freund and R. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," in Computational learning theory. New York, NY, USA: Springer, 1995, pp. 23-37.
17. J. H. Friedman, "Greedy function approximation: A gradient boosting machine," Ann. Statist., vol. 29, no. 5, pp. 1189-1232, 2001.
18. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
19. C. Breslin, "Generation and combination of complementary systems for automatic speech recognition," Ph.D. dissertation, Cambridge Univ. Engi. Dept. and Darwin College, Cambridge, U.K., 2008.
20. X. Cui, J. Huang, and J.-T. Chien, "Multi-view and multi-objective semi-supervised learning for HMM-based automatic speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 7, pp. 1923-1935, Sep. 2012.
21. X. Chen and Y. Zhao, "Building acoustic model ensembles by data sampling with enhanced trainings and features," IEEE Trans. Audio, Speech, Language Process., vol. 21, no. 3, pp. 498-507, Mar. 2013.
22. L. I. Kuncheva, Combining pattern classifiers: methods and algorithms. New York, NY, USA: Wiley-Interscience, 2004.
23. T. Dietterich, "Ensemble methods inmachine learning," Multiple Classifier Syst., pp. 1-15, 2000.
24. A. Krogh and J. Vedelsby, "Neural network ensembles, cross validation, and active learning," Adv. Neural Inf. Process. Syst., pp. 231-238, 1995.
25. N. Ueda and R. Nakano, "Generalization error of ensemble estimators," in Proc. IEEE Int. Conf. Neural Netw., 1996, vol. 1, pp. 90-95.
26. K. Tumer and J. Ghosh, "Analysis of decision boundaries in linearly combined neural classifiers," Pattern Recogn., vol. 29, no. 2, pp. 341-348, 1996. (Pubitemid 126397840)
27. Y. Liu and X. Yao, "Ensemble learning via negative correlation," Neural Netw., vol. 12, no. 10, pp. 1399-1404, 1999.
28. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Comput., vol. 4, no. 1, pp. 1-58, 1992.
29. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, and P. Schwarz, "The Kaldi Speech Recognition Toolkit," in Proc. ASRU, Dec. 2011.
30. J. Laferty, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. San Rafael, CA, USA: Morgan Kaufmann, 2001, pp. 282-289.
31. P. Xu, D. Karakos, and S. Khudanpur, "Self-supervised discriminative training of statistical language models," in Proc. ASRU, 2009, pp. 317-322.
32. L. I. Kuncheva and C. J. Whitaker, "Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy," Mach. Learn., vol. 51, no. 2, pp. 181-207, 2003.
33. C. A. Shipp and L. I. Kuncheva, "Relationships between combination methods and measures of diversity in combining classifiers," Inf. Fusion, vol. 3, no. 2, pp. 135-148, 2002.
34. S. Young, G. Evermann, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK book. Cambridge, U.K.: Cambridge Univ. Eng. Dept., 2002.
35. K.-F. Lee, H.-W. Hon, and R. Reddy, "An overview of the SPHINX speech recognition system," IEEE Trans. Acoust., Speech, Signal Process., vol. 38, no. 1, pp. 35-45, Jan. 1990. (Pubitemid 20665377)
36. C. Allauzen, M. Riley, J. Schalkwyk, W. Skut, and M. Mohri, "OpenFST: A general and efficient weighted finite-state transducer library," Implement. Applicat. Automata, pp. 11-23, 2007.
37. D. B. Paul and J. M. Baker, "The design for the Wall Street Journal-based CSR corpus," in Proc. Workshop Speech Natural Lang., 1992, pp. 357-362, Association for Computational Linguistics.
38. J. Fiscus, J. Garofolo, M. Przybocki, W. Fisher, and D. Pallett, "English broadcast news speech (HUB4)," Linguist. Data Consortium, Philadelphia, 1997.
39. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, and A. Stolcke, "The ICSI meeting corpus," in Proc. ICASSP, 2003, vol. 1, pp. 364-367.
40. D. Povey, "Discriminative training for large vocabulary speech recognition," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 2003.
41. H. Jiang, "Confidence measures for speech recognition: A survey," Speech Commun., vol. 45, no. 4, pp. 455-470, 2005. (Pubitemid 40423290)
42. L. Rabiner and B. Juang, "An introduction to hidden Markov models," IEEE ASSP Mag., vol. 3, no. 1, pp. 4-16, Jan. 1986.
43. T. Kemp and T. Schaaf, "Estimating confidence using word lattices," in Proc. Eurospeech, Rhodes, Greece, 1997, vol. 2, pp. 827-830.
44. F. Wessel, R. Schluter, K. Macherey, and H. Ney, "Confidence measures for large vocabulary continuous speech recognition," IEEE Trans. Speech Audio Process., vol. 9, no. 3, pp. 288-298, Mar. 2001. (Pubitemid 32286598)
45. C. White, J. Droppo, A. Acero, and J. Odell, "Maximum entropy confidence estimation for speech recognition," in Proc. ICASSP, 2007, vol. 4, pp. 809-812.
46. M. S. Seigel and P. C. Woodland, "Combining information sources for confidence estimation with CRF models," in Proc. INTERSPEECH, 2011, pp. 905-908.
47. P. J. Moreno, B. Logan, and B. Raj, "A boosting approach for confidence scoring," in Proc. 7th Eur. Conf. Speech Commun. Technol., 2001.
48. M. Weintraub, F. Beaufays, Z. Rivlin, Y. Konig, and A. Stolcke, "Neural-network based measures of confidence for word recognition," in Proc. ICASSP, 1997, vol. 2, pp. 887-890.
49. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. ICML-01, 2001, pp. 282-289.
50. D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization," Math. Program., vol. 45, no. 1-3, pp. 503-528, 1989. (Pubitemid 20660315)
51. F. Eyben, M. Wöllmer, and B. Schuller, "Opensmile: The Munich versatile and fast open-source audio feature extractor," in Proc. Int. Conf. Multimedia, 2010, pp. 1459-1462, ACM.
52. D. J. Litman, J. B. Hirschberg, and M. Swerts, "Predicting automatic speech recognition performance using prosodic cues," in Proc. 1st North Amer. Chap. Assoc. Comput. Linguist. Conf., 2000, pp. 218-225, Association for Computational Linguistics.
53. S. Goldwater, D. Jurafsky, and C. D. Manning, "Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase speech recognition error rates," Speech Commun., vol. 52, no. 3, pp. 181-200, 2010.
54. J. Hirschberg, D. Litman, and M. Swerts, "Prosodic and other cues to speech recognition failures," Speech Commun., vol. 43, no. 1, pp. 155-175, 2004.
55. Y. Liu, E. Shriberg, and A. Stolcke, "Automatic disfluency identification in conversational speech using multiple knowledge sources," in Proc. Eurospeech, Geneva, Switzerland, 2003, vol. 1, pp. 957-960.
56. C. Parada, M. Dredze, D. Filimonov, and F. Jelinek, "Contextual information improves OOV detection in speech," in Proc. NAACL, 2010.
57. W. Wang, P. Lu, and Y. Yan, "An improved hierarchical speaker clustering," Acta Acoustica, 2006.
58. A. Stolcke, "SRILM - an extensible language modeling toolkit," in Proc. ICSLP, 2002, pp. 901-904.
59. F. Diehl and P. C. Woodland, "Complementary phone error training," in Proc. Interspeech, 2012.
60. C. Breslin and M. J. F. Gales, "Generating complementary systems for speech recognition," in Proc. Interspeech, 2006.
61. Y. Tachioka and S. Watanabe, "Discriminative training of acoustic models for system combination," in Proc. Interspeech, 2013.
62. V. Doumpiotis and W. Byrne, "Lattice segmentation and minimum Bayes risk discriminative training for large vocabulary continuous speech recognition," Speech Commun., vol. 48, no. 2, pp. 142-160, 2006. (Pubitemid 43012028)
63. G. Heigold, H. Ney, R. Schluter, and S. Wiesler, "Discriminative training for automatic speech recognition: Modeling, criteria, optimization, implementation, and performance," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 58-69, Nov. 2012.
64. D. Povey, P. C. Woodland, and M. J. F. Gales, "Discriminative MAP for acoustic model adaptation," in Proc. ICASSP, 2003, vol. 1, pp. 312-315.
65. K. Audhkhasi, A. Sethy, B. Ramabhadran, and S. S. Narayanan, "Creating ensemble of diverse maximum entropy models," in Proc. ICASSP, 2012, pp. 4845-4848.