Hierarchical Bayesian combination of plug-in maximum a posteriori decoders in deep neural networks-based speech recognition and speaker adaptation

Pattern Recognition Letters

Volumn 98, Issue , 2017, Pages 1-7

  Huang, Zhen ^a   Siniscalchi, Sabato Marco ^a,b   Lee, Chin Hui ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b KORE UNIVERSITY OF ENNA   (Italy)

Author keywords

Automatic speech recognition; Bayesian learning; Deep neural networks; Sequential patterns; System combination

Indexed keywords

BAYESIAN NETWORKS; DECODING; DEEP NEURAL NETWORKS; HIDDEN MARKOV MODELS; HIERARCHICAL SYSTEMS; LOUDSPEAKERS; PATTERN RECOGNITION SYSTEMS;

AUTOMATIC SPEECH RECOGNITION; BAYESIAN LEARNING; CHARACTERISTIC DIFFERENCE; GAUSSIAN MIXTURE MODEL (GMMS); HIERARCHICAL BAYESIAN; MAXIMUM A POSTERIORI DECODERS; SEQUENTIAL PATTERNS; SYSTEM COMBINATION;

SPEECH RECOGNITION;

EID: 85026840738     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2017.08.001     Document Type: Article

References (63)

1. Abdel-Hamid, O., Jiang, H., Fast speaker adaptation of hybrid NN/HMM model for speech recognition based on discriminative learning of speaker code. Proceedings of ICASSP, 2013, 7942–7946.
2. Bernardo, J.M., Smith, A.F.M., Bayesian Theory. 1994, Wiley.
3. Bishop, C.M., Pattern Recognition and Machine Learning. 2006, Springer.
4. Boer, P.T.D., Kroese, D.P., Mannor, S., Rubinstein, R.Y., A tutorial on the cross-entropy method. Ann. Oper. Res. 134:1 (2005), 19–67.
5. Breiman, L., Bagging predictors. Mach. Learn. 24 (1996), 123–140.
6. Breiman, L., Random forests. Mach. Learn., 45, 2001, 2001.
7. Dahl, G.E., Yu, D., Deng, L., Acero, A., Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans. Audio Speech Lang. Process. 20:1 (2012), 30–42.
8. DeGroot, M.H., Morris, H., Optimal statistical decisions. 82, 2005, John Wiley & Sons.
9. Deng, L., Li, X., Machine learning paradigms for speech recognition: an overview. IEEE Trans. Audio Speech Lang. Process. 21 (2013), 1060–1089.
10. Deng, L., Platt, J.C., Ensemble deep learning for speech recognition. Proceedings of INTERSPEECH, 2014, 1915–1919.
11. Evermann, G., Woodland, P.C., Posterior probability decoding, confidence estimation and system combination. Proceedings of Speech Transcription Workshop, 27, 2000.
12. Fiscus, J., Fisher, W.M., Martin, A.F., Przybocki, M.A., Pallett, D.S., 2000 NIST evaluation of conversational speech recognition over the telephone: english and mandarin performance results. Proceedings of Speech Transcription Workshop, 2000.
13. Fiscus, J.G., A post-processing system to yield reduced word error rates: recognizer output voting error reduction (rover). Proceedings of ASRU, 1997, 347–354.
14. Freund, Y., Schapire, R.E., Experiments with a new boosting algorithm. Proceedings of International Conference on Machine Learning, 1996, 148–156.
15. Gauvain, J., Lee, C.-H., Maximum a posteriori estimation for multivariate gaussian mixture observations of Markov chains. IEEE Trans. Speech Audio Process., 2, 1994.
16. Gauvain, J.-L., Lamel, L., Large vocabulary continuous speech recognition: advances and applications. Proc. IEEE 88:8 (2000), 1181–1200.
17. Gemello, R., Mana, F., Scanzio, S., Laface, P., Mori, R.D., Linear hidden transformations for adaptation of hybrid ANN/HMM models. Speech Commun. 49:10 (2007), 827–835.
18. Gergen, S., Zeiler, S., Abdelaziz, A.H., Nickel, R., Kolossa, D., Dynamic stream weighting for turbo-decoding-based audiovisual ASR. Proceedings of Interspeech, San Francisco, CA, USA, 2016, 2135–2139.
19. Ghahramani, Z., Kim, H.-C., Bayesian model combination. Technical Report, 2003, Gatsby Computational Neuroscience Unit, University College London.
20. Godfrey, J.J., Holliman, E., Switchboard-1 release 2. Linguistic Data Consortium, 1997.
21. Grézl, F., Karafiát, M., Kontár, S., Cernocky, J., Probabilistic and bottle-neck features for LVCSR of meetings. Proceedings of ICASSP, 4, 2007, IV–757.
22. Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T.N., et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process. Mag. 29:6 (2012), 82–97.
23. Hinton, G.E., Salakhutdinov, R.R., Reducing the dimensionality of data with neural networks. Science 313:5786 (2006), 504–507.
24. Huang, Z., Li, J., Siniscalchi, S.M., Chen, I.-F., Wu, J., Lee, C.-H., Rapid adaptation for deep neural networks through multi-task learning. Proceedings of Interspeech, 2015.
25. Huang, Z., Siniscalchi, S.M., Chen, I.-F., Li, J., Wu, J., Lee, C.-H., Maximum a posteriori adaptation of network parameters in deep models. INTERSPEECH, 2015, 1076–1080.
26. Huang, Z., Siniscalchi, S.M., Lee, C.-H., Bayesian unsupervised batch and online speaker adaptation of activation function parameters in deep models for automatic speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process., 25, 2017.
27. Jalalvand, S., Negria, M., Falavignaa, D., Matassonia, M., Turchia, M., Automatic quality estimation for ASR system combination. Comput. Speech Lang., 2017.
28. Kessentini, Y., Burger, T., Paquet, T., A dempster-shafer theory based combination of handwriting recognition systems with multiple rejection strategies. Pattern Recognit. Lett. 48 (2015), 534–544.
29. Kittler, J., Hatef, M., Duin, R.P.W., Matas, J., On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20:3 (1998), 226–239.
30. Koo, M.-W., Lee, C.-H., Juang, B.-H., Speech recognition and utterance verification based on a generalized confidence score. IEEE Trans. Speech Audio Process. 9:8 (2001), 821–831.
31. Lamel, L., Adda, G., On designing pronunciation lexicons for large vocabulary continuous speech recognition. Proceedings of ICSLP, 1, 1996.
32. Lee, C.-H., Huo, Q., On adaptive decision rules and decision parameter adaptation for automatic speech recognition. Proc. IEEE 88 (2000), 1241–1269.
33. Lee, C.-H., Siniscalchi, S.M., An information-extraction approach to speech processing: analysis, detection, verification, and recognition. Proc. IEEE 101:5 (2013), 1089–1115.
34. Li, J., Huang, J.-T., Gong, Y., Factorized adaptation for deep neural network. Proceedings of ICASSP, 2014.
35. M.-Y. Hwang, M.-Y., Huang, X., Shared-distribution hidden markov models for speech recognition. IEEE Trans. Speech Audio Process. 1:4 (1993), 414–420.
36. Mangu, L., Brill, E., Stolcke, A., Finding consensus in speech recognition: word error minimization and other applications of confusion networks. Comput. Speech Lang. 14:4 (2000), 373–400.
37. Mitra, V., et al. Fusion strategies for robust speech recognition and keyword spotting for channel- and noise-degraded speech. Proceedings of Interspeech, San Francisco, CA, USA, 2016, 3683–3687.
38. Morris, J., Fosler-Lussier, E., Conditional random fields for integrating local discriminative classifiers. IEEE Trans. Audio Speech Lang. Process. 16:3 (2008), 617–628.
39. Nadas, A., Optimal solution of a training problem in speech recognition. IEEE Trans. Acoust. Speech Signal Process. 33:1 (1985), 326–329.
40. Ney, H., Ortmanns, S., Progresses in dynamic programming search for LVCSR. Proc. IEEE 88:8 (2000), 1224–1240.
41. Norris, J.R., Markov Chains 2. 1998, Cambridge University Press.
42. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., Schwarz, P., Silovskỳ, J., Stemmer, G., Veselỳ, K., The Kaldi speech recognition toolkit. Proceedings of ASRU, 2011.
43. Qian, Y., Bi, M., Tan, T., Yu, K., Very deep convolutional neural networks for noise robust speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 24:12 (2016), 2263–2276.
44. Rabiner, L., A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77:2 (1989), 257–286.
45. Remes, U., López, A.R., Palomäki, D., Bounded conditional mean imputation with observation uncertainties and acoustic model adaptation. IEEE/ACM Trans. Audio Speech Lang. Process. 23 (2015), 1198–1208.
46. Rokach, L., Ensemble-based classifiers. Artif. Intell. Rev. 33:1–2 (2010), 1–39.
47. Saon, G., Soltau, H., Nahamoo, D., Picheny, M., Speaker adaptation of neural network acoustic models using i-vectors. Proc. ASRU, 2013, 55–59.
48. Seide, F., Li, G., Chen, X., Yu, D., Feature engineering in context-dependent deep neural networks for conversational speech transcription. Proc. ASRU, 2011, 24–29.
49. Shinoda, K., Lee, C.-H., A structural Bayes approach to speaker adaptation. IEEE Trans. Speech Audio Process. 9:3 (2001), 276–287.
50. Siniscalchi, S.M., Li, J., Lee, C.-H., Hermitian polynomial for speaker adaptation of connectionist speech recognition systems. IEEE Trans. Audio Speech Lang. Process. 21:10 (2013), 2152–2161.
51. Swietojanski, P., Ghoshal, A., Renals, S., Revisiting hybrid and GMM-HMM system combination techniques. Proceedings of ICASSP, 2013, 6744–6748.
52. Swietojanski, P., Li, J., Renals, S., Learning hidden unit contributions for unsupervised acoustic model adaptation. IEEE/ACM Trans. Audio Speech Lang. Process. 24 (2016), 1450–1463.
53. Tang, X.-S., Has, K., Wei, H., Ding, Y., Using line segments to train multi-stream stacked autoencoders for image classification. Pattern Recognit. Lett. 94 (2017), 55–61.
54. Viterbi, A., Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans. Inf. Theory 13:2 (1967), 260–269.
55. Wiesler, S., Li, J., Xue, J., Investigations on hessian-free optimization for cross-entropy training of deep neural networks. Proc. INTERSPEECH, 2013, 3317–3321.
56. Wolpert, D., Stacked generalization. Neural Networks 5 (1992), 241–259.
57. Xu, H., Povey, D., Mangu, L., Zhu, J., Minimum Byes risk decoding and system combination based on a recursion for edit distance. Comput. Speech Lang. 25:4 (2011), 802–828.
58. Xue, S., Abdel-Hamid, O., Jiang, H., Dai, L., Liu, Q., Fast adaptation of deep neural network based on discriminant codes for speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 22:12 (2014), 1713–1725.
59. Yan, Z., Huo, Q., Xu, J., A scalable approach to using DNN-derived features in GMM-HMM based acoustic modeling for LVCSR. Proceedings of INTERSPEECH, 2013, 104–108.
60. Yang, J., Ragni, A., Gales, M.J.F., Knill, K.M., Log-linear system combination using structured support vector machines. Proceedings of Interspeech, San Francisco, CA, USA, 2016, 1898–1902.
61. Young, S.J., Odell, J.J., Woodland, P.C., Tree-based state tying for high accuracy acoustic modelling. Proceedings of the Workshop on Human Language Technology, Association for Computational Linguistics, 1994, 307–312.
62. Yu, D., Seltzer, M., Improved bottleneck features using pretrained deep neural networks. Proceedings of INTERSPEECH, 2011, 237–240.
63. Yu, D., Yao, K., Su, H., Li, G., Seide, F., KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition. Proceedings of ICASSP, 2013, 7893–7897.