Porting concepts from DNNs back to GMMs

2013 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2013 - Proceedings

Volumn , Issue , 2013, Pages 356-361

  Demuynck, Kris ^a   Triefenbach, Fabian ^a  

^a GHENT UNIVERSITY   (Belgium)

Author keywords

deep neural networks; deep structures; DNN; Gaussian mixture models; GMM; speech recognition

Indexed keywords

DEEP NEURAL NETWORKS; DEEP STRUCTURE; DNN; GAUSSIAN MIXTURE MODEL; GMM;

NEURAL NETWORKS;

SPEECH RECOGNITION;

EID: 84893695632     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2013.6707756     Document Type: Conference Paper

References (41)

1. G. Hinton, L. Deng, D. Yu, G.E. Dahl, A-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " Signal Processing Magazine, vol. 29, no. 6, pp. 82- 97, 2012.
2. A.-R. Mohamed, G.E. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks, " IEEE Trans. on ASLP, vol. 20, no. 1, pp. 14-22, 2012.
3. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors, " CoRR, p. abs/1207.0580, 2012.
4. F. Triefenbach, A. Jalalvand, K. Demuynck, and J-P. Martens, "Acoustic modeling with hierarchical reservoirs, " IEEE Trans. on ASLP, 2013, (accepted).
5. M.J.F. Gales, "Maximum likelihood linear transformations for HMM-based speech recognition, " Comp. Speech and Lang., vol. 12, pp. 75-98, 1998. (Pubitemid 128383747)
6. R. Kuhn, J.-C. Junqua, P. Nguyen, and N. Niedzielski, "Rapid speaker adaptation in eigenvoice space, " IEEE Trans. on SAP, vol. 8, no. 6, pp. 695-707, 2000. (Pubitemid 32025317)
7. A. Acero, L. Deng, T. Kristjansson, and J. Zhang, "HMM adaptation using vector Taylor series for noisy speech recognition, " in Proc. ICSLP, 2000, pp. 229-232.
8. J. Droppo, L. Deng, and A. Acero, "Evaluation of the SPLICE algorithm on the Aurora2 database, " in Proc. EUROSPEECH, 2001, pp. 217-220.
9. H. Liao and M.J.F Gales, "Adaptive training with joint uncertainty decoding for robust recognition of noisy data, " in Proc. ICASSP, 2007, pp. 389-392.
10. K. Demuynck, J. Duchateau, and D. Van Compernolle, "Reduced semi-continuous models for large vocabulary continuous speech recognition in Dutch, " in Proc. ICSLP, 1996, vol. IV, pp. 2289-2292.
11. G.E. Hinton, "Training products of experts by minimizing contrastive divergence, " Neural Computation, vol. 14, pp. 1771- 1800, 2002.
12. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis, pp. 114-121, John Wiley & Sons, 1973.
13. G. Schukat-Talamazzini, J. Hornegger, and H. Niemann, "Optimal linear feature transformations for semi-continuous hidden Markov models, " in Proc. ICASSP, 1995, vol. I, pp. 369- 372.
14. N. Kumar, Investigation of Silicon-Auditory Models and Generalization of LDA for Improved Speech Recognition, Ph.D. thesis, John Hopkins Univ., 1997.
15. K. Demuynck, J. Duchateau, and D. Van Compernolle, "Optimal feature sub-space selection based on discriminant analysis, " in Proc. EUROSPEECH, 1999, vol. III, pp. 1311-1314.
16. J. Duchateau, K. Demuynck, D. Van Compernolle, and P. Wambacq, "Class definition in discriminant feature analysis, " in Proc. EUROSPEECH, 2001, vol. III, pp. 1621-1624.
17. K. Demuynck, J. Duchateau, D. Van Compernolle, and P.Wambacq, "Improved feature decorrelation for HMM-based speech recognition, " in Proc. ICSLP, 1998, vol. VII, pp. 2907- 2910.
18. M.J.F. Gales, "Semi-tied covariance matrices, " in Proc. ICASSP, 1998, vol. II, pp. 657-660.
19. X. Cui, J. Xue, B. Xiang, and B. Zhou, "A study of bootstrapping with multiple acoustic features for improved automatic speech recognition, " in Proc. INTERSPEECH, 2009, pp. 240- 243.
20. D. Yu, M.L. Seltzer, J. Li, J.-T. Huang, and F. Seide, "Feature learning in deep neural networks - studies on speech recognition tasks, " CoRR, vol. abs/1301.3605, 2013, http://arxiv.org/abs/1301.3605.
21. Y-h. Sung and D. Jurafsky, "Hidden conditional random fields for phone recognition, " in Proc. ASRU, 2009, pp. 107-112.
22. G. Heigold, G. Zweig, X. Li, and P. Nguyen, "A flat direct model for speech recognition, " in Proc. ICASSP, 2009, pp. 3861-3864.
23. G. Zweig and S. Russell, "Speech recognition with dynamic Bayesian networks, " Tech. Rep., Microsoft, 1998.
24. D. Yu, S.Wang, and L. Deng, "Sequential labeling using deepstructured conditional random fields, " IEEE Journal of Selected Topics in Signal Processing, vol. 4, no. 6, pp. 965-973, 2010.
25. J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, and N. Dahlgren, "The DARPA TIMIT acoustic-phonetic continuous speech corpus CD-rom, " Tech. report, NIST, 1993.
26. K. Lee and H.-W. Hon, "Speaker-independent phone recognition using hidden Markov models, " IEEE Trans. on ASSP, vol. 37, pp. 1641-1648, 1989. (Pubitemid 20652953)
27. A.R. Mohamed, T.N. Sainath, G. Dahl, B. Ramabhadran, G.E. Hinton, and M.A. Picheny, "Deep belief networks using discriminative features for phone recognition, " in Proc. ICASSP, 2011, pp. 5060-5063.
28. K. Demuynck, J. Roelens, D. Van Compernolle, and P. Wambacq, "SPRAAK: An open source speech recognition and automatic annotation kit, " in Proc. INTERSPEECH, Sept. 2008, p. 495.
29. J. Duchateau, M. Wigham, K. Demuynck, and H. Van hamme, "A flexible recogniser architecture in a reading tutor for children, " in Proc. ITRWon Speech Recognition and Intrinsic Variation, 2006, pp. 59-64.
30. Y. Freund and R.E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting, " Journal of computer and system sciences, vol. 55, no. 1, pp. 119- 139, 1997. (Pubitemid 127433398)
31. J. Lu, K.N. Plataniotis, and A.N. Venetsanopoulos, "Boosting linear discriminant analysis for face recognition, " in Proc. ICIP, 2003, vol. 1, pp. 657-660.
32. T. Anastasakos, J. McDonough, and J. Makhoul, "Speaker adaptive training: A maximum likelihood approach to speaker normalization, " in Proc. ICASSP, 1997, vol. 2, pp. 1043-1046.
33. X. Zhang, K. Demuynck, and H. Van hamme, "latent variable speaker adaptation of gaussian mixture weights and means, " in Proc. ICASSP, 2012, pp. 4349-4352.
34. B. Merialdo, "Phonetic recognition using hidden Markov models and maximum mutual information training, " in Proc. ICASSP, 1988, vol. 1, pp. 111-114. (Pubitemid 18665963)
35. D. Povey and P.C. Woodland, "Improved discriminative training techniques for large vocabulary continuous speech recognition, " in Proc. ICASSP, 2001, vol. 1, pp. 45-48. (Pubitemid 32839184)
36. D. Povey and P.C. Woodland, "Minimum phone error and I-smoothing for improved discriminative training, " in Proc. ICASSP, 2002, vol. 1, pp. 105-108.
37. B. Kingsbury, "lattice-based optimization of sequence classification criteria for neural-network acoustic modeling, " in Proc. ICASSP, 2009, pp. 3761-3764.
38. D. Kanevsky, D. Nahamoo, T.N. Sainath, B. Ramabhadran, and P.A. Olsen, "A-functions: A generalization of extended Baum-Welch transformations to convex optimization, " in Proc. ICASSP, 2011, pp. 5164-5167.
39. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks, " in Proc. INTERSPEECH, 2011, pp. 437-440.
40. L.D. Erman, F. Hayes-Roth, V.R. Lesser, and D.R. Reddy, "The HEARSAY-II speech understanding system: Integrating knowledge to resolve uncertainty, " Computing Surveys, vol. 12, pp. 213-253, 1980.
41. R. Kumaran, J. Bilmes, and K. Kirchhoff, "Attention shift decoding for conversational speech recognition, " in Proc. INTERSPEECH, 2007, pp. 1493-1496.