Tikhonov regularization for deep neural network acoustic modeling

2014 IEEE Workshop on Spoken Language Technology, SLT 2014 - Proceedings

Volumn , Issue , 2014, Pages 147-152

  Chien, Jen Tzung ^a   Lu, Tsai Wei ^a  

^a NATIONAL CHIAO TUNG UNIVERSITY   (Taiwan)

Author keywords

Acoustic model; Deep neural network; Speech recognition; Tikhonov regularization

Indexed keywords

ACOUSTIC MODEL; DEEP NEURAL NETWORKS; OBJECTIVE FUNCTIONS; RESTRICTED BOLTZMANN MACHINE; ROBUSTNESS IN SPEECH RECOGNITION; STRUCTURAL MODELING; TIKHONOV REGULARIZATION; UNSUPERVISED FEATURE LEARNING;

SPEECH RECOGNITION;

EID: 84946033546     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2014.7078565     Document Type: Conference Paper

References (15)

1. L. Deng, G. Hinton, and B. Kingsbury, "New types of deep neural network learning for speech recognition and related applications: An overview, " in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013, pp. 8599-8603.
2. A. Krogh and J. A. Hertz, "A simple weight decay can improve generalization, " in Advances in Neural Information Processing Systems, John E. Moody, Steve J. Hanson, and Richard P. Lippmann, Eds, 1992, vol. 4, pp. 950-957.
3. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors, " The Computing Research Repository (CoRR), 2012, arXiv:1207. 0580.
4. G. E. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for LVCSR using rectified linear units and dropout, " in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013, pp. 8609-8613.
5. L. Wan, M. D. Zeiler, S. Zhang, Y. LeCun, and R. Fergus, "Regularization of neural networks using dropconnect, " in Proceeding of International Conference on Machine Learning (ICML), 2013, vol. 28, pp. 1058-1066.
6. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems, Washington, DC : Winston, 1977.
7. K. Cho, A. Ilin, and T. Raiko, "Tikhonov-type regularization for restricted Boltzmann machines, " in Proceeding of International Conference on Artificial Neural Networks (ICANN), 2012, pp. 81-88.
8. G. E. Hinton, "A practical guide to training restricted Boltzmann machines, " in Neural Networks: Tricks of the Trade (2nd ed. ), pp. 599-619. 2012.
9. C. M. Bishop, "Training with noise is equivalent to Tikhonov regularization, " Neural Computation, vol. 7, no. 1, pp. 108-116, 1995.
10. S. Rifai, P. Vincent, X. Muller, X. Glorot, and Y. Bengio, "Contractive auto-encoders: explicit invariance during feature extraction, " in Proceeding of International Conference on Machine Learning (ICML), 2011, pp. 833-840.
11. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, "The Kaldi speech recognition toolkit, " in Proceeding of IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), 2011.
12. N. Parihar and J. Picone, "Aurora working group: DSR front end LVCSR evaluation AU/384/02, " Tech. Rep, Institute for Signal and Information Processing, Mississippi State University, 2002.
13. R. A. Gopinath, "Maximum likelihood modeling with Gaussian distributions for classification, " in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 1998, pp. 661-664.
14. A. Mohamed, G. E. Hinton, and G. Penn, "Understanding how deep belief networks perform acoustic modelling, " in Proceedings of International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2012, pp. 4273-4276.
15. K. Veselý, A. Ghoshal, L. Burget, and D. Povey, "Sequence-discriminative training of deep neural networks, " in Proceedings of Annual Conference of International Speech Communication Association (INTERSPEECH), 2013, pp. 2345-2349.