Deep maxout neural networks for speech recognition

2013 IEEE Workshop on Automatic Speech Recognition and Understanding, ASRU 2013 - Proceedings

Volumn , Issue , 2013, Pages 291-296

  Cai, Meng ^a   Shi, Yongzhe ^a   Liu, Jia ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

acoustic modeling; Maxout networks; neuron nonlinearity; speech recognition

Indexed keywords

ACOUSTIC MODEL; ACTIVATION FUNCTIONS; BENCHMARK TESTS;

BENCHMARKING; LINEAR NETWORKS; SPEECH RECOGNITION; TELEPHONE CIRCUITS;

NEURAL NETWORKS;

EID: 84893651518     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ASRU.2013.6707745     Document Type: Conference Paper

References (18)

1. G.E. Hinton, L. Deng, D. Yu, G.E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " IEEE Signal Processing Magazine, vol. 29, pp. 82-97, November 2012.
2. G.E. Dahl, D. Yu, L. Deng, and A. Acero, "Contextdependent pre-trained deep neural networks for large vocabulary speech recognition, " IEEE Transactions on Audio, Speech and Language Processing, vol. 20, pp. 30-42, January 2012.
3. A. Mohamed, G. Hinton, and G. Penn, "Understanding how deep belief networks perform acoustic modeling, " in Proc. ICASSP. IEEE SPS, 2012, pp. 4273-4276.
4. J. Pan, C. Liu, Z. Wang, Y. Hu, and H. Jiang, "Investigation of deep neural networks (DNN) for large vocabulary continuous speech recognition: Why DNN surpasses GMMs in acoustic modeling, " in Proc. ISCSLP. IEEE, 2012, pp. 301-305.
5. A. Mohamed, G.E. Dahl, and G.E. Hinton, "Acoustic modeling using deep belief networks, " IEEE Transactions on Audio, Speech and Language Processing, vol. 20, pp. 14-22, January 2012.
6. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks, " in Proc. Interspeech. ISCA, 2011, pp. 437-440.
7. X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks, " in Proc. AISTATS, 2011, pp. 315- 323.
8. M.D. Zeiler, M. Ranzato, R. Monga, M. Mao, K. Yang, Q.V. Le, P. Nguyen, A. Senior, V. Vanhoucke, J. Dean, and G.E. Hinton, "On rectified linear units for speech processing, " in Proc. ICASSP. IEEE, 2013, pp. 3517- 3521.
9. A. Maas, A. Hannun, and A. Ng, "Rectifier nonlinearities improve neural network acoustic models, " in Proc. ICML. International Machine Learning Society, 2013.
10. G.E. Hinton, S. Osindero, and Y. Teh, "A fast learning algorithm for deep belief nets, " Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006. (Pubitemid 44024729)
11. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription, " in Proc. ASRU. IEEE, 2011, pp. 24-29.
12. V. Nair and G. Hinton, "Rectified linear units improve restricted Boltzmann machines, " in Proc. ICML. International Machine Learning Society, 2010.
13. I. Goodfellow, D.Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "Maxout networks, " in Proc. ICML. International Machine Learning Society, 2013.
14. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors, " arXiv: 1207.0580, 2012.
15. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, November 1998.
16. M. Cai, W.Q. Zhang, and J. Liu, "Improving deep neural network acoustic models using unlabeled data, " in Proc. ChinaSIP. IEEE, 2013.
17. V. Mnih, "CUDAMat: A CUDA-based matrix class for python, " Tech. Rep. UTML TR 2009-004, Department of Computer Science, University of Toronto, 2009.
18. R. Prabhavalkar, T. Sainath, D. Nahamoo, B. Ramabhadran, and D. Kanevsky, "An evaluation of posterior modeling techniques for phonetic recogntion, " in Proc. ICASSP. IEEE, 2013, pp. 7165-7169.