On rectified linear units for speech processing

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2013, Pages 3517-3521

  Zeiler, M D ^a   Ranzato, M ^b   Monga, R ^b   Mao, M ^b   Yang, K ^b   Le, Q V ^b   Nguyen, P ^b   Senior, A ^b   Vanhoucke, V ^b   Dean, J ^b   Hinton, G E ^c  

^a NEW YORK UNIVERSITY   (United States)

^b GOOGLE INC   (United States)

^c UNIVERSITY OF TORONTO   (Canada)

Author keywords

Deep Learning; Hybrid System; Neural Networks; Rectified Linear units; Unsupervised Learning

Indexed keywords

COMPUTATIONAL UNITS; DEEP LEARNING; DEEP NEURAL NETWORKS; DISTRIBUTED ENVIRONMENTS; LARGE VOCABULARY SPEECH RECOGNITION; LINEAR PROJECTIONS; LINEAR UNITS; SPEECH RECOGNITION SYSTEMS;

HYBRID SYSTEMS; NEURAL NETWORKS; SIGNAL PROCESSING; SPEECH RECOGNITION; UNSUPERVISED LEARNING;

SPEECH PROCESSING;

EID: 84890471125     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2013.6638312     Document Type: Conference Paper

References (16)

1. A.R. Mohamed, G.E. Dahl, and G.E. Hinton, "Deep belief networks for phone recognition," NIPS 22 workshop on deep learning for speech recognition, 2009.
2. G.E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Computation, vol. 14, pp. 1771-1800, 2002.
3. C. Plahl, T.N. Sainath, B. Ramabhadran, and D. Nahamoo, "Improved pre-training of deep belief networks using sparse encoding symmetric machines," in ICASSP, 2012.
4. T.N. Sainath, B. Kingsbury, and B. Ramabhadran, "Auto-encoder bottleneck features using deep belief networks," in ICASSP, 2012.
5. B. Kingsbury, T.N. Sainath, and H. Soltau, "Scalable minimum bayes risk training of deep neural network acoustic models using distributed hessian-free optimization," in Interspeech, 2012.
6. L. Deng, B. Hutchinson, and D. Yu, "Parallel training of deep stacking networks," in Interspeech, 2012.
7. V. Nair and G.E. Hinton, "Rectified linear units improve restricted boltzmann machines," in ICML, 2010.
8. X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks," Journal of Machine Learning Research-Proceedings Track, vol. 15, pp. 315-323, 2011.
9. A. Krizhevsky, I. Sutskever, and G.E. Hinton, "Imagenet classification with deep convolutional neural networks," in NIPS, 2012.
10. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Ng, "Large scale distributed deep networks," in NIPS, 2012.
11. M. Ranzato, "Unsupervised learning of feature hierarchies," Ph.D. thesis, ch. 1, 2009.
12. K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "Fast inference in sparse coding algorithms with applications to object recognition," Tech. Rep., Computational and Biological Learning Lab, Courant Institute, NYU, 2008, Tech Report CBLL-TR-2008-12-01.
13. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," in ICML, 2010.
14. C.J. Rozell, D.H. Johnson, Baraniuk R.G., and B.A. Olshausen, "Sparse coding via thresholding and local competition in neural circuits," Neural Computation, 2008.
15. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online leaning and stochastic optimization," in COLT, 2010.
16. N. Jaitly, P. Nguyen, A. Senior, and V. Vanhoucke, "Application of pretrained deep neural networks to large vocabulary speech recognition," in Interspeech, 2012.