Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2013, Pages 6655-6659

  Sainath, Tara N ^a   Kingsbury, Brian ^a   Sindhwani, Vikas ^a   Arisoy, Ebru ^a   Ramabhadran, Bhuvana ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Deep Neural Networks; Speech Recognition

Indexed keywords

ACOUSTIC MODEL; DEEP NEURAL NETWORKS; HIGH-DIMENSIONAL; LANGUAGE MODEL; LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION; LOW-RANK MATRICES; RECOGNITION ACCURACY; TRAINING PARAMETERS;

COMPUTATIONAL LINGUISTICS; CONTINUOUS SPEECH RECOGNITION; SIGNAL PROCESSING; SPEECH RECOGNITION;

NEURAL NETWORKS;

EID: 84890454527     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2013.6638949     Document Type: Conference Paper

References (15)

1. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep Neural Networks for Acoustic Modeling in Speech Recognition," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, 2012.
2. G. E. Dahl, M. Ranzato, A. Mohamed, and G. E. Hin-ton, "Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine," in Proc. NIPS, 2010.
3. F. Seide, G. Li, and D. Yu, "Conversational Speech Tran-scription Using Context-Dependent Deep Neural Networks," in Proc. Interspeech, 2011.
4. T. N. Sainath, B. Kingsbury, B. Ramabhadran, P. Fousek, P. Novak, and A. Mohamed, "Making Deep Belief Networks Effective for Large Vocabulary Continuous Speech Recogni-tion," in Proc. ASRU, 2011.
5. Y. LeCun and Y. Bengio, "Convolutional Networks for Images, Speech, and Time-series," in The Handbook of Brain Theory and Neural Networks. MIT Press, 1995.
6. T. N. Sainath, A. Mohamed, B. Kingsbury, and B. Ramabhad-ran, "Exploring Convolutional Neural Networks for Large Vo-cabulary Speech Recognition," in submitted to Proc. ICASSP, 2013.
7. M. Yuan, A. Ekici, Z. Lu, and R. Monteiro, "Dimension Re-duction and Coefficient Estimation in Multivariate Linear Re-gression," J. R. Statist. Soc. B., vol. 69, no. 3, pp. 329-346, 2007.
8. B. Recht and C. Re, "Parallel Stochastic Gradient Algorithms for Large-Scale Matrix Completion," Optimization Online, 2011.
9. B. Kingsbury, "Lattice-Based Optimization of Sequence Clas-sification Criteria for Neural-Network Acoustic Modeling," in Proc. ICASSP, 2009.
10. B. Kingsbury, T. N. Sainath, and H. Soltau, "Scalable Mini-mum Bayes Risk Training of Deep Neural Network Acoustic Models Using Distributed Hessian-free Optimization," in Proc. Interspeech, 2012.
11. G. Strang, Introduction to Linear Algebra, Wellesley-Cambridge Press, 4th edition, 2009.
12. H. Soltau, G. Saon, and B. Kingsbury, "The IBM Attila speech recognition toolkit," in Proc. IEEE Workshop on Spoken Lan-guage Technology, 2010, pp. 97-102.
13. H. Bourlard and N. Morgan, Connectionist Speech Recogni-tion: A Hybrid Approach, Kluwer Academic Publishers, 1993.
14. E. Arisoy, T. N. Sainath, B. Kingsbury, and B. Ramabhadran, "Deep neural network language models," in Proceedings of the NAACL-HLT 2012 Workshop. June 2012, pp. 20-28, Associa-tion for Computational Linguistics.
15. Stanley F. Chen, "Performance prediction for exponential lan-guage models," Tech. Rep. RC 24671, IBM Research Division, 2008