Learning compact recurrent neural networks

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2016-May, Issue , 2016, Pages 5960-5964

  Lu, Zhiyun ^a   Sindhwani, Vikas ^b   Sainath, Tara N ^b  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84973294673     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2016.7472821     Document Type: Conference Paper

References (19)

1. G. Saon, H. Soltau, A. Emami, and M. Picheny, "Unfolded Recurrent Neural Networks for Speech Recognition, " in Interspeech, 2014
2. H. Sak, A. Senior, and F. Beaufays, "Long Short-Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling, " in Proc. Interspeech, 2014
3. T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, "Convolutional, Long Short-Term Memory, Fully Connected Deep Neural Networks, " in Proc. ICASSP, 2015
4. T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran, "Low-Rank Matrix Factorization for Deep Belief Network Training, " in Proc. ICASSP, 2013
5. Jian Xue, Jinyu Li, and Yifan Gong, "Restructuring of Deep Neural Network Acoustic Models with Singular Value Decomposition, " in INTERSPEECH, 2013, pp. 2365-2369
6. M. Collins and P. Kohli, "Memory-bounded Deep Convolutional Neural Networks, " in Proc. ICASSP, 2013
7. Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan, "Deep Learning with Limited Numerical Precision, " in Proc. ICML, 2015
8. M. Courbariaux, J.-P. David, and Y. Bengio, "Low-precision Storage for Deep Learning, " in Proc. ICLR, 2015
9. Wenlin Chen, James T Wilson, Stephen Tyree, Kilian Q Weinberger, and Yixin Chen, "Compressing Neural Networks with the Hashing Trick, " arXiv preprint arXiv: 1504. 04788, 2015
10. V. Sindhwani, T. N. Sainath, and S. Kumar, "Structured Transforms for Small-footprint Deep Learning, " in Proc. NIPS, 2015
11. G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network, " in Deep Learning and Representation Learning Workshop, NIPS, 2014
12. Sepp Hochreiter and Jürgen Schmidhuber, "Long Short-term Memory, " Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997
13. Felix A Gers, Jürgen Schmidhuber, and Fred Cummins, "Learning to Forget: Continual Prediction with LSTM, " Neural computation, vol. 12, no. 10, pp. 2451-2471, 2000
14. F. Gers, J. Schmidhuber, et al., "Recurrent Nets that Time and Count, " in Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on. IEEE, 2000, vol. 3, pp. 189-194
15. G. Strang, Introduction to Linear Algebra, Wellesley-Cambridge Press, 4th edition, 2009
16. V. Pan, Structured Matrices and Polynomials: Unified Superfast Algorithms, Springer, 2001
17. J. Dean, G. S. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Y. Ng, "Large Scale Distributed Deep Networks, " in Proc. NIPS, 2012
18. Klaus Greff, Rupesh Kumar Srivastava, Jan Koutník, Bas R Steunebrink, and Jürgen Schmidhuber, "LSTM: A Search Space Odyssey, " arXiv preprint arXiv: 1503. 04069, 2015
19. Wojciech Zaremba, "An Empirical Exploration of Recurrent Network Architectures, " 2015.