Small-footprint high-performance deep neural network-based speech recognition using split-VQ

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2015-August, Issue , 2015, Pages 4984-4988

  Wang, Yongqiang ^a   Li, Jinyu ^a   Gong, Yifan ^a  

^a MICROSOFT   (United States)

Author keywords

DNN; model compression; on device speech recognition; split VQ

Indexed keywords

AUDIO SIGNAL PROCESSING; BACKPROPAGATION; DEEP NEURAL NETWORKS; MATRIX ALGEBRA; SPEECH; SPEECH COMMUNICATION; VECTOR QUANTIZATION;

COMMON PRACTICES; MODEL COMPRESSION; PERFORMANCE DEGRADATION; SCALAR QUANTIZATION; SMALL FOOTPRINTS; SPEECH RECOGNITION SYSTEMS; SPLIT VECTOR QUANTIZATION; SPLIT VQ;

SPEECH RECOGNITION;

EID: 84946014836     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2015.7178919     Document Type: Conference Paper

References (23)

1. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, N. Nguyen, T. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups," IEEE Signal Processing Magazine, 2012.
2. N. Jaitly, P. Nguyen, A. Senior, and V. Vanhoucke, "Application of pretrained deep neural networks to large vocabulary speech recognition," in Proceedings of Interspeech, 2012.
3. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks," in Proceedings of Interspeech, 2011, pp. 437-440.
4. G. E. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition," IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 1, pp. 30-42, 2012.
5. L. Deng, J. Li, J.-T. Huang, K. Yao, D. Yu, F. Seide, M. Seltzer, G. Zweig, X. He, J. Williams, et al., "Recent advances in deep learning for speech research at Microsoft," in Proceedings of ICASSP, 2013, pp. 8604-8608.
6. X. Lei, A. Senior, A. Gruenstein, and J. Sorensen, "Accurate and compact large vocabulary speech recognition on mobile devices," in Proceedings of Interspeech, 2013, pp. 662-665.
7. G. Chen, C. Parada, and G. Heiglod, "Small-footprint keyword spotting using deep neural networks," in Proceedings of ICASSP, 2014.
8. R. Takeda, N. Kanda, and N. Nukaga, "Boundary contraction training for acoustic models based on discrete deep neural networks," in Proceedings of Interspeech, 2014.
9. E. Variani, X. Lei, E. McDermott, I. Moreno, and J. Gonzalez-Dominguez, "Deep neural networks for small footprint textdependent speaker verification," in Proceedings of ICASSP, 2014, pp. 4052-4056.
10. J. Li, R. Zhao, J.-T. Huang, and Y. Gong, "Learning small-size DNN with output-distribution-based criteria," in Proceedings of Interspeech, 2014.
11. R. Reed, "Pruning algorithms: a survey," IEEE Transactions on Neural Networks, vol. 4, no. 5, pp. 740-747, 1993.
12. T. He, Y. Fan, Y. Qian, T. Tan, and K. Yu, "Reshaping deep neural network for fast decoding by node-pruning," in Proceedings of ICASSP, 2014, pp. 245-249.
13. D. Yu, F. Seide, G. Li, and L. Deng, "Exploiting sparseness in deep neural networks for large vocabulary speech recognition," in Proceedings of ICASSP, 2012, pp. 4409-4412.
14. C. Liu, Z. Zhang, and D. Wang, "Pruning deep neural networks by optimal brain damage," in Proceedings of Interspeech, 2014.
15. T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran, "Low-rank matrix factorization for deep neural network training with high-dimensional output targets," in Proceedings of ICASSP, 2013, pp. 6655-6659.
16. J. Xue, J. Li, and Y. Gong, "Restructuring of deep neural network acoustic models with singular value decomposition," in Proceedings of Interspeech, 2013, pp. 2365-2369.
17. V. Vanhoucke, A. Senior, and M. Mao, "Improving the speed of neural networks on CPUs," in Proc. Deep Learning and Unsupervised Feature Learning, NIPS Workshop, 2011.
18. A. Gersho and R. M. Gray, Vector quantization and signal compression, Springer, 1992.
19. Y. Ge and Q. Huo, "A study on the use of CDHMM for large vocabulary off-line recognition of handwritten Chinese characters," in Proceedings of International Workshop on Frontiers in Handwriting Recognition, 2002, pp. 334-338.
20. E. Bocchieri and B.-W. Mak, "Subspace distribution clustering hidden Markov model," IEEE Transactions on Speech and Audio Processing, vol. 9, no. 3, pp. 264-275, 2001.
21. B.-W. Mak and E. Bocchieri, "Direct training of subspace distribution clustering hidden Markov model," IEEE Transactions on Speech and Audio Processing, vol. 9, no. 4, pp. 378-387, 2001.
22. T. Long and L. Jin, "Building compact MQDF classifier for large character set recognition by subspace distribution sharing," Pattern Recognition, vol. 41, no. 9, pp. 2916-2925, 2008.
23. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Transactions on Communication, vol. 28, no. 1, pp. 84-95, 1980.