Advances in very deep convolutional neural networks for LVCSR

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 08-12-September-2016, Issue , 2016, Pages 3429-3433

  Sercu, Tom ^a   Goel, Vaibhava ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Acoustic modeling; Convolutional networks; Neural networks; Speech recognition

Indexed keywords

CONVOLUTION; NEURAL NETWORKS; SPEECH COMMUNICATION; SPEECH PROCESSING; STATISTICAL TESTS;

ACOUSTIC MODEL; CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; SEQUENCE DATA; SPEECH RECOGNITION SYSTEMS; SYSTEM COMBINATION; TIME DIMENSION; WORD ERROR RATE;

SPEECH RECOGNITION;

EID: 84994207081     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: 10.21437/Interspeech.2016-1033     Document Type: Conference Paper

References (35)

1. T. Sercu, C. Puhrsch, B. Kingsbury, and Y. LeCun, "Very deep multilingual convolutional neural networks for lvcsr, " Proc. ICASSP, 2016.
2. "Iarpa babel, " http://www.iarpa.gov/index.php/researchprograms/babel.
3. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " CoRR arXiv:1409.1556, 2014.
4. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift, " Proc. ICML, 2015.
5. T. N. Sainath, A.-R. Mohamed, B. Kingsbury, and B. Ramabhadran, "Deep convolutional neural networks for lvcsr, " in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 8614-8618.
6. H. Soltau, G. Saon, and T. N. Sainath, "Joint training of convolutional and non-convolutional neural networks, " to Proc. ICASSP, 2014.
7. G. Saon, H.-K. J. Kuo, S. Rennie, and M. Picheny, "The ibm 2015 english conversational telephone speech recognition system, " Proc. Interspeech, 2015.
8. G. Saon, T. Sercu, S. Rennie, and H.-K. J. Kuo, "The ibm 2016 english conversational telephone speech recognition system, " -, 2016.
9. Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time series, " The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
10. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
11. R. Collobert, K. Kavukcuoglu, and C. Farabet, "Torch7: A matlab-like environment for machine learning, " in BigLearn, NIPS Workshop, no. EPFL-CONF-192376, 2011.
12. I. Sutskever, J. Martens, G. Dahl, and G. Hinton, "On the importance of initialization and momentum in deep learning, " in Proc. ICML, 2013, pp. 1139-1147.
13. H. Su, G. Li, D. Yu, and F. Seide, "Error back propagation for sequence training of context-dependent deep networks for conversational speech transcription, " in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 6664-6668.
14. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Advances in neural information processing systems, 2012, pp. 1097-1105.
15. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks, " arXiv preprint arXiv:1312.6229, 2013.
16. C. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling, " Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, no. 8, pp. 1915- 1929, 2013.
17. O. Abdel-Hamid, A.-R. Mohamed, H. Jiang, and G. Penn, "Applying convolutional neural networks concepts to hybrid nn-hmm model for speech recognition, " in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012, pp. 4277-4280.
18. X. Zhang, J. Zhao, and Y. LeCun, "Character-level convolutional networks for text classification, " arXiv preprint arXiv:1509.01626, 2015.
19. Y. Kim, Y. Jernite, D. Sontag, and A. M. Rush, "Character-aware neural language models, " arXiv preprint arXiv:1508.06615, 2015.
20. K. Xu, J. Ba, R. Kiros, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio, "Show, attend and tell: Neural image caption generation with visual attention, " arXiv preprint arXiv:1502.03044, 2015.
21. R. Girshick, "Fast r-cnn, " in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1440-1448.
22. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation, " arXiv preprint arXiv:1411.4038, 2014.
23. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang, "Phoneme recognition using time-delay neural networks, " Acoustics, Speech and Signal Processing, IEEE Transactions on, vol. 37, no. 3, pp. 328-339, 1989.
24. T. N. Sainath, B. Kingsbury, G. Saon, H. Soltau, A.-R. Mohamed, G. Dahl, and B. Ramabhadran, "Deep convolutional neural networks for large-scale speech tasks, " Neural Networks, 2014.
25. T. Sainath and C. Parada, "Convolutional neural networks for small-footprint keyword spotting, " in Proc. Interspeech, 2015.
26. D. Palaz, R. Collobert, and M. M. Doss, "Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks, " arXiv preprint arXiv:1304.1018, 2013.
27. D. Amodei, R. Anubhai, E. Battenberg, C. Case, J. Casper, B. Catanzaro, J. Chen, M. Chrzanowski, A. Coates, G. Diamos et al., "Deep speech 2: End-to-end speech recognition in english and Mandarin, " CoRR arXiv:1512.02595, 2015.
28. B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling, " in Proc. ICASSP. IEEE, 2009, pp. 3761-3764.
29. B. Kingsbury, T. N. Sainath, and H. Soltau, "Scalable minimum bayes risk training of deep neural network acoustic models using distributed hessian-free optimization, " in Thirteenth Annual Conference of the International Speech Communication Association, 2012.
30. S. Wiesler, A. Richard, R. Schluter, and H. Ney, "Meannormalized stochastic gradient for large-scale deep learning, " in proc. ICASSP. IEEE, 2014, pp. 180-184.
31. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision, " CoRR arXiv:1512.00567, 2015.
32. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition, " CoRR arXiv:1512.03385, 2015.
33. C. Laurent, G. Pereyra, P. Brakel, Y. Zhang, and Y. Bengio, "Batch normalized recurrent neural networks, " Proc. ICASSP, 2016.
34. H. Li, R. Zhao, and X.Wang, "Highly efficient forward and backward propagation of convolutional neural networks for pixelwise classification, " arXiv preprint arXiv:1412.4526, 2014.
35. F. Yu and V. Koltun, "Multi-scale context aggregation by dilated convolutions, " arXiv preprint arXiv:1511.07122, 2015.