RASR/NN: The RWTH neural network toolkit for speech recognition

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 3281-3285

  Wiesler, Simon ^a   Richard, Alexander ^a   Golik, Pavel ^a   Schluter, Ralf ^a   Ney, Hermann ^a,b  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

Author keywords

acoustic modeling; GPU; neural networks; open source; RASR; speech recognition

Indexed keywords

ALGORITHMS; ELECTRIC NETWORK TOPOLOGY; NEURAL NETWORKS; OPTIMIZATION; SIGNAL PROCESSING;

ACOUSTIC MODEL; ACTIVATION FUNCTIONS; GPU; OPEN SOURCES; OPTIMIZATION ALGORITHMS; RASR; SPEECH RECOGNITION SYSTEMS; STATE-OF-THE-ART PERFORMANCE;

SPEECH RECOGNITION;

EID: 84905222840     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854207     Document Type: Conference Paper

References (27)

1. A. Mohamed, G. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. on Audio, Speech, and Language Processing, no. 99, pp. 14-22, 2010.
2. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012.
3. D. Rybach, C. Gollan, G. Heigold, B. Hoffmeister, J. Lööf, R. Schlüter, and H. Ney, "The RWTH Aachen university open source speech recognition system," in Proc. Interspeech, Brighton, UK, Sep. 2009, pp. 2111-2114.
4. W. Walker, P. Lamere, P. Kwok, B. Raj, R. Singh, E. Gouvea, P. Wolf, and J. Woelfel, "Sphinx-4: A flexible open source framework for speech recognition," Sun Microsystems, Inc., Mountain View, CA, USA, Tech. Rep., 2004.
5. S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK book version 3.4. Cambridge University Engineering Department, 2006.
6. A. Lee, T. Kawahara, and K. Shikano, "Julius-an open source real-time large vocabulary recognition engine," in Proc. Interspeech, Aalborg, Denmark, Sep. 2001, pp. 1691-1694.
7. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlcek, Y. Qian, P. Schwarz, J. Silovský, G. Stemmer, and K. Veselý, "The Kaldi speech recognition toolkit," in Proc. IEEE Automatic Speech Recognition and UnderstandingWorkshop (ASRU), Hawaii, USA, Dec. 2011.
8. D. Johnson. (2004) QuickNet, speech group at ICSI, Berkeley. [Online]. Available: http://www.icsi.berkeley.edu/Speech/qn.html
9. K. Veselý, L. Burget, and F. Grézl, "Parallel training of neural networks for speech recognition," in Proc. Interspeech, Makuhari, Japan, Sep. 2010, pp. 2934-2937.
10. K. Veselý, M. Karafíat, and F. Grézl, "Convolutive bottleneck network features for LVCSR," in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Hawaii, USA, Dec. 2011, pp. 42-47.
11. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," in Proc. of the 27th Int. Conf. on Machine Learning, Haifa, Israel, Jun. 2010, pp. 807-814.
12. S. A. Solla, E. Levin, and M. Fleisher, "Accelerated learning in layered neural networks," Complex Systems, vol. 2, no. 6, pp. 625-639, Dec. 1988.
13. J. Martens, "Deep learning via Hessian-free optimization," in Proc. of the 27th Int. Conf. on Machine Learning, vol. 951, 2010, p. 2010.
14. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Ng, "Large scale distributed deep networks," in Advances in Neural Information Processing Systems 25, 2012, pp. 1232-1240.
15. S. Wiesler, A. Richard, R. Schlüter, and H. Ney, "Meannormalized stochastic gradient for large-scale deep learning," in (submitted to) Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Florence, Italy, May 2014.
16. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in Proc. of the Int. Conf. on Neural Networks, 1993, pp. 586-591.
17. P. Golik, P. Doetsch, and H. Ney, "Cross-entropy vs. squared error training: A theoretical and experimental comparison," in Proc. Interspeech, Lyon, France, Aug. 2013, pp. 1756-1760.
18. A. Senior, G. Heigold, M. Ranzato, and K. Yang, "An empirical study of learning rates in deep neural networks for speech recognition," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, vol. 1, 2013, pp. 6724-6728.
19. H. Ney and S. Ortmanns, "Progress in dynamic programming search for LVCSR," Proc. of the IEEE, vol. 88, no. 8, pp. 1224-1240, Aug. 2000.
20. H. A. Bourlard and N. Morgan, Connectionist speech recognition: A hybrid approach. Norwell, MA, USA: Kluwer Academic Publishers, 1993.
21. H. Hermansky, D. Ellis, and S. Sharma, "Tandem connectionist feature extraction for conventional HMM systems," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, vol. 3, Istanbul, Turkey, Jun. 2000, pp. 1635-1638.
22. H. Hermansky and P. Fousek, "Multi-resolution RASTA filtering for TANDEM-based ASR," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005, pp. 361-364.
23. F. Seide, G. Li, X. Chen, and D. Yu, "Feature engineering in context-dependent deep neural networks for conversational speech transcription," in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Hawaii, USA, Dec. 2011, pp. 24-29.
24. M. Sundermeyer, M. Nusbaum-Thom, S. Wiesler, C. Plahl, A. E. Mousa, S. Hahn, D. Nolden, R. Schlüter, and H. Ney, "The RWTH 2010 QUAERO ASR evaluation system for English, French, and German," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Prague, Czech, May 2011, pp. 2212-2215.
25. A. Graves, N. Jaitly, and A.-r. Mohamed, "Hybrid speech recognition with deep bidirectional LSTM," in Proc. IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Olomouc, Czech Republic, Dec. 2013, pp. 273-278.
26. O. Abdel-Hamid, A. Mohamed, H. Jiang, and G. Penn, "Applying convolutional neural networks concepts to hybrid NNHMM model for speech recognition," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Kyoto, Japan, Mar. 2012, pp. 4277-4280.
27. B. Kingsbury, "Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling," in Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Taipei, Taiwan, Apr. 2009, pp. 3761-3764.