Parallel implementation of Artificial Neural Network training for speech recognition

Pattern Recognition Letters

Volumn 31, Issue 11, 2010, Pages 1302-1309

  Scanzio, Stefano ^a   Cumani, Sandro ^a   Gemello, Roberto ^b   Mana, Franco ^b   Laface, P ^a  

^a POLITECNICO DI TORINO   (Italy)

^b Loquendo   (Italy)

Author keywords

Artificial Neural Network; Block Back propagation; CUDA; Fast Training; Focused Attention Back Propagation; GPU

Indexed keywords

ACOUSTIC MODEL; ARTIFICIAL NEURAL NETWORK; BACKPROPAGATION LEARNING ALGORITHM; BLOCK SIZES; FEATURE VECTORS; FEED-FORWARD; LANGUAGE INTERFACE; LARGE SIZES; LARGE VOCABULARY SPEECH RECOGNITION; MULTI-CORE PROCESSOR; MULTI-THREAD; OPTIMIZED IMPLEMENTATION; PARALLEL IMPLEMENTATIONS; REAL-WORLD; SEQUENTIAL PATTERNS; SIMD ARCHITECTURE; SPEECH RECOGNITION SYSTEMS; SPEED-UPS; TRAINING PROCEDURES; TRAINING TIME;

BACKPROPAGATION ALGORITHMS; DATA FLOW ANALYSIS; LEARNING ALGORITHMS; LINGUISTICS; PATTERN RECOGNITION SYSTEMS; QUERY LANGUAGES; SPEECH RECOGNITION;

NEURAL NETWORKS;

EID: 77953127720     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2010.02.003     Document Type: Article

References (24)

1. Albesano, D., Gemello, R., Mana, F., 1997. Hybrid HMM-NN modeling of stationary-transitional units for continuous speech recognition. In: Proc. Internat. Conf. on Neural Information Processing, pp. 1112-1115.
2. Anguita D., Parodi G., and Zunino R. An efficient implementation of BP on RISC-based workstations. Neurocomputing 6 (1994) 57-65
3. Bilmes, J., Asanovic, K., Chin, C., Demmel, J., 1997. Using PHiPAC to speed error back-propagation learning. In: Proc. Internat. Conf. on Acoustics, Speech and Signal Processing, ICASSP-97, pp. 4153-4156.
4. Blackford L.S., et al. An updated set of basic linear algebra subprograms (BLAS). ACM Trans. Math. Software 28 2 (2002) 135-151
5. Bourlard H., and Morgan N. Connectionist Speech Recognition: A Hybrid Approach (1993), Kluwer Academic Publishers
6. Cardinal, P., Dumouchel, P., Boulianne, G., 2009. Using parallel architectures in speech recognition. In: Proc. INTERSPEECH 2009, pp. 3039-3042.
7. Cernansky, M., 2009. Training recurrent neural network using multistream extended Kalman filter on multicore processor and cuda enabled graphic processor unit. In: Alippi, C., et al. (Eds.), Lecture Notes in Computer Science. Artificial Neural Networks - LNCS 5768, Part 1, ICANN 2008, Limassol, Cyprus, pp. 381-390.
8. Chan, A., Sherwani, J., Mosur, R., Rudnicky, A., 2004. Four-layer categorization scheme of fast GMM computation techniques in large vocabulary continuous speech recognition systems. In: Proc. INTERSPEECH 2004, pp. 689-692.
9. Dixon P.R., Oonishi T., and Furui S. Harnessing graphics processors for the fast computation of acoustic likelihoods in speech recognition. Comput. Speech Lang. 23 (2009) 510-526
10. Fissore, L., Laface, P., Ravera, F., 1995. Acoustic-phonetic modeling for flexible vocabulary speech recognition. In: Proc. EUROSPEECH 95, pp. 799-802.
11. Hertz J., Krogh A., and Palmer R.G. Introduction to the Theory of Neural Computation (1991), Addison Wesley
12. Hoskins, J.C., 1989. Speeding up artificial neural networks in the "Real" World. In: Proc. Internat. Joint Conf. on Neural Networks, p. 626.
13. Intel MKL: Math Kernel Library, 2009. .
14. Intel IPP: Integrated Performance Primitives, 2009. .
15. Jang, H., Park, A. Jung, K., 2008. Neural network implementation using CUDA and OpenMP. In: Proc. Digital Image Computing: Techniques and Applications, DICTA '08, pp. 155-161.
16. Lahabar, S., Agarwal, P., Narayanan, P.J., 2008. High performance pattern recognition on GPU. In: Proc. National Conf. on Computer Vision Pattern Recognition, Image Processing and Graphics, NCVPRIPG'08, pp. 154-159.
17. Loquendo ASR. .
18. NVIDIA: NVIDIA CUDA Compute Unified Device Architecture Programming Guide, 2008. .
19. Oh K.S., and Jung K. GPU implementation of neural networks. Pattern Recognition 37 (2004) 1311-1314
20. Paul, D.B., Baker, J.M., 1992. The design of the wall street journal-based CSR corpus. In: Proc. Internat. Conf. on Speech and Language Processing, ICSLP-1992, pp. 899-902.
21. Rabiner L., and Juang B.H. Fundamentals of Speech Recognition (1993), Prentice-Hall
22. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning Internal Representations by Error Propagations. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Foundations vol. 1 (1986), MIT Press pp. 318-362
23. Sarkar D. Methods to speedup error back propagation learning algorithm. ACM Comput. Surveys 27 4 (1995) 519-542
24. Volkov, V., Demmel, J.W., 2008. Benchmarking GPUs to tune dense linear algebra. In: Proc. 2008 ACM/IEEE Conf. on Supercomputing, pp. 1-11.