Deep convex net: A scalable architecture for speech pattern classification

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2011, Pages 2285-2288

  Deng, Li ^a   Yu, Dong ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Batch mode training; Convex optimization; Deep belief network; Deep learning; Neural network; Parallel computing; Phone state classification; Scalability

Indexed keywords

CLASSIFICATION ACCURACY; CONTEXT DEPENDENT; DEEP BELIEF NETWORKS; DEEP LEARNING; FINE TUNING; LEARNING PROBLEM; PARALLEL TRAINING; RECOGNITION ERROR; SCALABLE ARCHITECTURES; STATE CLASSIFICATION; TRAINING DATA;

CONVEX OPTIMIZATION; NEURAL NETWORKS; PARALLEL ARCHITECTURES; PARALLEL PROCESSING SYSTEMS; SCALABILITY; SPEECH RECOGNITION;

NETWORK ARCHITECTURE;

EID: 84865768819     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. L. Deng, D. Yu, and A. Acero. "Structured speech modeling," IEEE Trans. Audio, Speech & Language Proc., vol. 14, no. 5, pp. 1492-1504, September 2006.
2. N. Morgan. "Deep and Wide: Multilayers in Automatic Speech Recognition," IEEE Trans. on Audio, Speech, and Language Processing, 2011 (in press).
3. G. Hinton and R. Salakhutdinov. "Reducing the Dimensionality of Data with Neural Networks", Science, Vol. 313. no. 5786, pp. 504 - 507, 2006.
4. A. Mohamed, G. Dahl, G. Hinton, "Deep belief networks for phone recognition", NIPS Workshop on Deep Learning for Speech Recognition and Related Applications, Dec. 2009.
5. G. Dahl, D. Yu, L. Deng, and A. Acero. "Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition", IEEE Trans. on Audio, Speech, and Language Processing, 2011 (in press).
6. D. Yu, L. Deng, and G. Dahl, "Roles of Pre-Training and Fine-Tuning in Context-Dependent DBN-HMMs for Real-World Speech Recognition," NIPS Workshop on Deep Learning and Unsupervised Feature Learning, December 2010.
7. A. Mohamed, D. Yu, and L. Deng, "Investigation of Full-Sequence Training of Deep Belief Networks for Speech Recognition," in Interspeech, September 2010.
8. L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. Hinton. "Binary Coding of Speech Spectrograms Using a Deep Auto-encoder," in Interspeech, Sept. 2010.
9. H. Sheikhzadeh and L. Deng. "Waveform-Based Speech Recognition Using Hidden Filter Models: Parameter Selection and Sensitivity to Power Normalization, IEEE Trans. on Speech and Audio Processing, Vol. 2, pp. 80-91, 1994.
10. N. Jaitly and G. Hinton. "Learning a Better Representation of Speech Sound Waves Using Restricted Boltzmann Machines," in Proc. ICASSP, 2011, Prague.
11. D. Yu, S. Wang, and L. Deng. "Sequential Labeling Using Deep-Structured Conditional Random Fields," IEEE J. Selected Topics in Sig. Proc., Vol. 4(6), pp.965-973, Dec. 2010.
12. Y. LeCun, L. Bottou, Y., Bengio, and P. Haffner "Gradient-Based Learning Applied to Document Recognition", Proc. IEEE, Vol. 86, pp. 2278-2324, 1998.
13. D. Yu and L. Deng, "Accelerated Parallelizable Neural Networks Learning Algorithms for Speech Recognition," Proc. Interspeech 2011, accepted.
14. G. B. Huang, Q-Y. Zhu, and C.K. Siew. "Extreme Learning Machine: Theory and Applications", Neurocomputing, vol. 70, pp. 489-501, 2006.
15. J. Baker, et. al. "Research Developments and Directions in Speech Recognition and Understanding," IEEE Sig. Proc. Mag., vol. 26, pp. 75-80, May 2009.
16. L. Deng, "Computational Models for Speech Production," Chapter in Computational Models of Speech Pattern Processing, pp. 199-213, Springer, 1999.