Structure discovery of deep neural network based on evolutionary algorithms

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2015-August, Issue , 2015, Pages 4979-4983

  Shinozaki, Takahiro ^a   Watanabe, Shinji ^b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

Author keywords

CMA ES; Deep neural network; evolutionary algorithm; genetic algorithm; structure optimization

Indexed keywords

AUDIO SIGNAL PROCESSING; COVARIANCE MATRIX; DIRECTED GRAPHS; EVOLUTIONARY ALGORITHMS; GENETIC ALGORITHMS; PARAMETER ESTIMATION; SPEECH COMMUNICATION; SPEECH PROCESSING; STRUCTURAL OPTIMIZATION; TUNING;

COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGIES; DIRECTED ACYCLIC GRAPH (DAG); NETWORK STRUCTURES; OPTIMAL PERFORMANCE; OPTIMIZATION STRATEGY; PHONEME RECOGNITION; PROCESSING APPLICATIONS; STRUCTURE OPTIMIZATION;

DEEP NEURAL NETWORKS;

EID: 84946084313     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2015.7178918     Document Type: Conference Paper

References (23)

1. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, 2012.
2. B. Kingsbury, J. Cui, X. Cui, M.J.F. Gales, K. Knill, J. Mamou, L. Mangu, D. Nolden, M. Picheny, B. Ramabhadran, et al., "A high-performance Cantonese keyword search system," in Proc. ICASSP. IEEE, 2013, pp. 8277-8281.
3. C. W. Chau, S. Kwong, C. K. Diu, and W. R. Fahrner, "Opti-mization of HMM by a genetic algorithm," in Proc. ICASSP. IEEE, 1997, vol. 3, pp. 1727-1730.
4. S. Kwong, C. W. Chau, K. F. Man, and K. S. Tang, "Optimi-sation of HMM topology and its model parameters by genetic algorithms," Pattern Recognition, vol. 34, no. 2, pp. 509-522, 2001.
5. F. Yang, C. Zhang, and T. Sun, "Comparison of particle swarm optimization and genetic algorithm for HMM training," in Proc. ICPR, 2008, pp. 1-4.
6. G. E. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for LVCSR using rectified linear units and dropout," in Proc. ICASSP. IEEE, 2013, pp. 8609-8613.
7. S. Watanabe and J. Le Roux, "Black box optimization for au-tomatic speech recognition," in Proc. ICASSP. IEEE, 2014, pp. 3256-3260.
8. A. R. Mohamed, G. E. Dahl, and G. Hinton, "Acoustic model-ing using deep belief networks," IEEE Transactions on Audio, Speech, and Language Processing, vol. 20, no. 1, pp. 14-22, 2012.
9. L. Deng, D. Yu, and J. Platt, "Scalable stacking and learning for building deep architectures," in Proc. ICASSP. IEEE, 2012, pp. 2133-2136.
10. H. Hermansky, DPW Ellis, and S. Sharma, "Tandem connec-tionist feature extraction for conventional HMM systems," in Proc. ICASSP. IEEE, 2000, vol. 3, pp. 1635-1638.
11. F. Grezl, M. Karafiát, S. Kontár, and J. Cernocky, "Probabilis-tic and bottle-neck features for LVCSR of meetings.," in Proc. ICASSP. IEEE, 2007, pp. 757-760.
12. K. Shinoda and T.Watanabe, "Acoustic modeling based on the MDL criterion for speech recognition," in Proc. Eurospeech, 1997, vol. 1, pp. 99-102.
13. S. S. Chen and P. S. Gopalakrishnan, "Clustering via the Bayesian information criterion with applications in speech recognition," in Proc. ICASSP. IEEE, 1998, vol. 2, pp. 645-648.
14. S. Watanabe, Y. Minami, A. Nakamura, and N. Ueda, "Vari-ational Bayesian estimation and clustering for speech recog-nition," IEEE Transactions on Speech and Audio Processing, vol. 12, pp. 365-381, 2004.
15. X. Liu and M. Gales, "Automatic model complexity control using marginalized discriminative growth functions," IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 4, pp. 1414-1424, 2007.
16. T. Shinozaki, S. Furui, and T. Kawahara, "Gaussian mixture optimization based on efficient cross-validation," IEEE Jour-nal of Selected Topics in Signal Processing, vol. 4, no. 3, pp. 540-547, 2010.
17. L. Davis et al., Handbook of genetic algorithms, vol. 115, Van Nostrand Reinhold New York, 1991.
18. N. Hansen, S. D. Müller, and P. Koumoutsakos, "Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)," Evolutionary Com-putation, vol. 11, no. 1, pp. 1-18, 2003.
19. N. Hansen, A. Auger, R. Ros, S. Finck, and P. Pošík, "Com-paring results of 31 algorithms from the black-box optimiza-tion benchmarking bbob-2009," in Proc. The 12th annual con-ference companion on Genetic and evolutionary computation (GECCO), 2010, pp. 1689-1696.
20. N. Hansen, "The CMA evolution strategy: a comparing re-view," in Towards a new evolutionary computation, pp. 75-102. Springer, 2006.
21. D. Pearce and H. Hirsch, "The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions," in ISCA ITRW ASR2000, 2000, pp. 29-32.
22. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio, "Theano: a CPU and GPU math expression compiler," in Proc. The Python for Scientific Computing Conference (SciPy), 2010.
23. K. Kishida, "Property of average precision and its generaliza-tion: an examination of evaluation indicator for information retrieval," Tech. Rep., National Institute of Informatics, 2005.