Black box optimization for automatic speech recognition

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 3256-3260

  Watanabe, Shinji ^a   Le Roux, Jonathan ^a  

^a MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

Author keywords

Bayesian optimization; Black box optimization; CMA ES; Gaussian process; Speech recognition

Indexed keywords

COMPLEX NETWORKS; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); OPTIMIZATION; SIGNAL PROCESSING;

AUTOMATIC SPEECH RECOGNITION; AUTOMATIC SPEECH RECOGNITION SYSTEM; BAYESIAN OPTIMIZATION; BLACK-BOX OPTIMIZATION; CMA-ES; EVOLUTION STRATEGIES; GAUSSIAN PROCESSES; SPEECH RECOGNITION PERFORMANCE;

SPEECH RECOGNITION;

EID: 84905216973     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854202     Document Type: Conference Paper

References (25)

1. X. D. Huang, A. Acero, and H.W. Hon, Spoken language processing, a guide to theory, algorithm, and system development, Prentice Hall PTR, 2001.
2. G. Saon and J-T Chien, "Large-vocabulary continuous speech recognition systems: A look at some recent advances," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 18-33, 2012.
3. Y. Tachioka, S. Watanabe, J. Le Roux, and J. R. Hershey, "Discriminative methods for noise robust speech recognition: A CHiME challenge benchmark," in Proc. International Workshop on Machine Listening in Multisource Environments (CHiME), 2013.
4. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, 2012.
5. K. Shinoda and T.Watanabe, "Acoustic modeling based on the MDL criterion for speech recognition," in Proc. Eurospeech, 1997, vol. 1, pp. 99-102.
6. S.S. Chen and P. S. Gopalakrishnan, "Clustering via the Bayesian information criterion with applications in speech recognition," in Proc. ICASSP, 1998, vol. 2, pp. 645-648.
7. S. Watanabe, Y. Minami, A. Nakamura, and N. Ueda, "Variational Bayesian estimation and clustering for speech recognition," IEEE Transactions on Speech and Audio Processing, vol. 12, pp. 365-381, 2004.
8. Y. Zhang, P. Liu, J.T. Chien, and F. Soong, "An evidence framework for Bayesian learning of continuous-density hidden Markov models," in Proc. ICASSP, 2009, pp. 3857-3860.
9. N. Hansen, S. D. Müller, and P. Koumoutsakos, "Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES)," Evolutionary Computation, vol. 11, no. 1, pp. 1-18, 2003.
10. E. Brochu, V. M. Cora, and N. De Freitas, "A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning," arXiv preprint arXiv:1012.2599, 2010.
11. J. Snoek, H. Larochelle, and R. P. Adams, "Practical bayesian optimization of machine learning algorithms," arXiv preprint arXiv:1206.2944, 2012.
12. C. E. Rasmussen and C. K. I. Williams, "Gaussian processes for machine learning," 2006.
13. J. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, "Algorithms for hyper-parameter optimization," in Proc. NIPS, 2011.
14. D.Wierstra, T. Schaul, T. Glasmachers, Y. Sun, and J. Schmidhuber., "Natural evolution strategies," arXiv preprint arXiv:1106.4487, 2011.
15. C. W. Chau, S. Kwong, C. K. Diu, and W. R. Fahrner, "Optimization of HMM by a genetic algorithm," in Proc. ICASSP, 1997, vol. 3, pp. 1727-1730.
16. S. Kwong, C. W. Chau, K. F. Man, and K. S. Tang, "Optimisation of HMM topology and its model parameters by genetic algorithms," Pattern Recognition, vol. 34, no. 2, pp. 509-522, 2001.
17. F. Yang, C. Zhang, and T. Sun, "Comparison of particle swarm optimization and genetic algorithm for HMM training," in Proc. ICPR, 2008, pp. 1-4.
18. G. E. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for LVCSR using rectified linear units and dropout," in Proc. ICASSP, 2013, pp. 8609-8613.
19. Y. Akimoto, Y. Nagata, I. Ono, and S. Kobayashi, "Bidirectional relation between CMA evolution strategies and natural evolution strategies," in Proc. Parallel Problem Solving from Nature (PPSN), 2010, pp. 154-163.
20. S. Amari, "Natural gradient works efficiently in learning," Neural computation, vol. 10, no. 2, pp. 251-276, 1998.
21. D. Barber, Bayesian reasoning and machine learning, Cambridge University Press, 2012.
22. N. Hansen, A. Auger, R. Ros, S. Finck, and P. Pósík, "Comparing results of 31 algorithms from the black-box optimization benchmarking bbob-2009," in Proc. The 12th annual conference companion on Genetic and evolutionary computation (GECCO), 2010, pp. 1689-1696.
23. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlcek, Y. Qian, P. Schwarz, J. Silovsk, G. Stemmer, and K. Vesel, "The Kaldi speech recognition toolkit," in Proc. ASRU, 2011.
24. K. Kinoshita, M. Delcroix, T. Yoshioka, T. Nakatani, E. Habets, R. Haeb-Umbach, V. Leutnant, A. Sehr, W. Kellermann, R. Maas, S. Gannot, and B. Raj, "The REVERB challenge: A common evaluation framework for dereverberation and recognition of reverberant speech," in Proc. of WASPAA, 2013.
25. F.Weninger, S.Watanabe, Y. Tachioka, and B. Schuller, "Deep recurrent de-noising auto-encoder and blind de-reverberation for reverberated speech recognition," in Proc. ICASSP, 2014, (accepted).