Kernel methods match deep neural networks on TIMIT

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 205-209

  Huang, Po Sen ^a   Avron, Haim ^b   Sainath, Tara N ^b   Sindhwani, Vikas ^b   Ramabhadran, Bhuvana ^b  

^a University of Illinois at Urbana Champaign   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

deep learning; distributed computing; large scale kernel machines; random features; speech recognition

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; REGRESSION ANALYSIS; SPEECH RECOGNITION;

BLOCK COORDINATE DESCENTS; CLASSIFICATION ERROR RATE; COMPUTATIONAL BOTTLENECKS; CONVEX OPTIMIZATION TECHNIQUES; DEEP LEARNING; KERNEL MACHINE; KERNEL RIDGE REGRESSIONS; RANDOM FEATURES;

SIGNAL PROCESSING;

EID: 84905226732     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6853587     Document Type: Conference Paper

References (15)

1. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kings-bury, "Deep neural networks for acoustic modeling in speech recognition," IEEE Signal Processing Magazine, vol. 29, pp. 82-97, Nov. 2012.
2. B. Schölkopf and A. J. Smola, Learning with kernels: support vector machines, regularization, optimization, and beyond, MIT press, 2001.
3. P. Drineas and M. W. Mahoney, "On the Nyström method for approximating a gram matrix for improved kernel-based learning," Journal of Machine Learning Research, vol. 6, pp. 2153-2175, Dec. 2005.
4. S. Kumar, M. Mohri, and A. Talwalkar, "Ensemble Nyström method," in Advances in Neural Information Processing Systems (NIPS), 2009.
5. A. Rahimi and B. Recht, "Random features for large-scale kernel machines," in Advances in Neural Information Processing Systems (NIPS), 2007.
6. Q. Le, T. Sarlós, and A. Smola, "Fastfood-Approximating kernel expansions in loglinear time," in International Conference on Machine Learning (ICML), 2013.
7. T. Yang, Y.-F. Li, M. Mahdavi, R. Jin, and Z.-H. Zhou, "Nyström method vs random fourier features: A theoretical and empirical comparison," in Advances in Neural Information Processing Systems (NIPS), 2012, pp. 485-493.
8. Y. Cho and L. K. Saul, "Kernel methods for deep learning," in Advances in Neural Information Processing Systems (NIPS), 2009.
9. C.-C. Cheng and B. Kingsbury, "Arccosine kernels: Acoustic modeling with infinite neural networks," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2011, pp. 5200-5203.
10. L. Deng, G. Tur, X. He, and D. Hakkani-Tur, "Use of kernel deep convex networks and end-to-end learning for spoken language understanding," in IEEEWorkshop on Spoken Language Technology, 2012.
11. P.-S. Huang, L. Deng, M. Hasegawa-Johnson, and X. He, "Random features for kernel deep convex network," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013.
12. H. Xia and S. C.H. Hoi, "Mkboost: A framework of multiple kernel boosting," IEEE Transactions on Knowledge and Data Engineering, vol. 25, no. 7, pp. 1574-1586, 2013.
13. M.J.F. Gales, "Maximum likelihood linear transformations for hmm-based speech recognition," Computer Speech & Language, vol. 12, no. 2, pp. 75-98, 1998.
14. A. Mohamed, T. N. Sainath, G. Dahl, B. Ramabhadran, G. Hinton, and M. Picheny, "Deep belief networks using discriminative features for phone recognition," in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2011, pp. 5060-5063.
15. J. Yang, V. Sindhwani, H. Avron, and M. Mahoney, "Quasi-Monte Carlo feature maps for shift-invariant kernels," in International Conference on Machine Learning (ICML), 2014.