Use of kernel deep convex networks and end-to-end learning for spoken language understanding

2012 IEEE Workshop on Spoken Language Technology, SLT 2012 - Proceedings

Volumn , Issue , 2012, Pages 210-215

  Deng, Li ^a   Tur, Gokhan ^a   He, Xiaodong ^a   Hakkani Tur, Dilek ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

deep learning; domain detection; kernel learning; slot filling; spoken language understanding

Indexed keywords

CLASSIFICATION TASKS; DEEP CONVEX NETWORKS; DEEP LEARNING; DISCRIMINATIVE FEATURES; DOMAIN DETECTIONS; ERROR REDUCTION; F-MEASURE; FILLING SYSTEM; HIDDEN UNITS; HIGHLY-CORRELATED; KERNEL LEARNING; KERNEL TRICK; LEARNING OBJECTIVES; LEARNING STRATEGY; PERFORMANCE MEASURE; SEARCH QUERY CLICK LOGS; SPOKEN LANGUAGE UNDERSTANDING;

FILLING;

EID: 84874256530     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2012.6424224     Document Type: Conference Paper

References (30)

1. G. Baudat and F. Anouar. "Feature vector selection and projection using kernels," Neurocomputing, Vol. 55, pp. 22-38, 2003.
2. C. Bishop. Pattern Recognition and Machine Learning, Springer, 2006.
3. G. Cawley and N. Talbot, "Reduced rank kernel ridge regression," Neural Processing Letters, vol. 16, 2002.
4. Z. Chen and S. Haykin, "On different facets of regularization theory," Neural Computation 14, 2791-2846, 2002.
5. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pretrained deep neural networks for large vocabulary speech recognition," IEEE Trans. Audio, Speech, and Lang. Proc. Jan. 2012.
6. L. Deng and D. Yu. "Deep Convex Network: A scalable architecture for speech pattern classification," Proc. Interspeech, 2011.
7. L. Deng, D. Yu, and J. Platt. "Scalable stacking and learning for building deep architectures," Proc. ICASSP, 2012.
8. L. Deng, M. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. Hinton. Binary coding of speech spectrograms using a deep auto-encoder. In Proc. Interspeech, September 2010.
9. S. Hahn, M. Dinarelli, C. Raymond, F. Lefevre, P. Lehnen, R. de Mori, A. Moschitti, H. Ney, and G. Riccardi. "Comparing Stochastic Approaches to Spoken Language Understanding in Multiple Languages," IEEE TASLP, vol. 19, 2011.
10. D. Hakkani-Tür, G. Tur, R. Iyer, L. Heck, "Translating Natural Language Utterances to Search Queries for SLU Domain Detection Using Query Click Logs," Proc. ICASSP 2012.
11. X. He and L. Deng. "Maximum Expected BLEU Training of Phrase and Lexicon Translation Models", Proc. ACL, 2012.
12. X. He, L. Deng, W. Chou. "Discriminative learning in sequential pattern recognition", IEEE Signal Processing Magazine, Sept. 2008.
13. G. Hinton, L. Deng, D. Yu, G. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. Sainath, and B. Kingsbury. "Deep Neural Networks for Acoustic Modeling in Speech Recognition." IEEE Signal Processing Magazine, Vol. 29, No. 6, November, 2012.
14. T. Hofmann and B. Scholkopf, and A. Smola, "Kernel methods in machine learning," The Annals of Statistics, Vol. 36, No. 3, pp. 1171-1220, 2008.
15. G. Huang, H. Zhou, and X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classification," IEEE Trans. Systems, Man, and Cybernetics (Part B), Vol. 42, No. 2, pp. 513-529, April 2012.
16. B. Hutchinson, L. Deng, and D. Yu, "A deep architecture with bilinear modeling of hidden representations: Applications to phonetic recognition," Proc. ICASSP 2012.
17. H. Kadri, A. Rabaoui, P. Preux, E. Duflos, and A. Rakotomanonjy, "Functional regularized least squares classification with operator-valued kernels," Proc. ICML, 2011.
18. Karasuyama, M. and Nakano, R. "Optimizing sparse kernel ridge regression hyperparameters based on leave-one-out cross-validation," Proc. IJCNN 2008. vol., no., pp. 3463-3468, June 2008.
19. K. Macherey, O. Bender, H. Ney, "Applications of Statistical Machine Translation Approaches to Spoken Language Understanding", IEEE TASLP, vol. 17, 2009
20. A. Mohamed, G. E. Dahl, and G. E. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. on Audio, Speech, and Lang. Proc. Jan. 2012.
21. A. Mohamed, D. Yu, and L. Deng, "Investigation of fullsequence training of deep belief networks for speech recognition," Proc. Interspeech 2010, pp. 1692-1695.
22. R. Monga, J. Dean, G. Corrado, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, P. Tucker, and A. Ng. "Downpour and Sandblaster: Two Complementary Approaches to Large-scale Distributed Optimization," ICML Workshop, 2012.
23. F. Och. "Minimum error rate training in statistical machine translation", Proc. of ACL 2003.
24. C. Raymond and G. Riccardi, Generative and Discriminative Algorithms for Spoken Language Understanding, Proc. Interspeech 2007.
25. R. Rosipal and L. Trejo, "Kernel partial least squares regression in reproducing kernel Hilbert space," J. Machine Learning Research, vol. 2, pp. 97-123, 2001.
26. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," Proc. ICML, 1998.
27. G. Tur, L. Deng, D. Hakkani-Tür, and X. He. "Towards deep understanding: Deep convex networks for semantic utterance classification," Proc. ICASSP, 2012.
28. G. Tur and R. De Mori (eds): Spoken Language Understanding-Systems for Extracting Semantic Information from Speech, John Wiley and Sons, 2011.
29. G. Tur, D. Hakkani-Tür, and L. Heck. "What's Left to Be Understood in ATIS?," Proc. IEEE SLT Workshop, 2010.
30. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu and P. Kuksa. "Natural Language Processing (Almost) from Scratch," Journal of Machine Learning Research (JMLR), 2011