Application of deep belief networks for natural language understanding

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 4, 2014, Pages 778-784

  Sarikaya, Ruhi ^a   Hinton, Geoffrey E ^b   Deoras, Anoop ^a  

^a MICROSOFT   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Call routing; DBN; Deep learning; Deep neural nets; Natural language understanding; RBM

Indexed keywords

CALL-ROUTING; DBN; DEEP LEARNING; DEEP NEURAL NETS; NATURAL LANGUAGE UNDERSTANDING; RBM; CALL ROUTING;

AUDIO ACOUSTICS; CLASSIFICATION (OF INFORMATION); IMAGE CLASSIFICATION; NEURAL NETWORKS; SPEECH RECOGNITION; COMPUTATIONAL LINGUISTICS; LEARNING ALGORITHMS; SUPPORT VECTOR MACHINES; TEXT PROCESSING;

SUPPORT VECTOR MACHINES; IMAGE CLASSIFICATION;

EID: 84898070747     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2303296     Document Type: Article

References (22)

1. P. J. Price, "Evaluation of spoken language systems: The ATIS domain," in Proc. DARPA Workshop Speech Nat. Lang., Hidden Valley, PA, USA, Jun. 1990.
2. Y.-Y. Wang and A. Acero, "Discriminative models for spoken language understanding," in Proc. ICSLP, Pittsburgh, PA, USA, Sep. 2006.
3. C. Raymond and G. Riccardi, "Generative and discriminative algorithms for spoken language understanding," in Proc. Interspeech, Antwerp, Belgium, 2007.
4. J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," in Proc. Int. Conf. Mach. Learn., 2001.
5. A. L. Gorin, G. Riccardi, and J. H. Wright, "How may I help you?," Speech Commun., vol. 23, pp. 113-127, 1997.
6. R. E. Schapire and Y. Singer, "Boostexter: A boosting based system for text categorization," Mach. Learn., vol. 39, no. 2/3, pp. 135-168, 2000.
7. V. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer-Verlag, 1995.
8. P. Haffner, G. Tur, and J. Wright, "Optimizing SVMs for complex call classification," in Proc. ICASSP, Hong Kong, Apr. 2003, pp. 632-635.
9. Spoken Language Understanding: Systems for Extracting Semantic Information from Speech, G. Tur and R. D. Mori, Eds. New York, NY, USA: Wiley, 2011.
10. G. E. Hinton, "Learning multiple layers of representation," TRENDS Cognitive Sci., vol. 11, no. 10, pp. 428-434, 2007.
11. G. E. Dahl, M. Ranzato, A. Momamed, and G. E. Hinton, "Phone Recognition with the Mean-Covariance Restricted Boltzmann Machines," in Advances in Neural Information Processing Systems NIPS. Cambridge, MA, USA: MIT Press, 2010.
12. M. Welling, M. Rosen-Zvi, and G. E. Hinton, "Exponential family of harmoniums with an application to information retrieval," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2005, pp. 1481-1488.
13. D. Erhan, Y. Bengio, A. Courville, P. Manzagol, and P. Vincent, "Why Does Unsupervised Pre-training Help Deep Learning?," J. Mach. Learn. Res., vol. 11, pp. 625-660, 2010.
14. A. Y. Ng and M. I. Jordan, "On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2002, vol. 11.
15. R. Sarikaya, G. Hinton, and B. Ramabhadran, "Deep belief networks for natural language call-routing," in Proc. ICASSP, 2011, pp. 5680-5683.
16. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, pp. 1771-1800, 2002.
17. G. E. Hinton, "A practical guide to training restricted Boltzmann machines," Univ. of Toronto Mach. Learn. Tech. Rep., UTML TR 2010-003.
18. G. E. Hinton, "Training product of experts by minimizing constrastive divergence," Neural Comput., vol. 14, no. 18, pp. 1527-1554, 2002.
19. G. E. Hinton, S. Osindero, and Y. W. Teh, "A fast learning algorithm for deep belief nets," Adv. Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
20. S. Chen and R. Rosenfeld, "A survey of smoothing techniques for ME models," IEEE Trans. Speech Audio Process., vol. 8, no. 1, pp. 37-50, Jan. 2001.
21. S. D. Pietra, V. D. Pietra, and J. Lafferty, "Inducing features of random fields," IEEE Trans. Pattern. Anal. Mach. Intell., vol. 19, no. 4, pp. 380-93, Apr. 1997.
22. R. Sarikaya, H-K. J. Kuo, V. Goel, and Y. Gao, "Exploiting Unlabeled Data Using Multiple Classifiers for Improved Natural Language Call-Routing," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005.