Recurrent conditional random field for language understanding

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 4077-4081

  Yao, Kaisheng ^a   Peng, Baolin ^a,b   Zweig, Geoffrey ^a   Yu, Dong ^a   Li, Xiaolong ^a   Gao, Feng ^a  

^a MICROSOFT   (United States)

^b BEIHANG UNIVERSITY   (China)

Author keywords

Conditional random fields; recurrent neural networks

Indexed keywords

ADVANCED TRAVELER INFORMATION SYSTEMS; CHARACTER RECOGNITION; COMPUTATIONAL LINGUISTICS; IMAGE SEGMENTATION; RECURRENT NEURAL NETWORKS; SIGNAL PROCESSING; WORLD WIDE WEB;

CONDITIONAL RANDOM FIELD; EXPLICIT MODELING; LANGUAGE MODEL; LANGUAGE UNDERSTANDING; OBJECTIVE FUNCTIONS; ON-LINE FASHION; RECURRENT NEURAL NETWORK (RNNS); TRANSITION FEATURES;

RANDOM PROCESSES;

EID: 84905222853     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854368     Document Type: Conference Paper

References (44)

1. T. Mikolov, M. Karafiat, L. Burget, J. Cernocky, and S. Khudanpur, "Recurrent neural network based language model," in INTERSPEECH, 2010, pp. 1045-1048.
2. T. Mikolov, S. Kombrink, L. Burget, J. Cernocky, and S. Khudanpur, "Extensions of recurrent neural network based language model," in ICASSP, 2011, pp. 5528-5531.
3. T. Mikolov, A. Deoras, S. Kombrink, L. Burget, and J. Cernocky, "Empirical evaluation and combination of advanced language modeling techniques," in INTERSPEECH, 2011, pp. 605-608.
4. T. Mikolov, A. Deoras, D. Povey, L. Burget, and J. Cernocky, "Strategies for training large scale neural network language models," in ASRU, 2011.
5. T. Mikolov and G. Zweig, "Context dependent recurrent neural network language model," in Proc. SLT, 2012, pp. 234-239.
6. E. Arisoy, T. N. Sainath, B. Kingsbury, and B. Ramabhadran, "Deep neural network language models," in Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model On the Future of Language Modeling for HLT, 2012, pp. 20-28.
7. K. Yao, G. Zweig, M. Hwang, Y. Shi, and Dong Yu, "Recurrent neural networks for language understanding," in INTERSPEECH, 2013.
8. G. Mesnil, X. He, L. Deng, and Y. Bengio, "Investigation of recurrentneural-network architectures and learning methods for language understanding," in INTERSPEECH, 2013.
9. M. Auli, M. Galley, C. Quirk, and G. Zweig, "Joint language and translation modeling with recurrent neural networks," in EMNLP, 2013.
10. H. Schwenk and J.L. Gauvain, "Connectionist language modeling for large vocabulary continuous speech recognition," in ICASSP. IEEE, 2002, vol. 1, pp. I-765.
11. Y. Bengio, R. Ducharme, Vincent, P., and C. Jauvin, "A neural probabilistic language model," Journal of Machine Learning Reseach, vol. 3, no. 6, 2003.
12. H. Schwenk, "Continuous space language models," Computer Speech and Language, vol. 21, no. 3, pp. 492-518, 2007.
13. H.-S. Le, I. Oparin, A. Allauzen, J.-L. Gauvain, and F. Yvon, "Structured output layer neural network language model," in ICASSP, 2011, pp. 5524-5527.
14. F. Morin and Y. Bengio, "Hierarchical probabilistic neural network language model," in Proceedings of the international workshop on artificial intelligence and statistics, 2005, pp. 246-252.
15. T. Mikolov, W.T. Yih, and G. Zweig, "Linguistic regularities in continuous space word representations," in NAACL-HLT, 2013.
16. C. Hemphill, J. Godfrey, and G. Doddington, "The ATIS spoken language systems pilot corpus," in Proceedings of the DARPA speech and natural language workshop, 1990, pp. 96-101.
17. P. Price, "Evaluation of spoken language systems: The ATIS domain," in Proceedings of the Third DARPA Speech and Natural Language Workshop. Morgan Kaufmann, 1990, pp. 91-95.
18. W. Ward et al., "The CMU air travel information service: Understanding spontaneous speech," in Proceedings of the DARPA Speech and Natural Language Workshop, 1990, pp. 127-129.
19. Y. He and S. Young, "A data-driven spoken language understanding system," in ASRU, 2003, pp. 583-588.
20. C. Raymond and G. Riccardi, "Generative and discriminative algorithms for spoken language understanding," INTERSPEECH, pp. 1605-1608, 2007.
21. R. De Mori, "Spoken language understanding: A survey," in ASRU, 2007, pp. 365-376.
22. F. Béchet, "Processing spontaneous speech in deployed spoken language understanding systems: A survey," SLT, vol. 1, 2008.
23. J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," in ICML, 2001, pp. 282-289.
24. Y.-Y. Wang, A. Acero, M. Mahajan, and J. Lee, "Combining statistical and knowledge-based spoken language understanding in conditional models," in COLING/ACL, 2006, pp. 882-889.
25. A. Moschitti, G. Riccardi, and C. Raymond, "Spoken language understanding with kernels for syntactic/semantic structures," in ASRU, 2007, pp. 183-188.
26. A. Ratnaparkhi, "A maximum entropy model for part-of-speech tagging," in Proc. EMNLP, 1996.
27. K. Macherey, F. Och, H. Ney, et al., "Natural language understanding using statistical machine translation," in European Conf. on Speech Communication and Technology. Citeseer, 2001, pp. 2205-2208.
28. M. Jordan, "Serial order: A parallel distributed processing approach," Tech. Rep. 8604, Institute for Cognitive Science, University of California at San Diego, 1986.
29. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, "Natural language processing (almost) from scratch," Journal of Machine Learning Research, vol. 12, pp. 2493-2537, Aug 2011.
30. A.-R. Mohamed, D. Yu, and L. Deng, "Investigation of full-sequence training of deep belief networks for speech recognition," in INTERSPEECH, 2010, pp. 2846-2849.
31. J. Peng, L. Bo, and J. Xu, "Conditional Neural Fields," in Advances in Neural Information Processing Systems, December 2009.
32. Y. Fujii, K. Yamamoto, and S. Nakagawa, "Deep-hidden conditional neural fields for continuous phoneme speech recognition," in Proc. International Workshop of Statistical Machine Learning for Speech Processing (IWSML), 2012.
33. D. Yu, S. Wang, and L. Deng, "Sequential labeling using deepstructured conditional random fields," Journal of Selected Topics in Signal Processing, vol. 4, no. 6, pp. 965-973, 2010.
34. P. Xu and R. Sarikaya, "Convolutional neural network based triangular CRF for joint intent detection and slot filling," in ASRU, 2013.
35. K. Vesely, A. Ghoshal, L. Burget, and D. Povey, "Sequencediscriminative training of deep neural networks," in INTERSPEECH, 2013.
36. B. Kingsbury, T. N. Sainath, and H. Soltau, "Scalable minimum Bayes risk training of deep neural network acoustic models using distributed Hessian-free optimization," in INTERSPEECH, 2012.
37. H. Su, G. Li, D. Yu, and F. Seide, "Error back propagation for sequence training of context-dependent deep neural networks for conversational speech transcription," in ICASSP, 2013.
38. J.L. Elman, "Finding structure in time," Cognitive Science, vol. 14, no. 2, pp. 179-211, 1990.
39. S. Sarawagi andW. Cohen, "Semi-Markov Conditional Random Fields for Information Extraction," in Proc. NIPS, 2005.
40. H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach, Kluwer Press, 1993.
41. Y.-Y. Wang and A. Acero, "Evaluation of spoken language grammar learning in the ATIS domain," in ICASSP, 2002.
42. D. Dahl, M. Bates, M. Brown,W. Fisher, K. Hunicke-Smith, D. Pallett, C. Pao, A. Rudnicky, and E. Shriberg, "Expanding the scope of the ATIS task: The ATIS-3 corpus," in Proceedings of the workshop on Human Language Technology. Association for Computational Linguistics, 1994, pp. 43-48.
43. G. Tur, D. Hakkani-Tur, L. Heck, and S. Parthasarathy, "Sentence simplification for spoken language understanding," in ICASSP, 2011, pp. 5628-5631.
44. L. Deng, G. Tur, X. He, and D. Hakkani-Tur, "Use of kernal deep convex networks and end-to-end learning for spoken language understanding," in SLT. IEEE, 2013, pp. 210-215.