A latent variable recurrent neural network for discourse relation language models

2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference

Volumn , Issue , 2016, Pages 332-342

  Ji, Yangfeng ^a   Haffari, Gholamreza ^b   Eisenstein, Jacob ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b MONASH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); LINGUISTICS; NETWORK ARCHITECTURE; RECURRENT NEURAL NETWORKS;

LANGUAGE MODEL; LATENT VARIABLE; RELATION CLASSIFICATIONS; STATE OF THE ART; SWITCHBOARD CORPUS; TREEBANKS; VECTOR REPRESENTATIONS; WORD PREDICTION;

COMPUTATIONAL LINGUISTICS;

EID: 84994171886     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (48)

1. Yoshua Bengio. 2012. Practical recommendations for gradient-based training of deep architectures. In Neural Networks: Tricks of the Trade, pages 437-478. Springer.
2. Or Biran and Kathleen McKeown. 2013. In Proceedings of the Association for Computational Linguistics (ACL), pages 69-73, Sophia, Bulgaria.
3. Chloé Braud and Pascal Denis. 2015. Comparing word representations for implicit discourse relation classification. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 2201-2211, Lisbon, September.
4. Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. 2016. Importance weighted autoencoders. In Proceedings of the International Conference on Learning Representations (ICLR).
5. Rich Caruana. 1997. Multitask learning. Machine learning, 28(1):41-75.
6. Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. In Neural Information Processing Systems (NIPS), Montréal.
7. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493-2537.
8. João FG de Freitas, Mahesan Niranjan, Andrew H. Gee, and Arnaud Doucet. Sequential monte carlo methods to train neural network models. Neural computation, 12(4):955-993.
9. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121-2159.
10. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the Association for Computational Linguistics (ACL), pages 334-343, Beijing, China.
11. Jenny Rose Finkel, Christopher D Manning, and Andrew Y Ng. 2006. Solving the problem of cascading errors: Approximate bayesian inference for linguistic annotation pipelines. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 618-626.
12. John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992. Switchboard: Telephone speech corpus for research and development. In ICASSP, volume 1, pages 517-520. IEEE.
13. Shixiang Gu, Zoubin Ghahramani, and Richard E Turner. 2015. Neural adaptive sequential monte carlo. In Neural Information Processing Systems (NIPS), Montréal.
14. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
15. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the Association for Computational Linguistics (ACL), Baltimore, MD.
16. Yangfeng Ji and Jacob Eisenstein. 2015. One vector is not enough: Entity-augmented distributional semantics for discourse relations. Transactions of the Association for Computational Linguistics (TACL), June.
17. Yangfeng Ji, Trevor Cohn, Lingpeng Kong, Chris Dyer, and Jacob Eisenstein. 2015. Document context language models. In International Conference on Learning Representations, Poster Paper, volume abs/1511.03962.
18. Shafiq Joty, Giuseppe Carenini, and Raymond Ng. 2012. A novel discriminative framework for sentence-level discourse analysis. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP).
19. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 119-126, Sofia, Bulgaria, August. Association for Computational Linguistics.
20. Diederik P Kingma and Max Welling. 2014. Autoencoding variational bayes. In Proceedings of the International Conference on Learning Representations (ICLR).
21. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the International Conference on Machine Learning (ICML).
22. Jiwei Li, Rumeng Li, and Eduard Hovy. 2014. Recursive deep models for discourse parsing. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP).
23. Ziheng Lin, Min-Yen Kan, and Hwee Tou Ng. 2009. Recognizing implicit discourse relations in the penn discourse treebank. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 343-351, Singapore.
24. Rui Lin, Shujie Liu, Muyun Yang, Mu Li, Ming Zhou, and Sheng Li. 2015. Hierarchical recurrent neural network for document modeling. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 899-907, Lisbon, September.
25. Christopher D. Manning. 2016. Computational linguistics and deep learning. Computational Linguistics, 41(4).
26. Tomas Mikolov and Geoffrey Zweig. 2012. Context dependent recurrent neural network language model. In Proceedings of Spoken Language Technology (SLT), pages 234-239.
27. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In INTERSPEECH, pages 1045-1048.
28. Dmitrijs Milajevs and Matthew Purver. 2014. Investigating the contribution of distributional semantic information for dialogue act classification. In Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC), pages 40-47.
29. Joonsuk Park and Claire Cardie. Improving implicit discourse relation recognition through feature set optimization. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 108-112, Seoul, South Korea, July. Association for Computational Linguistics.
30. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2012. On the difficulty of training recurrent neural networks. arXiv preprint arXiv:1211.5063.
31. Vu Pham, Théodore Bluche, Christopher Kermorvant, and Jérôme Louradour. 2014. Dropout improves recurrent neural networks for handwriting recognition. In Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on, pages 285-290. IEEE.
32. Emily Pitler, Mridhula Raghupathy, Hena Mehta, Ani Nenkova, Alan Lee, and Aravind Joshi. 2008. Easily identifiable discourse relations. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 87-90, Manchester, UK.
33. Emily Pitler, Annie Louis, and Ani Nenkova. 2009. Automatic sense prediction for implicit discourse relations in text. In Proceedings of the Association for Computational Linguistics (ACL), Suntec, Singapore.
34. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of LREC.
35. Attapol T Rutherford and Nianwen Xue. 2014. Discovering implicit discourse relations through brown cluster pair representation and coreference patterns. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL).
36. Attapol Rutherford and Nianwen Xue. 2015. Improving the inference of implicit discourse relations via classifying explicit discourse connectives. pages 799-808, May-June.
37. Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. In Proceedings of the 11th International Conference on Parsing Technologies (IWPT'09), pages 81-84, Paris, France, October. Association for Computational Linguistics.
38. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), Seattle, WA.
39. Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Meg Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), Denver, CO, May.
40. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-1958.
41. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339-373.
42. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. pages 384-394.
43. Tian Wang and Kyunghyun Cho. 2015. Larger-context language modelling. arXiv preprint arXiv:1511.03729.
44. Nick Webb, Mark Hepple, and Yorick Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding.
45. Bonnie Webber, Markus Egg, and Valia Kordoni. 2012. Discourse structure and language technology. Journal of Natural Language Engineering, 1.
46. Jason D Williams and Steve Young. 2007. Partially observable markov decision processes for spoken dialog systems. Computer Speech & Language, 21(2):393-422.
47. Nianwen Xue, Hwee Tou Ng, Sameer Pradhan, Rashmi Prasad, Christopher Bryant, and Attapol T Rutherford. 2015. The CoNLL-2015 shared task on shallow discourse parsing. In Proceedings of the Conference on Natural Language Learning (CoNLL).
48. Biao Zhang, Jinsong Su, Deyi Xiong, Yaojie Lu, Hong Duan, and Junfeng Yao. 2015. Shallow convolutional neural network for implicit discourse relation recognition. In Proceedings of Empirical Methods for Natural Language Processing (EMNLP), pages 2230-2235, Lisbon, September.