A model of coherence based on distributed sentence representation

EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

Volumn , Issue , 2014, Pages 2039-2048

  Li, Jiwei ^a   Hovy, Eduard ^b  

^a Stanford University   (United States)

^b Carnegie Mellon University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS; SYNTACTICS;

ARGUMENTATION SEMANTICS; COREFERENCE; FEATURE ENGINEERINGS; NEURAL NETWORK MODEL; RECURSIVE NEURAL NETWORKS; SENTENCE STRUCTURES; STATE-OF-ART PERFORMANCE; TASK-BASED;

RECURRENT NEURAL NETWORKS;

EID: 84961375196     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/d14-1218     Document Type: Conference Paper

References (51)

1. Regina Barzilay and Mirella Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34(1):1-34.
2. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In HLT-NAACL, pages 113-120.
3. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. Neural Networks, IEEE Transactions on, 5(2):157-166.
4. Yoshua Bengio, Holger Schwenk, Jean-Sébastien Senećal, Fréderic Morin, and Jean-Luc Gauvain. 2006. Neural probabilistic language models. In Innovations in Machine Learning, pages 137-186. Springer.
5. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160-167. ACM.
6. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493-2537.
7. Ronan Collobert. 2011. Deep learning for efficient discriminative parsing. In International Conference on Artificial Intelligence and Statistics, number EPFLCONF- 192374.
8. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive sub gradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121-2159.
9. Micha Eisner and Eugene Charniak. 2011. Extending the entity grid with entity-specific features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 125-129. Association for Computational Linguistics.
10. Jeffrey L Elman. 1990. Finding structure in time. Cognitive science, 14(2):179-211.
11. Micha Elsner and Eugene Charniak. 2008. Coreference-inspired coherence modeling. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pages 41-44. Association for Computational Linguistics.
12. Micha Elsner, Joseph L Austerweil, and Eugene Charniak. 2007. A unified local and global model for discourse coherence. In HLT-NAACL, pages 436- 443.
13. Katja Filippova and Michael Strube. 2007. Extending the entity-grid coherence model to semantically related entities. In Proceedings of the Eleventh European Workshop on Natural Language Generation, pages 139-142. Association for Computational Linguistics.
14. Barbara J Grosz and Candace L Sidner. 1986. Attention, intentions, and the structure of discourse. Computational linguistics, 12(3):175-204.
15. Barbara J Grosz, Scott Weinstein, and Aravind K Joshi. 1995. Centering: A framework for modeling the local coherence of discourse. Computational linguistics, 21(2):203-225.
16. Camille Guinaudeau and Michael Strube. 2013. Graph-based local coherence modeling. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 93-103.
17. Laura Hasler. 2004. An investigation into the use of centering transitions for summarisation. In Proceedings of the 7th Annual CLUK Research Colloquium, pages 100-107.
18. Jerry R Hobbs, Mark Stickel, Paul Martin, and Douglas Edwards. 1988. Interpretation as abduction. In Proceedings of the 26th annual meeting on Association for Computational Linguistics, pages 95-103. Association for Computational Linguistics.
19. Eduard H Hovy. 1988. Planning coherent multi sentential text. In Proceedings of the 26th annual meeting on Association for Computational Linguistics, pages 163-169. Association for Computational Linguistics.
20. Ozan Irsoy and Claire Cardie. 2013. Bidirectional recursive neural networks for token-level labeling with structure. arXiv preprint arXiv:1312.0493.
21. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584.
22. Mirella Lapata and Regina Barzilay. 2005. Automatic evaluation of text coherence: Models and representations. In IJCAI, volume 5, pages 1085-1090.
23. Alex Lascarides and Nicholas Asher. 1991. Discourse relations and defeasible knowledge. In Proceedings of the 29th annual meeting on Association for Computational Linguistics, pages 55-62. Association for Computational Linguistics.
24. Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents.
25. Ziheng Lin, Hwee Tou Ng, and Min-Yen Kan. 2011. Automatically evaluating text coherence using discourse relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 997-1006. Association for Computational Linguistics.
26. Annie Louis and Ani Nenkova. 2012. A coherence model based on syntactic patterns. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1157-1168. Association for Computational Linguistics.
27. William C Mann and Sandra A Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243-281.
28. Kathleen R McKeown. 1985. Discourse strategies for generating natural-language text. Artificial Intelligence, 27(1):1-41.
29. Grégoire Mesnil, Xiaodong He, Li Deng, and Yoshua Bengio. 2013. Investigation of recurrent-neuralnetwork architectures and learning methods for spoken language understanding. Interspeech.
30. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In INTERSPEECH, pages 1045-1048.
31. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
32. Eleni Miltsakaki and Karen Kukich. 2000. The role of centering theory's rough-shift in the teaching and evaluation of writing skills. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 408-415. Association for Computational Linguistics.
33. Andriy Mnih and Geoffrey Hinton. 2007. Three new graphical models for statistical language modelling. In Proceedings of the 24th international conference on Machine learning, pages 641-648. ACM.
34. Johanna D Moore and Cecile L Paris. 1989. Planning text for advisory dialogues. In Proceedings of the 27th annual meeting on Association for Computational Linguistics, pages 203-211. Association for Computational Linguistics.
35. Wenzhe Pei, Tao Ge, and Chang Baobao. 2014. Maxmargin tensor neural network for chinese word segmentation. In Proceedings of ACL.
36. Massimo Poesio, Rosemary Stevenson, Barbara Di Eugenio, and Janet Hitzeman. 2004. Centering: A parametric theory and its instantiations. Computational linguistics, 30(3):309-363.
37. David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1988. Learning representations by backpropagating errors. MIT Press, Cambridge, MA, USA.
38. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. Signal Processing, IEEE Transactions on, 45(11):2673-2681.
39. Sarah E Schwarm and Mari Ostendorf. 2005. Reading level assessment using support vector machines and statistical language models. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 523-530. Association for Computational Linguistics.
40. Noah A Smith and Jason Eisner. 2005. Contrastive estimation: Training log-linear models on unlabeled data. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 354-362. Association for Computational Linguistics.
41. Richard Socher, Christopher D Manning, and Andrew Y Ng. 2010. Learning continuous phrase representations and syntactic parsing with recursive neural networks. In Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop, pages 1-9.
42. Richard Socher, Eric H Huang, Jeffrey Pennington, Andrew Y Ng, and Christopher D Manning. 2011a. Dynamic pooling and unfolding recursive auto encoders for paraphrase detection. In NIPS, volume 24, pages 801-809.
43. Richard Socher, Cliff C Lin, Chris Manning, and Andrew Y Ng. 2011b. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 129-136.
44. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201-1211. Association for Computational Linguistics.
45. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment tree bank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1631-1642.
46. Michael Strube and Udo Hahn. 1999. Functional centering: Grounding referential coherence in information structure. Computational linguistics, 25(3):309-344.
47. Ilya Sutskever, James Martens, and Geoffrey E Hinton. 2011. Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 1017-1024.
48. Marilyn A Walker, Aravind Krishna Joshi, and Ellen Friedman Prince. 1998. Centering theory in discourse. Oxford University Press.
49. Sida Wang and Christopher D Manning. 2012. Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, pages 90-94. Association for Computational Linguistics.
50. Houfeng Wang, Longkai Zhang, Li Li, He Zhengyan, and Ni Sun. 2013. Improving chinese word segmentation on micro-blog using rich punctuations.
51. Will Y Zou, Richard Socher, Daniel Cer, and Christopher D Manning. 2013. Bilingual word embeddings for phrase-based machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP 2013).