Improved semantic representations from tree-structured long short-Term memory networks

ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference

Volumn 1, Issue , 2015, Pages 1556-1566

  Tai, Kai Sheng ^a   Socher, Richard ^b   Manning, Christopher D ^a  

^a Stanford University   (United States)

^b Meta   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; CLASSIFICATION (OF INFORMATION); COMPUTATIONAL LINGUISTICS; FORESTRY; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTIC WEB; SEMANTICS; SYNTACTICS;

COMPUTATIONAL UNITS; NATURAL LANGUAGES; SEMANTIC RELATEDNESS; SEMANTIC REPRESENTATION; SENTIMENT CLASSIFICATION; SEQUENCE INFORMATIONS; SYNTACTIC PROPERTIES; TREE-STRUCTURED NETWORKS;

LONG SHORT-TERM MEMORY;

EID: 84943797465     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p15-1150     Document Type: Conference Paper

References (41)

1. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015).
2. Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. 1994. Learning long-Term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5(2).
3. Bjerva, Johannes, Johan Bos, Rob van der Goot, and Malvina Nissim. 2014. The Meaning Factory: Formal semantics for recognizing textual entailment and determining semantic similarity. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).
4. Blunsom, Phil, Edward Grefenstette, Nal Kalchbrenner, et al. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
5. Chen, Danqi and Christopher D Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
6. Collobert, Ronan, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research 12:2493-2537.
7. Duchi, John, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research 12:2121-2159.
8. Elman, Jeffrey L. 1990. Finding structure in time. Cognitive science 14(2):179-211.
9. Foltz, Peter W, Walter Kintsch, and Thomas K Landauer. 1998. The measurement of textual coherence with latent semantic analysis. Discourse processes 25(2-3):285-307.
10. Ganitkevitch, Juri, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The Paraphrase Database. In Proceedings of HLTNAACL 2013.
11. Goller, Christoph and Andreas Kuchler. 1996. Learning task-dependent distributed representations by backpropagation through structure. In IEEE International Conference on Neural Networks.
12. Graves, Alex, Navdeep Jaitly, and A-R Mohamed. 2013. Hybrid speech recognition with deep bidirectional LSTM. In IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).
13. Grefenstette, Edward, Georgiana Dinu, Yao-Zhong Zhang, Mehrnoosh Sadrzadeh, and Marco Baroni. 2013. Multi-step regression learning for compositional distributional semantics. In Proceedings of the 10th International Conference on Computational Semantics.
14. Hochreiter, Sepp. 1998. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6(02):107-116.
15. Hochreiter, Sepp and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9(8).
16. Huang, Eric H., Richard Socher, Christopher D. Manning, and Andrew Y. Ng. 2012. Improving word representations via global context and multiple word prototypes. In Annual Meeting of the Association for Computational Linguistics (ACL).
17. Irsoy, Ozan and Claire Cardie. 2014. Deep recursive neural networks for compositionality in language. In Advances in Neural Information Processing Systems.
18. Jimenez, Sergio, George Duenas, Julia Baquero, Alexander Gelbukh, Av Juan Dios Bátiz, and Av Mendizábal. 2014. UNAL-NLP: Combining soft cardinality features for semantic textual similarity, relatedness and entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).
19. Kim, Yoon. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).
20. Klein, Dan and Christopher D Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics.
21. Lai, Alice and Julia Hockenmaier. 2014. Illinois-LH: A denotational and distributional approach to semantics. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).
22. Landauer, Thomas K and Susan T Dumais. 1997. A solution to Platos problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological review 104(2):211.
23. Le, Quoc and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning (ICML-14).
24. Marelli, Marco, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. SemEval-2014 Task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).
25. Mikolov, Tomáš. 2012. Statistical Language Models Based on Neural Networks. Ph.D. Thesis, Brno University of Technology.
26. Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems.
27. Mitchell, Jeff and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive science 34(8):1388-1429.
28. Pennington, Jeffrey, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empiricial Methods in Natural Language Processing (EMNLP).
29. Rumelhart, David E, Geoffrey E Hinton, and Ronald J Williams. 1988. Learning representations by back-propagating errors. Cognitive modeling 5.
30. Socher, Richard, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrixvector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP).
31. Socher, Richard, Andrej Karpathy, Quoc V Le, Christopher D Manning, and Andrew Y Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. Transactions of the Association for Computational Linguistics 2.
32. Socher, Richard, Cliff C Lin, Chris Manning, and Andrew Y Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11).
33. Socher, Richard, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP).
34. Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15:1929-1958.
35. Srivastava, Nitish, Ruslan Salakhutdinov, and Geoffrey Hinton. 2013. Modeling documents with a Deep Boltzmann Machine. In Uncertainty in Artificial Intelligence.
36. Sutskever, Ilya, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems.
37. Turian, Joseph, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.
38. Vinyals, Oriol, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2014. Show and tell: A neural image caption generator. arXiv preprint arXiv:1411.4555 .
39. Yessenalina, Ainur and Claire Cardie. 2011. Compositional matrix-space models for sentiment analysis. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP).
40. Zaremba, Wojciech and Ilya Sutskever. 2014. Learning to execute. arXiv preprint arXiv:1410.4615 .
41. Zhao, Jiang, Tian Tian Zhu, and Man Lan. 2014. ECNU: One stone two birds: Ensemble of heterogenous measures for semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014).