Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014 - Proceedings of the Conference

Volumn 1, Issue , 2014, Pages 238-247

  Baroni, Marco ^a   Dinu, Georgiana ^a   Kruszewski, Germán ^a  

^a UNIVERSITY OF TRENTO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; FORECASTING;

DISTRIBUTIONAL SEMANTICS; LANGUAGE MODEL; LEXICAL SEMANTICS; PARAMETER SETTING; PREDICTING MODELS; PREDICTIVE MODELS; SEMANTIC APPROACH; SEMANTIC VECTORS;

SEMANTICS;

EID: 84906930943     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/v1/p14-1023     Document Type: Conference Paper

References (46)

1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasça, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and WordNet-based approaches. In Proceedings of HLT-NAACL, pages 19-27, Boulder, CO.
2. Abdulrahman Almuhareb. 2006. Attributes in Lexical Acquisition. Phd thesis, University of Essex.
3. Marco Baroni and Alessandro Lenci. 2010. Distributional Memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673-721.
4. Marco Baroni, Stefan Evert, and Alessandro Lenci, editors. 2008. Bridging the Gap between Semantic Theory and Computational Simulations: Proceedings of the ESSLLI Workshop on Distributional Lexical Semantic. FOLLI, Hamburg.
5. Marco Baroni, Eduard Barbu, Brian Murphy, and Massimo Poesio. 2010. Strudel: A distributional semantic model based on properties and types. Cognitive Science, 34(2):222-254.
6. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155.
7. William Blacoe and Mirella Lapata. 2012. A comparison of vector-based representations for semantic composition. In Proceedings of EMNLP, pages 546-556, Jeju Island, Korea.
8. David Blei, Andrew Ng, and Michael Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022.
9. Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2013. Multimodal distributional semantics. Journal of Artificial Intelligence Research. In press; http://clic.cimec.unitn.it/marco/publications/mmds-jair.pdf.
10. John Bullinaria and Joseph Levy. 2007. Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39:510-526.
11. John Bullinaria and Joseph Levy. 2012. Extracting semantic representations from word co-occurrence statistics: Stop-lists, stemming and SVD. Behavior Research Methods, 44:890-907.
12. Yanqing Chen, Bryan Perozzi, Rami Al-Rfou', and Steven Skiena. 2013. The expressive power of word embeddings. In Proceedings of the ICML Workshop on Deep Learning for Audio, Speech and Language Processing, Atlanta, GA. Published online: https://sites.google.com/site/ deeplearningicml2013/accepted- papers.
13. Stephen Clark. 2013. Vector space models of lexical meaning. In Shalom Lappin and Chris Fox, editors, Handbook of Contemporary Semantics, 2nd ed. Blackwell, Malden, MA. In press; http://www.cl.cam.ac.uk/ ~sc609/pubs/sem- handbook.pdf.
14. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML, pages 160-167, Helsinki, Finland.
15. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493-2537.
16. Katrin Erk. 2012. Vector space models of word meaning and phrase meaning: A survey. Language and Linguistics Compass, 6(10):635-653.
17. Stefan Evert. 2005. The Statistics of Word Cooccurrences. Ph.D dissertation, Stuttgart University.
18. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1):116-131.
19. Andrea Frome, Greg Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. 2013. DeViSE: A deep visual-semantic embedding model. In Proceedings of NIPS, pages 2121-2129, Lake Tahoe, Nevada.
20. Gene Golub and Charles Van Loan. 1996. Matrix Computations (3rd ed.). JHU Press, Baltimore, MD.
21. Tom Griffiths, Mark Steyvers, and Josh Tenenbaum. 2007. Topics in semantic representation. Psychological Review, 114:211-244.
22. Guy Halawi, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. 2012. Large-scale learning of word relatedness with constraints. In Proceedings of KDD, pages 1406-1414.
23. Samer Hassan and Rada Mihalcea. 2011. Semantic relatedness using salient semantic analysis. In Proceedings of AAAI, pages 884-889, San Francisco, CA.
24. Amaç Herdaǧdelen and Marco Baroni. 2009. Bag- Pack: A general framework to represent semantic relations. In Proceedings of GEMS, pages 33-40, Athens, Greece.
25. Eric Huang, Richard Socher, Christopher Manning, and Andrew Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of ACL, pages 873-882, Jeju Island, Korea.
26. George Karypis. 2003. CLUTO: A clustering toolkit. Technical Report 02-017, University of Minnesota Department of Computer Science.
27. Sophia Katrenko and Pieter Adriaans. 2008. Qualia structures and their impact on the concrete noun categorization task. In Proceedings of the ESSLLI Workshop on Distributional Lexical Semantics, pages 17-24, Hamburg, Germany.
28. Thomas Landauer and Susan Dumais. 1997. A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211-240.
29. Daniel Lee and Sebastian Seung. 2000. Algorithms for Non-negative Matrix Factorization. In Proceedings of NIPS, pages 556-562.
30. Chih-Jen Lin. 2007. Projected gradient methods for Nonnegative Matrix Factorization. Neural Computation, 19(10):2756-2779.
31. Ken McRae, Michael Spivey-Knowlton, and Michael Tanenhaus. 1998. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38:283-312.
32. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. http://arxiv.org/abs/1301. 3781/.
33. Tomas Mikolov, Quoc Le, and Ilya Sutskever. 2013b. Exploiting similarities among languages for Machine Translation. http://arxiv.org/abs/1309. 4168.
34. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. 2013c. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111-3119, Lake Tahoe, Nevada.
35. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013d. Linguistic regularities in continuous space word representations. In Proceedings of NAACL, pages 746-751, Atlanta, Georgia.
36. George Miller and Walter Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1-28.
37. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161-199.
38. Ulrike Padó. 2007. The Integration of Syntax and Semantic Plausibility in a Wide-Coverage Model of Sentence Processing. Dissertation, Saarland University, Saarbrücken.
39. Klaus Rothenhäusler and Hinrich Schütze. 2009. Unsupervised classification with dependency based word spaces. In Proceedings of GEMS, pages 17-24, Athens, Greece.
40. Herbert Rubenstein and John Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627-633.
41. Magnus Sahlgren. 2006. The Word-Space Model. Ph.D dissertation, Stockholm University.
42. Richard Socher, Brody Huval, Christopher Manning, and Andrew Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of EMNLP, pages 1201-1211, Jeju Island, Korea.
43. Richard Socher, Milind Ganjoo, Christopher Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In Proceedings of NIPS, pages 935-943, Lake Tahoe, Nevada.
44. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of ACL, pages 384-394, Uppsala, Sweden.
45. Peter Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141-188.
46. Peter Turney. 2012. Domain and function: A dualspace model of semantic relations and compositions. Journal of Artificial Intelligence Research, 44:533-585.