Skip-Gram - Zipf + Uniform = Vector additivity

ACL 2017 - 55th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)

Volumn 1, Issue , 2017, Pages 69-76

  Gittens, Alex ^a   Achlioptas, Dimitris ^b   Mahoney, Michael W ^c  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b University of California Santa Cruz   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CALCULATIONS; COMPUTATIONAL LINGUISTICS; LINGUISTICS; SEMANTICS;

COMPOSITION OPERATORS; COMPOSITIONALITY; DIMENSIONALITY REDUCTION; EMBEDDINGS; GRAM MODELS; SIDE EFFECT; VECTOR CALCULUS; WORD VECTORS;

VECTORS;

EID: 85040943135     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/P17-1007     Document Type: Conference Paper

References (16)

1. Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016. A latent variable model approach to PMI-based word embeddings. Transactions of the Association for Computational Linguistics 4:385-399.
2. Ehsaneddin Asgari and Mohammad R.K. Mofrad. 2015. Continuous distributed representation of biological sequences for deep proteomics and ge-nomics. PloS One 10(11).
3. Yoshua Bengio, Rejean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A Neural Probabilistic Language Model. Journal Of Machine Learning Research 3:1137-1155.
4. Kenneth Ward Church and Patrick Hanks. 1990. Word Association Norms, Mutual Information, and Lexicography. Computational Linguistics 16(1):22-29.
5. J.R. Firth. 1957. A synopsis of linguistic theory 1930-1955. Studies in Linguistic Analysis pages 1-32.
6. Amir Globerson and Naftali Tishby. 2003. Sufficient Dimensionality Reduction. Journal of Machine Learning Research 3:1307-1331.
7. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable Feature Learning for Networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pages 855-864.
8. Omer Levy and Yoav Goldberg. 2014. Linguistic Regularities in Sparse and Explicit Word Representations. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning. pages 171-180.
9. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient Estimation of Word Representations in Vector Space. In International Conference on Learning Representations.
10. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013b. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems. pages 3111-3119.
11. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013c. Linguistic regularities in continuous space word representations. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings. pages 746-751.
12. Andriy Mnih and Geoffrey Hinton. 2007. Three New Graphical Models for Statistical Language Mod-elling. In Proceedings of the 24th International Conference on Machine Learning. ACM, pages 641-648.
13. Denis Paperno and Marco Baroni. 2016. When the Whole is Less than the Sum of Its Parts: How Composition Affects PMI Values in Distributional Semantic Vectors. Computational Linguistics 42:345-350.
14. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP). pages 1532-1543.
15. Steven T. Piantadosi. 2014. Zipf's word frequency law in natural language: A critical review and future directions. Psychonomic Bulletin & Review 21(5):1112-1130.
16. Douglas L. T. Rohde, Laura M. Gonnerman, and David C. Plaut. 2006. An improved model of semantic similarity based on lexical co-occurence. Communications of the ACM 8:627-633.