Linguistic regularities in continuous spaceword representations

NAACL HLT 2013 - 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Main Conference

Volumn , Issue , 2013, Pages 746-751

  Mikolov, Tomas ^a,b   Yih, Wen Tau ^a   Zweig, Geoffrey ^a  

^a MICROSOFT RESEARCH   (United States)

^b GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SEMANTICS; VECTOR SPACES; VECTORS;

CONTINUOUS SPACES; INPUT LAYERS; LANGUAGE MODEL; OFFSET METHOD; VECTOR REPRESENTATIONS; WORD REPRESENTATIONS; WORD VECTORS;

SYNTACTICS;

EID: 84926179397     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. Y. Bengio, R. Ducharme, Vincent, P., and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Reseach, 3(6).
2. Y. Bengio, H. Schwenk, J. S. Senécal, F. Morin, and J. L. Gauvain. 2006. Neural probabilistic language models. Innovations in Machine Learning, pages 137-186.
3. A. Bordes, X. Glorot, J. Weston, and Y. Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of 15th International Conference on Artificial Intelligence and Statistics.
4. R. Collobert and J. Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160-167. ACM.
5. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(96).
6. J. L. Elman. 1991. Distributed representations, simple recurrent networks, and grammatical structure. Machine learning, 7(2):195-225.
7. G. E. Hinton and R. R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507.
8. G. Hinton and R. Salakhutdinov. 2010. Discovering binary codes for documents by learning deep generative models. Topics in Cognitive Science, 3(1):74-91.
9. G. E. Hinton. 1986. Learning distributed representations of concepts. In Proceedings of the eighth annual conference of the cognitive science society, pages 1-12.
10. Amherst, MA. David Jurgens, Saif Mohammad, Peter Turney, and Keith Holyoak. 2012. Semeval-2012 task 2: Measuring degrees of relational similarity. In SEM 2012: The First Joint Conference on Lexical and Computational Semantics (SemEval 2012), pages 356-364. Association for Computational Linguistics.
11. Hai-Son Le, I. Oparin, A. Allauzen, J.-L. Gauvain, and F. Yvon. 2011. Structured output layer neural network language model. In Proceedings of ICASSP 2011.
12. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: the penn treebank. Computational Linguistics, 19(2):313-330.
13. Tomas Mikolov, Martin Karafiat, Jan Cernocky, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Proceedings of Interspeech 2010.
14. Tomas Mikolov, Anoop Deoras, Daniel Povey, Lukas Burget, and Jan Cernocky. 2011a. Strategies for Training Large Scale Neural Network Language Models. In Proceedings of ASRU 2011.
15. Tomas Mikolov, Stefan Kombrink, Lukas Burget, Jan Cernocky, and Sanjeev Khudanpur. 2011b. Extensions of recurrent neural network based language model. In Proceedings of ICASSP 2011. Tomas Mikolov. 2012. RNN toolkit.
16. A. Mnih and G. E. Hinton. 2009. A scalable hierarchical distributed language model. Advances in neural information processing systems, 21:1081-1088.
17. F. Morin and Y. Bengio. 2005. Hierarchical probabilistic neural network language model. In Proceedings of the international workshop on artificial intelligence and statistics, pages 246-252.
18. J. B. Pollack. 1990. Recursive distributed representations. Artificial Intelligence, 46(1):77-105.
19. Bryan Rink and Sanda Harabagiu. 2012. UTD: Determining relational similarity using lexical patterns. In SEM 2012: The First Joint Conference on Lexical and Computational Semantics (SemEval 2012), pages 413-418. Association for Computational Linguistics.
20. Holger Schwenk. 2007. Continuous space language models. Computer Speech and Language, 21(3):492-518.
21. J. Turian, L. Ratinov, and Y. Bengio. 2010. Word representations: A simple and general method for semisupervised learning. In Proceedings of Association for Computational Linguistics (ACL 2010).
22. P. D. Turney. 2012. Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research, 44:533-585.