RC-NET: A general framework for incorporating knowledge into word representations

CIKM 2014 - Proceedings of the 2014 ACM International Conference on Information and Knowledge Management

Volumn , Issue , 2014, Pages 1219-1228

  Xu, Chang ^a   Bai, Yalong ^b   Bian, Jiang ^c   Gao, Bin ^c   Wang, Gang ^a   Liu, Xiaoguang ^a   Liu, Tie Yan ^c  

^a NANKAI UNIVERSITY   (China)

^b HARBIN INSTITUTE OF TECHNOLOGY   (China)

^c MICROSOFT RESEARCH   (United States)

Author keywords

Deep learning; Distributed word representations; Knowledge graph

Indexed keywords

BACKPROPAGATION; DEEP LEARNING; ENCODING (SYMBOLS); KNOWLEDGE MANAGEMENT; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS; TEXT MINING; VECTOR SPACES;

ANALOGICAL REASONING; BACK PROPAGATION NEURAL NETWORKS; COMBINED OPTIMIZATION PROBLEM; COMPLEX RELATIONSHIPS; KNOWLEDGE GRAPHS; NATURAL LANGUAGE PROCESSING; SEMANTIC INFORMATION; WORD REPRESENTATIONS;

KNOWLEDGE REPRESENTATION;

EID: 84937567033     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2661829.2662038     Document Type: Conference Paper

References (25)

1. Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin. A neural probabilistic language model. In The Journal of Machine Learning Research, pages 3:1137-1155, 2003.
2. J. Bian, B. Gao, and T.-Y. Liu. Knowledge-powered deep learning for word embedding. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2014.
3. K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250. ACM, 2008.
4. A. Bordes, X. Glorot, J. Weston, and Y. Bengio. A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233-259, 2014.
5. A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, pages 2787-2795, 2013.
6. A. Bordes, J. Weston, R. Collobert, Y. Bengio, et al. Learning structured embeddings of knowledge bases. In AAAI, 2011.
7. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160-167. ACM, 2008.
8. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493-2537, 2011.
9. L. Finkelstein, E. Gabrilovich, Y. Matias, E. Rivlin, Z. Solan, G. Wolfman, and E. Ruppin. Placing search in context: The concept revisited. In Proceedings of the 10th international conference on World Wide Web, pages 406-414. ACM, 2001.
10. Freebase. http://www.freebase.com.
11. B. Gao, J. Bian, and T.-Y. Liu. Wordrep: A benchmark for research on learning word representations. In ICML 2014 Workshop on Knowledge-Powered Deep Learning for Text Mining, 2014.
12. X. Glorot, A. Bordes, and Y. Bengio. Domain adaptation for large-scale sentiment classification: A deep learning approach. In In Proceedings of the Twenty-eight International Conference on Machine Learning, ICML, 2011.
13. M. U. Gutmann and A. Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. The Journal of Machine Learning Research, 13:307-361, 2012.
14. G. E. Hinton. Distributed representations. 1984.
15. R. Jenatton, N. Le Roux, A. Bordes, G. Obozinski, et al. A latent factor model for highly multi-relational data. In NIPS, pages 3176-3184, 2012.
16. M.-T. Luong, R. Socher, and C. D. Manning. Better word representations with recursive neural networks for morphology. CoNLL-2013, 104, 2013.
17. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
18. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111-3119, 2013.
19. A. Mnih and K. Kavukcuoglu. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in Neural Information Processing Systems, pages 2265-2273, 2013.
20. A. Mnih and Y. W. Teh. A fast and simple algorithm for training neural probabilistic language models. arXiv preprint arXiv:1206.6426, 2012.
21. R. Socher, D. Chen, C. D. Manning, and A. Ng. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems, pages 926-934, 2013.
22. G. Tur, L. Deng, D. Hakkani-Tur, and X. He. Towards deeper understanding: Deep convex networks for semantic utterance classification. In ICASSP, pages 5045-5048, 2012.
23. J. Weston, A. Bordes, O. Yakhnenko, and N. Usunier. Connecting language and knowledge bases with embedding models for relation extraction. arXiv preprint arXiv:1307.7973, 2013.
24. WordNet. "about wordnet", princeton university. http://wordnet.princeton.edu. 2010.
25. M. Yu and M. Dredze. Improving lexical embeddings with semantic knowledge. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 545-550, Baltimore, Maryland, June 2014. Association for Computational Linguistics.