Learning word representations with hierarchical sparse coding

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 87-96

  Yogatama, Dani ^a   Faruqui, Manaal ^a   Dyer, Chris ^a   Smith, Noah A ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CODES (SYMBOLS); DATA MINING; LEARNING SYSTEMS; SEMANTICS; STOCHASTIC SYSTEMS;

EFFICIENT LEARNING; PROXIMAL METHODS; SENTENCE COMPLETIONS; SENTIMENT ANALYSIS; SPARSE CODING; STATE-OF-THE-ART METHODS; WORD REPRESENTATIONS; WORD SIMILARITY;

LEARNING ALGORITHMS;

EID: 84947774702     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Bamman, David, Dyer, Chris, and Smith, Noah A. Distributed representations of situated language. In Proc. of ACL, 2014.
2. Bengio, Yoshua, Ducharme, Rejean, Vincent, Pascal, and Jauvin, Chrstian. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155, 2003.
3. Collins, Allan M. and Quillian, M. Ross. Retrieval time from semantic memory. Journal of Verbal Learning and Verbal Behaviour, 8:240-247, 1969.
4. Collobert, Ronan, Weston, Jason, Bottou, Leon, Karlen, Michael, Kavukcuoglu, Koray, and Kuska, Pavel. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2461-2505, 2011.
5. Duchi, John, Hazan, Elad, and Singer, Yoram. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12: 2121-2159, 2011.
6. Faruqui, Manaal and Dyer, Chris. Improving vector space word representations using multilingual correlation. In Proc. ofEACL, 2014.
7. Finkelstein, Lev, Gabrilovich, Evgeniy, Matias, Yossi, Rivlin, Ehud, Solan, Zach, Wolfman, Gadi, and Ruppin, Eytan. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1): 116-131,2002.
8. Fyshe, Alona, Talukdar, Partha P., Murphy, Brian, and Mitchell, Tom M. Interpretable semantic vectors from a joint model of brain-and text-based meaning. In Proc. of ACL, 2014.
9. Gutmann, Michael and Hyvarinen, Aapo. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proc. of AISTATS, 2010.
10. Huang, Eric H., Socher, Richard, Manning, Christopher D., and Ng, Andew Y. Improving word representations via global context and multiple word prototypes. In Proc. of ACL, 2012.
11. Jenatton, Rodolphe, Mairal, Julien, Obozinski, Gullaume, and Bach, Francis. Proximal methods for hierarchical sparse coding. Journal of Machine Learning Research, 12:2297-2334, 2011.
12. Lebret, Remi and Collobert, Ronan. Word embeddings through hellinger PC A. In Proc. ofEACL, 2014.
13. Lee, Honglak, Battle, Alexis, Raina, Rajat, and Ng, Andrew Y. Efficient sparse coding algorithms. In Proc. of NIPS, 2007.
14. Lee, Honglak, Raina, Rajat, Teichman, Alex, and Ng, Andrew Y. Exponential family sparse coding with application to self-taught learning. In Proc. of IJCAI, 2009.
15. Levy, Omar and Goldberg, Yoav. Neural word embeddings as implicit matrix factorization. In Proc. of NIPS, 2014.
16. Luong, Minh-Thang, Socher, Richard, and Manning, Christopher D. Better word representations with recursive neural networks for morphology. In Proc. ofCONLL, 2013.
17. Mairal, Julien, Bach, Francis, Ponce, Jean, and Sapiro, Guillermo. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:19-60, 2010.
18. Mikolov, Tomas, Martin, Karafiat, Burget, Lukas, Cernocky, Jan, and Khudanpur, Sanjeev. Recurrent neural network based language model. In Proc. of Interspeech, 2010.
19. Mikolov, Tomas, Chen, Kai, Corrado, Greg, and Dean, Jeffrey. Efficient estimation of word representations in vector space. In Proc. oflCLR Workshop, 2013a.
20. Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg, and Dean, Jeffrey. Distributed representations of words and phrases and their compositionality. In Proc. of NIPS, 2013b.
21. Miller, George A. Wordnet: A lexical database for english. Communications of the ACM, 38(11):39-41, 1995.
22. Mnih, Andriy and Hinton, Geoffrey. A scalable hierarchical distributed language model. In Proc. of NIPS, 2008.
23. Mnih, Andriy and Teh, Yee Whye. A fast and simple algorithm for training neural probabilistic language models. In Proc. oflCML, 2012.
24. Murphy, Brian, Talukdar, Partha, and Mitchell, Tom. Learning effective and interpretable semantic models using non-negative sparse embedding. In Proc. of COLING, 2012.
25. Pennington, Jeffrey, Socher, Richard, and Manning, Christopher D. Glove: Global vectors for word representation. In Proc. ofEMNLP, 2014.
26. Petrov, S. and Klein, D. Sparse multi-scale grammars for discriminative latent variable parsing. In Proc. ofEMNLP, 2008.
27. Qin, Zhiwei (Tony) and Goldfarb, Donald. Structured spar-sity via alternating direction methods. Journal of Machine Learning Research, 13:1435-1468, 2012.
28. Raposo, Ana, Mendes, Mafalda, and Marques, J. Frederico. The hierarchical organization of semantic memory: Executive function in the processing of superordinate concepts. Neurolmage, 59:1870-1878, 2012.
29. Schutze, Hinrich. Automatic word sense discrimination. Computational Linguistics-Special issue on word sense disambiguation, 24(1):97-123, 1998.
30. Socher, Richard, Perelygin, Alex, Wu, Jean, Chuang, Jason, Manning, Chris, Ng, Andrew, and Potts, Chris. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. ofEMNLP, 2013.
31. Spearman, Charles. The proof and measurement of association between two things. The American Journal of Psychology, 15:72-101, 1904.
32. Sra, Suvrit. Scalable nonconvex inexact proximal splitting. In Proc. of NIPS, 2012.
33. Turian, Joseph, Ratinov, Lev, and Bengio, Yoshua. Word representations: A simple and general method for semi-supervised learning. In Proc. of ACL, 2010.
34. Turney, Peter D. and Pantel, Patrick. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141-188, 2010.
35. van der Maaten, Laurens and Hinton, Geoffrey. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579-2605, 2008.
36. Yogatama, Dani and Smith, Noah A. Making the most of bag of words: Sentence regularization with alternating direction method of multipliers. In Proc. of ICML, 2014.
37. Yuan, Ming and Lin, Yi. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68(l):49-67, 2006.
38. Zhao, Peng, Rocha, Gulherme, and Yu, Bin. The composite and absolute penalties for grouped and hierarchical variable selection. The Annals of Statistics, 37(6A):3468-3497, 2009.
39. Zweig, Geoffry and Burges, Christopher J. C. The microsoft research sentence completion challenge. Technical report, Microsoft Research Technical Report MSR-TR-2011-129, 2011.