Learning effective and interpretable semantic models using non-negative sparse embedding

24th International Conference on Computational Linguistics - Proceedings of COLING 2012: Technical Papers

Volumn , Issue , 2012, Pages 1933-1950

  Murphy, Brian ^a   Talukdar, Partha Pratim ^a   Mitchell, Tom ^a  

^a Carnegie Mellon University   (United States)

Author keywords

Distributional semantics; Interpretability; Neuro semantics; Sparse coding; Vector space models; Word embeddings

Indexed keywords

DISTRIBUTIONAL SEMANTICS; EMBEDDINGS; INTERPRETABILITY; NEURO-SEMANTICS; SPARSE CODING; VECTOR-SPACE MODELS;

COMPUTATIONAL LINGUISTICS;

SEMANTICS;

EID: 84876796142     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (50)

1. Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., and Pas, M. (2009). A study on similarity and relatedness using distributional and WordNet-based approaches. Proceedings of NAACL-HLT 2009.
2. Almuhareb, A. and Poesio, M. (2004). Attribute-based and value-based clustering: An evaluation. In Proceedings of EMNLP, pages 158-165.
3. Baroni, M. and Lenci, A. (2010). Distributional Memory: A General Framework for Corpus-Based Semantics. Computational Linguistics, 36(4):673-721.
4. Baroni, M., Murphy, B., Barbu, E., and Poesio, M. (2010). Strudel: A corpus-based semantic model based on properties and types. Cognitive Science, 34(2):222-254.
5. Battig, W. F. and Montague, W. E. (1969). Category Norms for Verbal Items in 56 Categories: A Replication and Extension of the Connecticut Category Norms. Journal of Experimental Psychology Monographs, 80(3):1-46.
6. Blei, D. M., Ng, A. Y., and Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3(4-5):993-1022.
7. Boyd-Graber, J., Chang, J., Gerrish, S., Wang, C., and Blei, D. (2009). Reading tea leaves: How humans interpret topic models. In Proceedings of the 23rd Annual Conference on Neural Information Processing Systems.
8. Bradford, R. B. (2008). An empirical study of required dimensionality for large-scale latent semantic indexing applications. Proceedings of the 17th ACM, CIKM, pages 153-162.
9. Bullinaria, J. A. and Levy, J. P. (2007). Extracting semantic representations from word cooccurrence statistics: A computational study. Behavior Research Methods, 39(3):510-526.
10. Callan, J. and Hoy, M. (2009). The ClueWeb09 Dataset.
11. Chen, S., Donoho, D., and Saunders, M. (2001). Atomic decomposition by basis pursuit. SIAM review, pages 129-159.
12. Collobert, R. and Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, pages 160-167. ACM.
13. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., and Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391-407.
14. Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., and Ruppin, E. (2002). Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1):116-131.
15. Garrard, P., Lambon Ralph, M. A., Hodges, J. R., and Patterson, K. (2001). Prototypicality, distinctiveness, and intercorrelation: Analyses of the semantic attributes of living and nonliving concepts. Cognitive Neuropsychology, 18(2):25-174.
16. Griffiths, T. L., Steyvers, M., and Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114(2):211-244.
17. Hall, J., Nilsson, J., Nivre, J., Eryigit, G., Megyesi, B., Nilsson, M., and Saers, M. (2007). Single Malt or Blended? A Study in Multilingual Parser Optimization. Proceedings of CoNLL Shared Task Session, pages 933-939.
18. Hastie, T., Tibshirani, R., and Friedman, J. (2011). The Elements of Statistical Learning, volume 18 of Springer Series in Statistics. Springer, 5th edition.
19. Hoffman, M. D., Blei, D. M., and Bach, F. (2010). Online Learning for Latent Dirichlet Allocation. In Lafferty, J., Williams, C. K. I., Shawe-Taylor, J., Zemel, R. S., and Culotta, A., editors, Proceedings of NIPS.
20. Hoyer, P. O. (2002). Non-negative sparse coding. In Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, pages 557-565. Ieee.
21. Huang, E., Socher, R., Manning, C., and Ng, A. (2012). Improving word representations via global context and multiple word prototypes. In Proceedings of ACL.
22. Jones, E., Oliphant, T., Peterson, P., and Et Al. (2001). SciPy: Open source scientific tools for Python.
23. Karypis, G. (2003). CLUTO: A Clustering Toolkit. Technical Report 02-017, Department of Computer Science, University of Minnesota.
24. Landauer, T. and Dumais, S. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211-240.
25. Lehoucq, R. B., Sorensen, D. C., and Yang, C. (1998). Arpack users' guide: Solution of large scale eigenvalue problems with implicitly restarted Arnoldi methods. SIAM.
26. Levy, J. P. and Bullinaria, J. A. (2012). Using Enriched Semantic Representations in Predictions of Human Brain Activity. Proceedings of the 12th Neural Computation and Psychology Workshop, pages 292-308.
27. Lin, D. (1998). Automatic Retrieval and Clustering of Similar Words. In Proceedings of COLINGACL, pages 768-774.
28. Lund, K. and Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, and Computers, 28:203-208.
29. Lund, K., Burgess, C., and Atchley, R. (1995). Semantic and associative priming in high dimensional semantic space. In Proceedings of the 17th Cognitive Science Society Meeting, pages 660-665.
30. Mairal, J., Bach, F., Ponce, J., and Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. The Journal of Machine Learning Research, 11:19-60.
31. McRae, K., Cree, G. S., Seidenberg, M. S., and McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, Instruments, & Computers, 37(4):547-559.
32. Miller, G. A., Beckwith, R., Fellbaum, C., Gross, D., and Miller, K. (1990). Introduction to WordNet: An on-line lexical database. International Journal of Lexicography, 3(4):235-244.
33. Mitchell, T. M., Shinkareva, S. V., Carlson, A., Chang, K.-M., Malave, V. L., Mason, R. A., and Just, M. A. (2008). Predicting Human Brain Activity Associated with the Meanings of Nouns. Science, 320:1191-1195.
34. Murphy, B., Korhonen, A., and Chang, K. K.-M., editors (2010). Proceedings of the 1st Workshop on Computational Neurolinguistics, NAACL-HLT, Los Angeles. ACL.
35. Murphy, B., Talukdar, P., and Mitchell, T. (2012). Selecting Corpus-Semantic Models for Neurolinguistic Decoding. In Proceedings of *SEM-2012.
36. Murphy, G. (2004). The big book of concepts. MIT Press.
37. Suderman.
38. Nation, P. and Waring, R. (1997). Vocabulary size, text coverage and word lists. In Schmitt, N. and McCarthy, M., editors, Vocabulary Description acquisition and pedagogy, pages 6-19. Cambridge University Press.
39. Padó, S. and Lapata, M. (2007). Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161-199.
40. Palatucci, M., Hinton, G., Pomerleau, D., and Mitchell, T. M. (2009). Zero-Shot Learning with Semantic Output Codes. Advances in Neural Information Processing Systems, 22:1-9.
41. Rapp, R. (2003). Word Sense Discovery Based on Sense Descriptor Dissimilarity. Proceedings of the Ninth Machine Translation Summit, pp:315-322.
42. Ratinov, L., Turian, J., and Bengio, Y. (2010). Word representations: A simple and general method for semi-supervised learning. Proceedings of the 48th ACL, 96(July):384-394.
43. Ř ehuřek, R. and Sojka, P. (2010). Software Framework for Topic Modelling with Large Corpora. In Proceedings of New Challenges Workshop, LREC 2010, pages 45-50. ELRA.
44. Rubenstein, H. and Goodenough, J. B. (1965). Contextual correlates of synonymy. Communications of the ACM, 8(10):627-633.
45. Sahlgren, M. (2006). The Word-Space Model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. Dissertation, Stockholm University.
46. Schunn, C. D. (1999). The Presence and Absence of Category Knowledge in LSA. In Proceedings of the 21st Annual Conference of the Cognitive Science Society., Mahwah. Erlbaum.
47. Schütze, H. and Pedersen, J. (1993). A Vector Model for syntagmatic and paradigmatic relatedness. In Making Sense of Words Proceedings of the 9th Annual Conference of the University of Waterloo Centre for the New OED and Text Research, pages 104-113.
48. Socher, R., Huang, E., Pennington, J., Ng, A., and Manning, C. (2011). Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Advances in Neural Information Processing Systems, 24.
49. Turney, P. D. and Pantel, P. (2010). From Frequency to Meaning: Vector Space Models of Semantics. Artificial Intelligence, 37(1):141-188.
50. Vinson, D. and Vigliocco, G. (2008). Semantic feature production norms for a large set of objects and events. Behavior Research Methods, 40(1):183-190.