Multimodal distributional semantics

Journal of Artificial Intelligence Research

Volumn 49, Issue , 2014, Pages 1-47

  Bruni, Elia ^a   Tran, Nam Khanh ^b   Baroni, Marco ^a  

^a UNIVERSITY OF TRENTO   (Italy)

^b L3S RESEARCH CENTER   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ELECTRONICS ENGINEERING;

COMPUTER VISION TECHNIQUES; DISTRIBUTIONAL INFORMATION; DISTRIBUTIONAL MODELS; DISTRIBUTIONAL SEMANTICS; FLEXIBLE ARCHITECTURES; INTEGRATED MODELING; SEMANTIC INFORMATION; SEMANTIC RELATEDNESS;

SEMANTICS;

EID: 84894469728     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.4135     Document Type: Article

References (114)

1. Abdi, H., & Williams, L. (2010). Newman-Keuls and Tukey test. In Salkind, N., Frey, B., & Dougherty, D. (Eds.), Encyclopedia of Research Design, pp. 897-904. Sage, Thousand Oaks, CA.
2. Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Pasça, M., & Soroa, A. (2009). A study on similarity and relatedness using distributional and Word Net-based approaches. In Proceedings of HLT-NAACL, pp. 19-27, Boulder, CO.
3. Almuhareb, A., & Poesio, M. (2005). Concept learning and categorization from the web. In Proceedings of Cog Sci, pp. 103-108, Stresa, Italy.
4. Andrews, M., Vigliocco, G., & Vinson, D. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116 (3), 463-498.
5. Baayen, H. (2008). Analyzing Linguistic Data: A Practical Introduction to Statistics using R. Cambridge University Press, Cambridge, UK.
6. Barnard, K., Duygulu, P., Forsyth, D., de Freitas, N., Blei, D., & Jordan, M. (2003). Matching words and pictures. Journal of Machine Learning Research, 3, 1107-1135.
7. Baroni, M., Barbu, E., Murphy, B., & Poesio, M. (2010). Strudel: A distributional semantic model based on properties and types. Cognitive Science, 34 (2), 222-254.
8. Baroni, M., & Lenci, A. (2008). Concepts and properties in word spaces. Italian Journal of Linguistics, 20 (1), 55-88.
9. Baroni, M., & Lenci, A. (2010). Distributional Memory: A general framework for corpusbased semantics. Computational Linguistics, 36 (4), 673-721.
10. Baroni, M., & Lenci, A. (2011). How we BLESSed distributional semantic evaluation. In Proceedings of the EMNLP GEMS Workshop, pp. 1-10, Edinburgh, UK.
11. Barsalou, L. (2008). Grounded cognition. Annual Review of Psychology, 59, 617-645.
12. Berg, T., Berg, A., & Shih, J. (2010). Automatic attribute discovery and characterization from noisy Web data. In ECCV, pp. 663-676, Crete, Greece.
13. Bergsma, S., & Goebel, R. (2011). Using visual information to predict lexical preference. In Proceedings of RANLP, pp. 399-405, Hissar, Bulgaria.
14. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993-1022.
15. Bosch, A., Zisserman, A., & Munoz, X. (2007). Image classification using random forests and ferns. In Proceedings of ICCV, pp. 1-8, Rio de Janeiro, Brazil.
16. Bosch, A., Zisserman, A., & Munoz, X. (2008). Scene classification using a hybrid generative/discriminative approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30 (4).
17. Bruni, E., Boleda, G., Baroni, M., & Tran, N. K. (2012). Distributional semantics in Technicolor. In Proceedings of ACL, pp. 136-145, Jeju Island, Korea.
18. Bruni, E., Bordignon, U., Liska, A., Uijlings, J., & Sergienya, I. (2013). Vsem: An open library for visual semantics representation. In Proceedings of ACL, Sofia, Bulgaria.
19. Bruni, E., Tran, G. B., & Baroni, M. (2011). Distributional semantics from text and images. In Proceedings of the EMNLP GEMS Workshop, pp. 22-32, Edinburgh, UK.
20. Bruni, E., Uijlings, J., Baroni, M., & Sebe, N. (2012). Distributional semantics with eyes: Using image analysis to improve computational representations of word meaning. In Proceedings of ACM Multimedia, pp. 1219-1228, Nara, Japan.
21. Budanitsky, A., & Hirst, G. (2006). Evaluating Word Net-based measures of lexical semantic relatedness. Computational Linguistics, 32 (1), 13-47. (Pubitemid 43764024)
22. Bullinaria, J., & Levy, J. (2007). Extracting semantic representations from word cooccurrence statistics: A computational study. Behavior Research Methods, 39, 510-526.
23. Bullinaria, J., & Levy, J. (2012). Extracting semantic representations from word cooccurrence statistics: Stop-lists, stemming and SVD. Behavior Research Methods, 44, 890-907.
24. Burgess, C. (2000). Theory and operational definitions in computational memory models: A response to Glenberg and Robertson. Journal of Memory and Language, 43 (3), 402-408.
25. Caicedo, J., Ben-Abdallah, J., González, F., & Nasraoui, O. (2012). Multimodal representation, indexing, automated annotation and retrieval of image collections via nonnegative matrix factorization. Neurocomputing, 76 (1), 50-60.
26. Chatfield, K., Lempitsky, V., Vedaldi, A., & Zisserman, A. (2011). The devil is in the details: an evaluation of recent feature encoding methods. In Proceedings of BMVC, Dundee, UK.
27. Church, K., & Hanks, P. (1990). Word association norms, mutual information, and lexicography. Computational Linguistics, 16 (1), 22-29.
28. Clark, S. (2013). Vector space models of lexical meaning. In Lappin, S., & Fox, C. (Eds.), Handbook of Contemporary Semantics, 2nd ed. Blackwell, Malden, MA. In press.
29. Coltheart, M. (1981). The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology, 33.
30. Connolly, A., Gleitman, L., & Thompson-Schill, S. (2007). Effect of congenital blindness on the semantic representation of some everyday concepts. Proceedings of the National Academy of Sciences, 104 (20), 8241-8246. (Pubitemid 47175472)
31. Csurka, G., Dance, C., Fan, L., Willamowski, J., & Bray, C. (2004). Visual categorization with bags of keypoints. In In Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1-22, Prague, Czech Republic.
32. Curran, J., & Moens, M. (2002). Improvements in automatic thesaurus extraction. In Proceedings of the ACL Workshop on Unsupervised Lexical Acquisition, pp. 59-66, Philadelphia, PA.
33. de Sande, K. V., Uijlings, J., Gevers, T., & Smeulders, A. (2011). Segmentation as selective search for object recognition. In Proceedings of ICCV, pp. 1879-1886, Barcelona, Spain.
34. de Vega, M., Glenberg, A., & Graesser, A. (Eds.). (2008). Symbols and Embodiment: Debates on Meaning and Cognition. Oxford University Press, Oxford, UK.
35. Deng, J., Dong, W., Socher, R., Li, L.-J., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In Proceedings of CVPR, pp. 248-255, Miami Beach, FL.
36. Dumais, S. (2003). Data-driven approaches to information access. Cognitive Science, 27, 491-524. (Pubitemid 36598418)
37. Erk, K. (2012). Vector space models of word meaning and phrase meaning: A survey. Language and Linguistics Compass, 6 (10), 635-653.
38. Escalante, H. J., Hérnadez, C. A., Sucar, L. E., & Montes, M. (2008). Late fusion of heterogeneous methods for multimedia image retrieval. In Proceedings of ICMR, Vancouver, Canada.
39. Evert, S. (2005). The Statistics of Word Cooccurrences. Dissertation, Stuttgart University.
40. Farhadi, A., Hejrati, M., Sadeghi, M. A., Young, P., Rashtchian, C., Hockenmaier, J., & Forsyth, D. (2010). Every picture tells a story: Generating sentences from images. In Proceedings of ECCV, Crete, Greece.
41. Felzenszwalb, P., Girshick, R., McAllester, D., & Deva Ramanan, D. (2010). Object detection with discriminatively trained part based models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32, 1627-1645.
42. Feng, Y., & Lapata, M. (2010). Visual information in semantic representation. In Proceedings of HLT-NAACL, pp. 91-99, Los Angeles, CA.
43. Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2002). Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20 (1), 116-131. (Pubitemid 44642309)
44. Firth, J. R. (1957). Papers in Linguistics, 1934-1951. Oxford University Press, Oxford, UK.
45. Fodor, J. (1975). The Language of Thought. Crowell Press, New York.
46. Glenberg, A., & Robertson, D. (2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 3 (43), 379-401.
47. Grauman, K., & Darrell, T. (2005). The pyramid match kernel: Discriminative classification with sets of image features. In Proceedings of ICCV, pp. 1458-1465, Beijing, China. (Pubitemid 44042350)
48. Grauman, K., & Leibe, B. (2011). Visual Object Recognition. Morgan & Claypool, San Francisco.
49. Grefenstette, G. (1994). Explorations in Automatic Thesaurus Discovery. Kluwer, Boston, MA.
50. Griffin, L., Wahab, H., & Newell, A. (2013). Distributional learning of appearance. PLoS ONE, 8 (2). Published online: http://www.plosone.org/ article/info:doi/10.1371/journal.pone.0058074.
51. Griffiths, T., Steyvers, M., & Tenenbaum, J. (2007). Topics in semantic representation. Psychological Review, 114, 211-244.
52. Hansen, T., Olkkonen, M., Walter, S., & Gegenfurtner, K. (2006). Memory modulates color appearance. Nature Neuroscience, 9, 1367-1368. (Pubitemid 44646207)
53. Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42 (1-3), 335-346.
54. Harris, Z. (1954). Distributional structure. Word, 10 (2-3), 1456-1162.
55. Johns, B., & Jones, M. (2012). Perceptual inference through global lexical similarity. Topics in Cognitive Science, 4 (1), 103-120.
56. Karypis, G. (2003). CLUTO: A clustering toolkit. Tech. rep. 02-017, University of Minnesota Department of Computer Science.
57. Kaschak, M., Madden, C., Therriault, D., Yaxley, R., Aveyard, M., Blanchard, A., & Zwaan, R. (2005). Perception of motion affects language processing. Cognition, 94, B79-B89. (Pubitemid 40247429)
58. Kievit-Kylar, B., & Jones, M. (2011). The Semantic Pictionary project. In Proceedings of CogSci, pp. 2229-2234, Austin, TX.
59. Kulkarni, G., Premraj, V., Dhar, S., Li, S., Choi, Y., Berg, A. C., & Berg, T. L. (2011). Baby talk: Understanding and generating simple image descriptions. In Proceedings of CVPR, Colorado Springs, MSA.
60. Landauer, T., & Dumais, S. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104 (2), 211-240.
61. Lazebnik, S., Schmid, C., & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In Proceedings of CVPR, pp. 2169-2178, Washington, DC. (Pubitemid 44931582)
62. Leong, C. W., & Mihalcea, R. (2011). Going beyond text: A hybrid image-text approach for measuring word relatedness. In Proceedings of IJCNLP, pp. 1403-1407.
63. Lloyd, S. (1982). Least squares quantization in PCM. IEEE Transactions on Information Theory, 28, 129-137.
64. Louwerse, M. (2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3, 273-302.
65. Louwerse, M., & Connell, L. (2011). A taste of words: Linguistic context and perceptual simulation predict the modality of words. Cognitive Science, 35, 381-398.
66. Lowe, D. (1999). Object recognition from local scale-invariant features. In Proceedings of ICCV, pp. 1150-1157. (Pubitemid 32079997)
67. Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60 (2).
68. Lowe, W. (2001). Towards a theory of semantic space. In Proceedings of CogSci, pp. 576-581, Edinburgh, UK.
69. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, 28, 203-208. (Pubitemid 126454814)
70. Manning, C., Raghavan, P., & Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK.
71. Manning, C., & Schütze, H. (1999). Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA.
72. McDonald, S., & Brew, C. (2004). A distributional model of semantic context effects in lexical processing. In Proceedings of ACL, pp. 17-24, Barcelona, Spain.
73. McRae, K., Cree, G., Seidenberg, M., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37 (4), 547-559. (Pubitemid 43679323)
74. Miller, G., & Charles, W. (1991). Contextual correlates of semantic similarity. Language and Cognitive Processes, 6 (1), 1-28.
75. Moore, D., & McCabe, G. (2005). Introduction to the Practice of Statistics (5 edition). Freeman, New York.
76. Murphy, G. (2002). The Big Book of Concepts. MIT Press, Cambridge, MA.
77. Nelson, D., McEvoy, C., & Schreiber, T. (1998). The University of South Florida word association, rhyme, and word fragment norms. http://www.usf.edu/FreeAssociation/.
78. Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Volume 2, CVPR '06, pp. 2161-2168. (Pubitemid 44931581)
79. Nowak, E., Jurie, F., & Triggs, B. (2006). Sampling strategies for bag-of-features image classification. In Proceedings of ECCV, pp. 490-503, Graz, Austria. (Pubitemid 44029653)
80. Padó, S., & Lapata, M. (2007). Dependency-based construction of semantic space models. Computational Linguistics, 33 (2), 161-199.
81. Padó, U., Padó, S., & Erk, K. (2007). Flexible, corpus-based modelling of human plausibility judgements. In Proceedings of EMNLP, pp. 400-409, Prague, Czech Republic.
82. Pecher, D., Zeelenberg, R., & Raaijmakers, J. (1998). Does pizza prime coin? Perceptual priming in lexical decision and pronunciation. Journal of Memory and Language, 38, 401-418. (Pubitemid 128348606)
83. Perronnin, F., Sanchez, J., & Mensink, T. (2010). Improving the fisher kernel for large-scale image classification. In Proceedings of ECCV, pp. 143-156, Berlin, Heidelberg.
84. Pham, T.-T., Maillot, N., Lim, J.-H., & Chevallet, J.-P. (2007). Latent semantic fusion model for image retrieval and annotation. In Proceedings of CIKM, pp. 439-443, Lisboa, Portugal.
85. Poesio, M., & Almuhareb, A. (2005). Identifying concept attributes using a classifier. In Proceedings of the ACL Workshop on Deep Lexical Semantics, pp. 18-27, Ann Arbor, MI.
86. Pulvermueller, F. (2005). Brain mechanisms linking language and action. Nature Reviews Neuroscience, 6, 576-582.
87. Radinsky, K., Agichtein, E., Gabrilovich, E., & Markovitch, S. (2011). A word at a time: computing word relatedness using temporal semantic analysis. In Proceedings of WWW, pp. 337-346, Hyderabad, India.
88. Rapp, R. (2003). Word sense discovery based on sense descriptor dissimilarity. In Proceedings of the 9th MT Summit, pp. 315-322, New Orleans, LA.
89. Recchia, G., & Jones, M. (2012). The semantic richness of abstract concepts. Frontiers in Human Neuroscience, 6 (315).
90. Reisinger, J., & Mooney, R. J. (2010). Multi-prototype vector-space models of word meaning. In Proceedings of NAACL, pp. 109-117, Los Angeles, CA.
91. Riordan, B., & Jones, M. (2011). Redundancy in perceptual and linguistic experience: Comparing feature-based and distributional models of semantic representation. Topics in Cognitive Science, 3 (2), 1-43.
92. Rothenhäusler, K., & Schütze, H. (2009). Unsupervised classification with dependency based word spaces. In Proceedings of the EACL GEMS Workshop, pp. 17-24, Athens, Greece.
93. Rubenstein, H., & Goodenough, J. (1965). Contextual correlates of synonymy. Communications of the ACM, 8 (10), 627-633.
94. Sahlgren, M. (2005). An introduction to random indexing. http://www.sics.se/~mange/papers/RI-intro.pdf.
95. Sahlgren, M. (2006). The Word-Space Model. Dissertation, Stockholm University.
96. Sahlgren, M. (2008). The distributional hypothesis. Italian Journal of Linguistics, 20 (1), 33-53.
97. Schütze, H. (1997). Ambiguity Resolution in Natural Language Learning. CSLI, Stanford, CA.
98. Silberer, C., & Lapata, M. (2012). Grounded models of semantic representation. In Proceedings of EMNLP-CoNLL, pp. 1423-1433, Jeju, Korea.
99. Sivic, J., & Zisserman, A. (2003). Video Google: A text retrieval approach to object matching in videos. In Proceedings of ICCV, pp. 1470-1477, Nice, France.
100. Steyvers, M. (2010). Combining feature norms and text data with topic models. Acta Psychologica, 133 (3), 234-243.
101. Therriault, D., Yaxley, R., & Zwaan, R. (2009). The role of color diagnosticity in object recognition and representation. Cognitive Processing, 10 (4), 335-342.
102. Tillman, R., Datla, V., Hutchinson, S., & Louwerse, M. (2012). From head to toe: Embodiment through statistical linguistic frequencies. In Proceedings of CogSci, pp. 2434-2439, Austin, TX.
103. Turney, P., Neuman, Y., Assaf, D., & Cohen, Y. (2011). Literal and metaphorical sense identification through concrete and abstract context. In Proceedings of EMNLP, pp. 680-690, Edinburgh, UK.
104. Turney, P., & Pantel, P. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37, 141-188.
105. Van de Sande, K., Gevers, T., & Snoek, C. (2010). Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32 (9), 1582-1596.
106. Van Overschelde, J., Rawson, K., & Dunlosky, J. (2004). Category norms: An updated and expanded version of the Battig and Montague (1969) norms. Journal of Memory and Language, 50, 289-335. (Pubitemid 38457687)
107. Vedaldi, A., & Fulkerson, B. (2010). Vlfeat-an open and portable library of computer vision algorithms. In Proceedings of ACM Multimedia, pp. 1469-1472, Firenze, Italy.
108. Von Ahn, L. (2006). Games with a purpose. Computer, 29 (6), 92-94.
109. Vreeswijk, D. T., Huurnink, B., & Smeulders, A. W. (2011). Text and image subject classifiers: dense works better. In Proceedings of ACM Multimedia, pp. 1449-1452, Scottsdale, AZ.
110. Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., & Gong, Y. (2010). Locality-constrained linear coding for image classification. In Proceedings of CVPR, pp. 3360-3367, San Francisco, CA.
111. Weeds, J. (2003). Measures and Applications of Lexical Distributional Similarity. Ph.D. thesis, Department of Informatics, University of Sussex.
112. Wittgenstein, L. (1953). Philosophical Investigations. Blackwell, Oxford, UK. Translated by G.E.M. Anscombe.
113. Yang, J., Jiang, Y.-G., Hauptmann, A., & Ngo, C.-W. (2007). Evaluating bag-of-visualwords representations in scene classification. In Wang, J. Z., Boujemaa, N., Bimbo, A. D., & Li, J. (Eds.), Multimedia Information Retrieval, pp. 197-206. ACM.
114. Zhao, Y., & Karypis, G. (2003). Criterion functions for document clustering: Experiments and analysis. Tech. rep. 01-40, University of Minnesota Department of Computer Science.