Redundancy in perceptual and linguistic experience: Comparing feature-based and distributional models of semantic representation

Topics in Cognitive Science

Volumn 3, Issue 2, 2011, Pages 303-345

  Riordan, Brian ^a   Jones, Michael N ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

Child directed speech; Co occurrence models; Latent semantic analysis; Semantic categorization; Semantic modeling; Word learning

Indexed keywords

ALGORITHM; CHILD; CLUSTER ANALYSIS; CONCEPT FORMATION; HUMAN; LEARNING; LINGUISTICS; SEMANTICS; SPEECH; STATISTICAL DISTRIBUTION; THEORETICAL MODEL;

ALGORITHMS; CHILD; CLUSTER ANALYSIS; CONCEPT FORMATION; HUMANS; LEARNING; LINGUISTICS; MODELS, THEORETICAL; SEMANTICS; SPEECH; STATISTICAL DISTRIBUTIONS;

EID: 79961229829     PISSN: 17568757     EISSN: 17568765     Source Type: Journal    
DOI: 10.1111/j.1756-8765.2010.01111.x     Document Type: Article

References (89)

1. Andrews, M., Vigliocco, G., & Vinson, D. P. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116(3), 463-498.
2. Baayen, R. H. (2001). Word frequency distributions. Dordrecht, The Netherlands: Kluwer Academic Publishers.
3. Barsalou, L. (2003). Abstraction in perceptual symbol systems. Philosophical Transactions of the Royal Society of London: Series B, 358, 1177-1187.
4. Barsalou, L., Santos, A., Simmons, K., & Wilson, C. (2008). Language and simulation in conceptual processing. In M. de Vega, A. Glenberg, & A. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 245-283). New York: Oxford University Press.
5. Bloom, P. (2000). How children learn the meanings of words. Cambridge, MA: MIT Press.
6. Bullinaria, J. A., & Levy, J. P. (2007). Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39(3), 510-526.
7. Burgess, C. (1998). From simple associations to the building blocks of language: Modeling meaning in memory with the HAL model. Behavior Research Methods, Instruments, and Computers, 30(2), 188-198.
8. Burgess, C., & Lund, K. (1997). Representing abstract words and emotional connotation in a high-dimensional memory space. In M. G. Shafto & P. Langley (Eds.), Proceedings of the 19th annual conference of the cognitive science society (pp. 61-66). Mahwah, NJ: Erlbaum.
9. Church, K. W., & Hanks, P. (1990). Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1), 22-29.
10. Connell, L., & Ramscar, M. (2001). Using distributional measures to model typicality in categorization. In J. Moore & K. Stenning (Eds.), Proceedings of the 23rd annual conference the cognitive science society (pp. 226-231). Mahwah, NJ: Erlbaum.
11. Cree, G. S., & McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). Journal of Experimental Psychology: General, 132(2), 163-201.
12. Cree, G. S., McRae, K., & McNorgan, C. (1999). An attractor model of lexical conceptual processing: Simulating semantic priming. Cognitive Science: A Multidisciplinary Journal, 23(3), 371-414.
13. Daille, B. (1996). Study and implementation of combined techniques for automatic extraction of terminology. In J. Klavans & P. Resnik (Eds.), The balancing act: Combining symbolic and statistical approaches to language (pp. 49-66). Cambridge, MA: MIT Press.
14. De Deyne, S., & Storms, G. (2008). Word associations: Network and semantic properties. Behavior Research Methods, 40, 213-231.
15. Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29(2), 145-193.
16. Dennis, S. (2007). How to use the LSA web site. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 57-70). Mahwah, NJ: Erlbaum.
17. Dunning, T. (1993). Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1), 61-74.
18. Durda, K., & Buchanan, L. (2008). WINDSORS: Windsor improved norms of distance and similarity of representations of semantics. Behavior Research Methods, 40(3), 705-712.
19. Durda, K., Buchanan, L., & Caron, R. (2009). Grounding co-occurrence: Identifying features in a lexical co-occurrence model of semantic memory. Behavior Research Methods, 41, 1210-1223.
20. Fellbaum, C. (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.
21. Fenson, L., Dale, P., Reznick, S., Bates, E., Thal, D., Pethick, S., Tomasello, M., Mervis, C. B. & Stiles, J. (1994). Variability in early communicative development. Monographs of the Society for Research in Child Development, 59(5), i185.
22. Firth, J. R. (1957). A synopsis of linguistic theory, 1930-1955. In Philological Society (Great Britain) (Ed.), Studies in linguistic analysis (pp. 1-32). Oxford, England: Blackwell.
23. Gentner, D. (2006). Why verbs are hard to learn. In K. Hirsh-Pasek & R. Golinkoff (Eds.), Action meets word: How children learn verbs (pp. 544-564). New York: Oxford University Press.
24. Gillette, J., Gleitman, H., Gleitman, L., & Lederer, A. (1999). Human simulations of vocabulary learning. Cognition, 73(2), 135-176.
25. Glenberg, A., & Robertson, D. (2000). Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 43(3), 379-401.
26. Griffiths, T. L., Steyvers, M., Blei, D. M., & Tenenbaum, J. B. (2005). Integrating topics and syntax. Advances in Neural Information Processing Systems, 17, 537-544.
27. Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114(2), 211-244.
28. Harris, Z. (1970). Distributional structure. In Papers in structural and transformational linguistics (pp. 775-794). Dordrecht, The Netherlands: D. Reidel Publishing Company.
29. Howell, S., Jankowicz, D., & Becker, S. (2005). A model of grounded language acquisition: Sensorimotor features improve lexical and grammatical learning. Journal of Memory and Language, 53(2), 258-276.
30. Hummell, J.E., & Holyoak, K.J. (2003). A symbolic-connectionist theory of relational inference and generalization. Psychological Review, 110, 220-264.
31. Jankowicz, D. (2005). Modeling category-specific deficits using topographic, corpus-derived representations. Unpublished PhD Thesis, McMaster University.
32. Johns, B. T., & Jones, M. N. (2009). Simulating false recall as an integration of semantic search and recognition. In N. Taatgen, H. van Rijn, L. Schomaker, & J. Nerbonne (Eds.), Proceedings of the 31st annual conference of the cognitive science society (pp. 2511-2516). Austin, TX: Cognitive Science Society.
33. Jones, M. N., Kintsch, W., & Mewhort, D. J. K. (2006). High-dimensional semantic space accounts of priming. Journal of Memory and Language, 55(4), 534-552.
34. Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114(1), 1-37.
35. Jurafsky, D., & Martin, J. (2008). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition (2nd ed.). Engelwood Cliff, NJ: Prentice Hall.
36. Kanerva, P., Kristoferson, J., & Holst, A. (2000). Random indexing of text samples for latent semantic analysis. In L. R. Gleitman & A. K. Joshi (Eds.), Proceedings of the 22nd annual conference of the cognitive science society (pp. 1036). Mahwah, NJ: Erlbaum.
37. Karypis, G. (2003). CLUTO: A Clustering Toolkit. Available at:. Accessed April 15, 2008.
38. Kintsch, W. (2007). Meaning in context. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 89-105). Mahwah, NJ: Erlbaum.
39. Kintsch, W. (2008). Symbol systems and perceptual representations. In M. de Vega, A. Glenberg, & A. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 145-163). New York: Oxford University Press.
40. Laham, D. (2000). Automated content assessment of text using latent semantic analysis to simulate human cognition. Unpublished PhD Thesis, University of Colorado.
41. Landauer, T., & Dumais, S. (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211-240.
42. Lavelli, A., Sebastiani, F., & Zanoli, R. (2004). Distributional term representations: An experimental comparison. In D. Grossman (Ed.), Proceedings of the thirteenth international conference on information and knowledge management (CIKM) (pp. 615-624). New York: ACM Press.
43. Logan, G. D. (1988). Towards an instance theory of automization. Psychological Review, 95(4), 492-527.
44. Louwerse, M. M. (2007). Symbolic or embodied representations: A case for symbol interdependency. In T. K. Landauer, D. S. Macnamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 107-120). Mahwah, NJ: Erlbaum.
45. Louwerse, M. M. (2008). Embodied relations are encoded by language. Psychonomic Bulletin and Review, 15(4), 838-844.
46. Louwerse, M. M., Cai, Z., Hu, X., Ventura, M., & Jeuniaux, P. (2006). Cognitively inspired NLP-based knowledge representations: Further explorations of Latent Semantic Analysis. International Journal on Artificial Intelligence Tools, 15(6), 1021-1039.
47. Louwerse, M. M., & Jeuniaux, P. (2008). Language comprehension is both embodied and symbolic. In M. de Vega, A. Glenberg, & A. C. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 309-326). New York: Oxford University Press.
48. Louwerse, M. M., & Zwaan, R. A. (2009). Language encodes geographic information. Cognitive Science, 33(1), 51-73.
49. Lowe, W. (2000). Topographic maps of semantic space. Unpublished Ph.D. thesis, University of Edinburgh.
50. Lowe, W. (2001). Towards a theory of semantic space. In J. Moore & K. Stenning (Eds.), Proceedings of the 23rd conference of the cognitive science society (pp. 576-581). Mahwah, NJ: Erlbaum.
51. Lowe, W., & McDonald, S. (2000). The direct route: Mediated priming in semantic space. In L. Glietman & A. Joshi (Eds.), Proceedings of the 22nd conference of the cognitive science society (pp. 806-811). Mahwah, NJ: Erlbaum.
52. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavioral Research Methods, Instrumentation, and Computers, 28, 203-208.
53. MacWhinney, B. (2000). The CHILDES Project: Tools for analyzing talk (3rd ed.). Mahwah, NJ: Erlbaum.
54. Maki, W. S., & Buchanan, E. (2008). Latent structure in measures of associative, semantic, and thematic knowledge. Behavior Research Methods, 15(3), 598-603.
55. Maki, W. S., McKinley, L. N., & Thompson, A. G. (2004). Semantic distance norms computed from an electronic dictionary (WordNet). Behavior Research Methods, Instruments, & Computers, 36(3), 421-431.
56. Martin, D., & Berry, M. (2007). Mathematical foundations behind latent semantic analysis. In T. K. Landauer, D. S. Macnamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 35-55). Mahwah, NJ: Erlbaum.
57. McDonald, S. (2000). Environmental determinants of lexical processing effort. Unpublished PhD Thesis, University of Edinburgh.
58. McDonald, S., & Lowe, W. (1998). Modelling functional priming and the associative boost. In S. J. Derry & M. A. Gernsbacher (Eds.), Proceedings of the 20th annual conference of the cognitive science society (pp. 675-680). Mahwah, NJ: Erlbaum.
59. McRae, K., Cree, G. S., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37(4), 547-559.
60. McRae, K., Cree, G. S., Westmacott, R., & de Sa, V. R. (1999). Further evidence for feature correlations in semantic memory. Canadian Journal of Experimental Psychology, 53(4), 360-373.
61. McRae, K., de Sa, V. R., & Seidenberg, M. S. (1997). On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology: General, 126(2), 99-130.
62. Miller, G. A. (1998). Nouns in WordNet. In C. Fellbaum (Ed.), WordNet: An electronic lexical database (pp. 23-46). Cambridge, MA: MIT Press.
63. Miller, G. A., & Charles, W. G. (1991). Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1), 1-28.
64. Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (2004). The University of South Florida free association, rhyme, and word fragment norms. Behavior Research Methods, Instruments, & Computers, 36(3), 402-407.
65. Padó, S., & Lapata, M. (2007). Dependency-based construction of semantic space models. Computational Linguistics, 33(2), 161-199.
66. Perfetti, C. (1998). The limits of co-occurrence: Tools and theories in language research. Discourse Processes, 25, 363-377.
67. Porter, M., & Boulton, R. (2006). Snowball stemmer [computer software]. Available at: .
68. Pulvermüller, F. (2008). Grounding language in the brain. In M. de Vega, A. Glenberg, & A. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 85-116). New York: Oxford University Press.
69. Quesada, J. (2007). Creating your own LSA spaces. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 71-85). Mahwah, NJ: Erlbaum.
70. Rao, V. A., & Howard, M. W. (2008). Retrieved context and the discovery of semantic structure. In J. C. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), Advances in neural information processing systems 20. Cambridge, MA: MIT Press.
71. Rasmussen, M., Newman, M., & Karypis, G. (2003). gCLUTO Documentation, Version 1.2. Minneapolis, MN: University of Minnesota.
72. Recchia, G., & Jones, M. N. (2009). More data trumps smarter algorithms: Comparing pointwise mutual information to latent semantic analysis. Behavior Research Methods, 41, 657-663.
73. Riordan, B., & Jones, M. N. (2007). Comparing semantic space models using child-directed speech. In D. S. McNamara & J. G. Trafton (Eds.), Proceedings of the 29th conference of the Cognitive Science Society (pp. 599-604). Austin, TX: Cognitive Science Society.
74. Sahlgren, M. (2006). The word-space model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector Spaces. Unpublished PhD Thesis, Stockholm University.
75. Sahlgren, M., Holst, A., & Kanerva, P. (2008). Permutations as a means to encode order in word space. In V. Sloutsky, B. Love, & K. McRae (Eds.), Proceedings of the 30th annual conference of the Cognitive Science Society (pp. 1300-1305). Austin, TX: Cognitive Science Society.
76. Sanford, A. J. (2006). Semantics in psychology. In K. Brown (Ed.), Encyclopedia of language and linguistics (2nd ed. Vol. 11, pp. 152-158). Amsterdam: Elsevier.
77. Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. In D. B. Jones & H. Somers (Eds.), Proceedings of the international conference on new methods in language processing (pp. 44-49). London: Routledge.
78. Shaoul, C., & Westbury, C. (2006). Word frequency effects in high-dimensional co-occurrence models: A new approach. Behavior Research Methods, 38(2), 190-195.
79. Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory: A featural model for semantic decisions. Psychological Review, 81, 214-241.
80. Steyvers, M., & Griffiths, T. L. (2007). Probabilistic topic models. In T. K. Landauer, D. S. McNamara, S. Dennis, & W. Kintsch (Eds.), Handbook of latent semantic analysis (pp. 427-448). Mahwah, NJ: Erlbaum.
81. de Vega, M., Graesser, A., & Glenberg, A. (2008). Reflecting on the debate. In M. de Vega, A. Glenberg, & A. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 397-440). New York: Oxford University Press.
82. Vigliocco, G., Vinson, D. P., Damian, M. F., & Levelt, W. (2002). Semantic distance effects on object and action naming. Cognition, 85(3), B61-B69.
83. Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. (2004). Representing the meanings of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology, 48(4), 422-488.
84. Vigliocco, G., Vinson, D. P., & Siri, S. (2005). Semantic and grammatical class effects in naming actions. Cognition, 94, B91-100.
85. Vinson, D. P., & Vigliocco, G. (2008). Semantic feature production norms for a large set of objects and events. Behavior Research Methods, 40(1), 183-190.
86. Vinson, D. P., Vigliocco, G., Cappa, S., & Siri, S. (2003). The breakdown of semantic knowledge: Insights from a statistical model of meaning representation. Understanding Language, 86(3), 347-365.
87. Zeno, S., Ivens, S., Millard, R., & Duvvuri, R. (Eds.) (1995). The Educator's Word Frequency Guide. Brewster, NY: Touchstone Applied Science Associates.
88. Zhao, Y., & Karypis, G. (2001). Criterion functions for document clustering: Experiments and analysis (University of Minnesota Computer Science Technical Report CS 01-40). Minneapolis, MN: University of Minnesota.
89. Zhao, Y., & Karypis, G. (2002). Evaluation of hierarchical clustering algorithms for document datasets. In C. Nicholas (Ed.), Proceedings of the eleventh international conference of information and knowledge management (CIKM) (pp. 515-524). MacLean, VA: ACM Press.