An amodal shared resource model of language-mediated visual attention

Frontiers in Psychology

Volumn 4, Issue AUG, 2013, Pages

  Smith, Alastair C ^a,b   Monaghan, Padraic ^c   Huettig, Falk ^a,d  

^a MAX PLANCK INSTITUTE FOR PSYCHOLINGUISTICS   (Netherlands)

^b International Max Planck Research School for Language Sciences   (Netherlands)

^c LANCASTER UNIVERSITY   (United Kingdom)

^d RADBOUD UNIVERSITY   (Netherlands)

Author keywords

Attention; Computational modeling; Eye movements; Language; Semantics; Vision

Indexed keywords

EID: 84885362119     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00528     Document Type: Article

References (60)

1. Allopenna, P. D., Magnuson, J. S., and Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. J. Mem. Lang. 38, 419-439. doi: 10.1006/jmla.1997.2558
2. Anderson, S. E., Chiu, E., Huette, S., and Spivey, M. J. (2011). On the temporal dynamics of language-mediated vision and vision-mediated language. Acta Psychol. 137, 181-189. doi: 10.1016/j.actpsy.2010.09.008
3. Bar, M. (2003). A cortical mechanism for triggering top-down facilitation in visual object recognition. J. Cogn. Neurosci. 15, 600-609. doi: 10.1162/089892903321662976
4. Barsalou, L. W., Kyle Simmons, W., Barbey, A. K., and Wilson, C. D. (2003). Grounding conceptual knowledge in modality-specific systems. Trends Cogn. Sci. 7, 84-91. doi: 10.1016/S1364-6613(02)00029-3
5. Bloom, P. (2000). How Children Learn the Meanings of Words. Cambridge; London: MIT Press.
6. Coltheart, M. (2004). Are there lexicons? Q. J. Exp. Psychol. Sec. A 57, 1153-1171.
7. Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: a new methodology for the real-time investigation of speech perception, memory, and language processing. Cogn. Psychol. 6, 84-107. doi: 10.1016/0010-0285(74)90005-X
8. Cree, G. S., and McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). J. Exp. Psychol. Gen. 132, 163-200. doi: 10.1037/0096-3445.132.2.163
9. Cree, G. S., McRae, K., and McNorgan, C. (1999). An attractor model of lexical conceptual processing: simulating semantic priming. Cogn. Sci. 23, 371-414. doi: 10.1207/s15516709cog2303_4
10. Dahan, D., and Tanenhaus, M. K. (2005). Looking at the rope when looking for the snake: conceptually mediated eye movements during spoken-word recognition. Psychon. Bull. Rev. 12, 453-459. doi: 10.3758/BF03193787
11. Desroches, A. S., Joanisse, M. F., and Robertson, E. K. (2006). Specific phonological impairments in dyslexia revealed by eyetracking. Cognition 100, B32-B42. doi: 10.1016/j.cognition.2005.09.001
12. Dilkina, K., McClelland, J. L., and Plaut, D. C. (2008). A single-system account of semantic and lexical deficits in five semantic dementia patients. Cogn. Neuropsychol. 25, 136-164. doi: 10.1080/02643290701723948
13. Dilkina, K., McClelland, J. L., and Plaut, D. C. (2010). Are there mental lexicons. The role of semantics in lexical decision. Brain Res. 1365, 66-81. doi: 10.1016/j.brainres.2010.09.057
14. Dove, G. (2009). Beyond perceptual symbols: a call for representational pluralism. Cognition 110, 412-431. doi: 10.1016/j.cognition.2008.11.016
15. Elman, J. L. (1990). Finding structure in time. Cogn. Sci. 14, 179-211. doi: 10.1207/s15516709cog1402_1
16. Goldstone, R. L., and Barsalou, L. W. (1998). Reuniting perception and conception. Cognition 65, 231-262. doi: 10.1016/S0010-0277(97)00047-4
17. Halberda, J. (2006). Is this a dax which I see before me. Use of the logical argument disjunctive syllogism supports word-learning in children and adults. Cogn. Psychol. 53, 310-344. doi: 10.1016/j.cogpsych.2006.04.003
18. Harm, M. W., and Seidenberg, M. S. (2004). Computing the meanings of words in reading: cooperative division of labor between visual and phonological processes. Psychol. Rev. 111, 662-720. doi: 10.1037/0033-295X.111.3.662
19. Huettig, F., and Altmann, G. T. (2005). Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm. Cognition 96, B23-B32. doi: 10.1016/j.cognition.2004.10.003
20. Huettig, F., and Altmann, G. T. (2007). Visual-shape competition during language-mediated attention is based on lexical input and not modulated by contextual appropriateness. Vis. Cogn. 15, 985-1018. doi: 10.1080/13506280601130875
21. Huettig, F., and McQueen, J. M. (2007). The tug of war between phonological, semantic and shape information in language-mediated visual search. J. Mem. Lang. 57, 460-482. doi: 10.1016/j.jml.2007.02.001
22. Huettig, F., Mishra, R. K., and Olivers, C. N. (2012). Mechanisms and representations of language-mediated visual attention. Front. Psychol. 2:394. doi: 10.3389/fpsyg.2011.00394
23. Huettig, F., Olivers, C. N., and Hartsuiker, R. J. (2011a). Looking, language, and memory: bridging research from the visual world and visual search paradigms. Acta Psychol. 137, 138-150. doi: 10.1016/j.actpsy.2010.07.013
24. Huettig, F., Rommers, J., and Meyer, A. S. (2011b). Using the visual world paradigm to study language processing: a review and critical evaluation. Acta Psychol. 137, 151-171. doi: 10.1016/j.actpsy.2010.11.003
25. Huettig, F., Singh, N., and Mishra, R. K. (2011c). Language-mediated visual orienting behavior in low and high literates. Front. Psychol. 2:285. doi: 10.3389/fpsyg.2011.00285
26. Huettig, F., Quinlan, P. T., McDonald, S. A., and Altmann, G. (2006). Models of high-dimensional semantic space predict language-mediated eye movements in the visual world. Acta Psychol. 121, 65-80. doi: 10.1016/j.actpsy.2005.06.002
27. Kello, C. T., and Plaut, D. C. (2000). Strategic control in word reading: evidence from speeded responding in the tempo-naming task. J. Exp. Psychol. Learn. Mem. Cogn. 26, 719-750. doi: 10.1037/0278-7393.26.3.719
28. Kintsch, W. (2008). "Symbol systems and perceptual representations," in Symbols and Embodiment: Debates on Meaning and Cognition, eds M. de Vega, A. Glenberg, and A. Graesser (New York, NY: Oxford University Press), 145-163. doi: 10.1093/acprof:oso/9780199217274.003.0008
29. Kukona, A., and Tabor, W. (2011). Impulse processing: a dynamical systems model of incremental eye movements in the visual world paradigm. Cogn. Sci. 35, 1009-1051. doi: 10.1111/j.1551-6709.2011.01180.x
30. Lambon Ralph, M. A., and Patterson, K. (2008). Generalization and differentiation in semantic memory. Ann. N.Y. Acad. Sci. 1124, 61-76. doi: 10.1196/annals.1440.006
31. Lambon Ralph, M. A., Sage, K., Jones, R. W., and Mayberry, E. J. (2010). Coherent concepts are computed in the anterior temporal lobes. Proc. Natl. Acad. Sci. U.S.A. 107, 2717-2722. doi: 10.1073/pnas.0907307107
32. Lupyan, G. (2012). Linguistically modulated perception and cognition: the label-feedback hypothesis. Front. Psychol. 3:54. doi: 10.3389/fpsyg.2012.00054
33. Magnuson, J. S., Tanenhaus, M. K., Aslin, R. N., and Dahan, D. (2003). The time course of spoken word learning and recognition: studies with artificial lexicons. J. Exp. Psychol. Gen. 132, 202-227. doi: 10.1037/0096-3445.132.2.202
34. Mahon, B. Z., and Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. J. Physiol. 102, 59-70.
35. Mani, N., Johnson, E., McQueen, J. M., and Huettig, F. (2013). How yellow is your banana. Toddlers' language-mediated visual search in referent-present tasks. Dev. Psychol. 49, 1036-1044. doi: 10.1037/a0029382
36. Markman, E. M. (1994). Constraints on word meaning in early language acquisition. Lingua 92, 199-227. doi: 10.1016/0024-3841(94)90342-5
37. Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word-recognition. Cognition 25, 71-102. doi: 10.1016/0010-0277(87)90005-9
38. Marslen-Wilson, W. (1993). "Issues of process and representation in lexical access," in Cognitive Models of Speech Processing: The Second Sperlonga Meeting, (Hillsdale, NJ: Erlbaum), 187-210.
39. Mayberry, M. R., Crocker, M. W., and Knoeferle, P. (2009). Learning to attend: a connectionist model of situated language comprehension. Cogn. Sci. 33, 449-496. doi: 10.1111/j.1551-6709.2009.01019.x
40. McClelland, J. L., and Elman, J. L. (1986). The trace model of speech perception. Cogn. Psychol. 18, 1-86. doi: 10.1016/0010-0285(86)90015-0
41. McNorgan, C., Reid, J., and McRae, K. (2011). Integrating conceptual knowledge within and across representational modalities. Cognition 118, 211-233. doi: 10.1016/j.cognition.2010.10.017
42. McQueen, J. M., and Huettig, F. (2012). Changing only the probability that spoken words will be distorted changes how they are recognized. J. Acoust. Soc. Am. 131, 509-517. doi: 10.1121/1.3664087
43. McQueen, J. M., and Viebahn, M. C. (2007). Tracking recognition of spoken words by tracking looks to printed words. Q. J. Exp. Psychol. 60, 661-671. doi: 10.1080/17470210601183890
44. Mirman, D., and Magnuson, J. S. (2009). Dynamics of activation of semantically similar concepts during spoken word recognition. Mem. Cogn. 37, 1026-1039. doi: 10.3758/MC.37.7.1026
45. Monaghan, P., and Mattock, K. (2012). Integrating constraints for learning word-referent mappings. Cognition 123, 133-143. doi: 10.1016/j.cognition.2011.12.010
46. Monaghan, P., and Nazir, T. (2009). "Modelling sensory integration and embodied cognition in a model of word recognition," in Connectionist Models of Behaviour and Cognition II, eds J. Mayor, N. Ruh, and K. Plunkett (Singapore: World Scientific), 337-348.
47. Norris, D. (1994). Shortlist: a connectionist model of continuous speech recognition. Cognition 52, 189-234. doi: 10.1037/0033-295X.115.2.357.
48. Plaut, D. C. (2002). Graded modality-specific specialisation in semantics: a computational account of optic aphasia. Cogn. Neuropsychol. 19, 603-639. doi: 10.1080/02643290244000112
49. Pobric, G., Jefferies, E., and Ralph, M. A. L. (2007). Anterior temporal lobes mediate semantic representation: mimicking semantic dementia by using rTMS in normal participants. Proc. Natl. Acad. Sci. U.S.A. 104, 20137-20141. doi: 10.1073/pnas.0707383104
50. Prinz, J. J. (2002). Furnishing the Mind: Concepts and Their Perceptual Basis. Cambridge, MA: MIT Press.
51. Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L., Hodges, J. R., et al. (2004). Structure and deterioration of semantic memory: a neuropsycho-logical and computational investigation. Psychol. Rev. 111, 205-234. doi: 10.1037/0033-295X.111.1.205
52. Seidenberg, M. S., and Plaut, D. C. (2006). "Progress in understanding word reading: data fitting versus theory building," in From Inkmarks to Ideas: Current Issues in Lexical Processing, ed S. Andrews (New York, NY: Taylor & Francis), 25-49.
53. Simmons, W. K., and Barsalou, L. W. (2003). The similarity-in-topography principle: Reconciling theories of conceptual deficits. Cogn. Neuropsychol. 20, 451-486. doi: 10.1080/02643290342000032
54. Smith, A. C., Monaghan, P., and Huettig, F. (2013). "Modelling the effects of formal literacy training on language mediated visual attention," in Proceedings of the 35th Annual Conference of the Cognitive Science Society (2013), eds M. Knauff, M. Pauen, N. Sebanz, and I. Wachsmuth (Austin, TX: Cognitive Science Society).
55. Spivey, M. (2008). The Continuity of Mind. Vol. 40. New York, NY: Oxford University Press.
56. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632-1634. doi: 10.1126/science.7777863
57. Vandenberghe, R., Price, C., Wise, R., Josephs, O., and Frackowiak, R. S. (1996). Functional anatomy of a common semantic system for words and pictures. Nature 383, 254-256. doi: 10.1038/383254a0
58. Yee, E., and Sedivy, J. C. (2006). Eye movements to pictures reveal transient semantic activation during spoken word recognition. J. Exp. Psychol. Learn. Mem. Cogn. 32, 1-14. doi: 10.1037/0278-7393.32.1.1
59. Yoon, E. Y., Heinke, D., and Humphreys, G. W. (2002). Modelling direct perceptual constraints on action selection: the naming and action model (NAM). Vis. Cogn. 9, 615-661. doi: 10.1080/13506280143000601
60. Ziegler, J. C., and Goswami, U. (2005). Reading acquisition, developmental dyslexia, and skilled reading across languages: a psycholinguistic grain size theory. Psychol. Bull. 131, 3-29. doi: 10.1037/0033-2909.131.1.3