Mechanisms and representations of language-mediated visual attention

Frontiers in Psychology

Volumn 3, Issue JAN, 2012, Pages

  Huettig, Falk ^a,b   Mishra, Ramesh Kumar ^c   Olivers, Christian N L ^d  

^a MAX PLANCK INSTITUTE FOR PSYCHOLINGUISTICS   (Netherlands)

^b RADBOUD UNIVERSITY   (Netherlands)

^c UNIVERSITY OF ALLAHABAD   (India)

^d VU UNIVERSITY AMSTERDAM   (Netherlands)

Author keywords

Attention; Eye movements; Language; Memory; Vision

Indexed keywords

EID: 84874283512     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2011.00394     Document Type: Review

References (107)

1. Allopenna, P. D., Magnuson, J. S., and Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: evidence for continuous mapping models. J. Mem. Lang. 38, 419-439.
2. Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: the "blank screen paradigm." Cognition 93,79-87.
3. Altmann, G. T. M., and Kamide, Y. (2007). The real-time mediation of visual attention by language and world knowledge: linking anticipatory (and other) eye movements to linguistic processing. J. Mem. Lang. 57,502-518.
4. Altmann, G. T. M., and Mirkovic, J. (2009). Incrementality and prediction in human sentence processing. Cogn. Sci. 33, 583-609.
5. Arnett, J. (2008). The neglected 95%: why American psychology needs to become less American. Am. Psychol. 63, 602-614.
6. Bornstein, M. H. (1985). On the development of color naming in young children: data and theory. Brain Lang. 26, 72-93.
7. Cave, K. R. (1999). The feature gate model of visual attention. Psychol. Res. 62, 182-194.
8. Cooper, R.M. (1974). The control of eye fixation by the meaning of spoken language: a new methodology for the real-time investigation of speech perception, memory, and language processing. Cogn. Psychol. 6, 84-107.
9. Cowan, N. (2001). The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav. Brain Sci. 24, 87-185.
10. Cree, G. S., and McRae, K. (2003). Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). J. Exp. Psychol Gen. 132, 163-201.
11. Dahan, D., and Tanenhaus, M. K. (2005). Looking at the rope when looking for the snake: conceptually mediated eye movements during spoken-word recognition. Psychon. Bull Rev. 12, 453-459.
12. Davidoff, J., and Mitchell, D. (1993). The color cognition of children. Cognition 48, 121-137.
13. Dell'Acqua, R., and Grainger, J. (1999). Unconscious semantic priming form pictures. Cognition 73, B1-B15.
14. Desimone, R., and Duncan, J. (1995). Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18,193-222.
15. Dunabeitia, J. A., Aviles, A., Afonso, O., Scheepers, C., and Carreiras, M. (2009). Qualitative differences in the representation of abstract versus concrete words: evidence from the visual-world paradigm. Cognition 110,284-292.
16. Ferreira, F., Apel, J., and Henderson, J. M. (2008). Taking a new look at looking at nothing. Trends Cogn. Sci. (Regul.Ed.) 12,405-410.
17. Friedman-Hill, S. R., Robertson, L. C., and Treisman, A. (1995). Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions. Science 269, 853-855.
18. Gaskell, M. G., and Marslen-Wilson, W. D. (1997). Integrating form and meaning: a distributed model of speech perception. Lang. Cogn. Process. 12, 613-656.
19. Hamker, F. H. (2004). A dynamic model of how feature cues guide spatial attention. Vision Res. 44, 501-521.
20. Henrich, J., Heine, S. J., and Norenza-yan, A. (2010). The weirdest people in the world? Behav. Brain Sci. 33, 61-135.
21. Hoover, M. A., and Richardson, D. C. (2008). When facts go down the rabbit hole: contrasting features and object hood as indexes to memory. Cognition 108, 533-542.
22. Huettig, F., and Altmann, G. (2011). Looking at anything that is green when hearing "frog": how object surface colour and stored object colour knowledge influence language-mediated overt attention. Q. J. Exp. Psychol. 64,122-145.
23. Huettig, F., and Altmann, G. T. M. (2004). "The online processing of ambiguous and unambiguous words in context: evidence from head-mounted eye-tracking," in The Online Study of Sentence Comprehension: Eyetracking, ERP and Beyond, eds M. Carreiras and C. Clifton (New York, NY: Psychology Press), 187-207.
24. Huettig, F., and Altmann, G. T. M. (2005). Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm. Cognition 96, B23-B32.
25. Huettig, F., and Altmann, G. T. M. (2007). Visual-shape competition during language-mediated attention is based on lexical input and not modulated by contextual appropriateness. Vis. Cogn. 15, 985-1018.
26. Huettig, F., and McQueen, J. M. (2007). The tug of war between phonological, semantic, and shape information in language-mediated visual search. J. Mem. Lang. 54, 460-482.
27. Huettig, F., and McQueen, J. M. (2011). The nature of the visual environment induces implicit biases during language-mediated visual search. Mem. Cognit. 39, 1068-1084.
28. Huettig, F., Olivers, C. N. L., and Hartsuiker, R. J. (2011a). Looking, language, and memory: bridging research from the visual world and visual search paradigms. Acta Psychol. (Amst.) 137, 138-150.
29. Huettig, F., Rommers, J., and Meyer, A. S. (201 1b). Using the visual world paradigm to study language processing: a review and critical evaluation. Acta Psychol. (Amst.) 137, 151-171.
30. Huettig, F., Singh, N., and Mishra, R. K. (2011c). Language-mediated visual orienting behavior in low and high literates. Front. Psychol. 2:285. doi:10.3389/fpsyg.2011.00285
31. Huettig, F., Singh, N., Singh, S., and Mishra, R. K. (2011d). "Language-mediated prediction is related to reading ability and formal literacy," in Paper Presented at the AMLaP 2011 Conference in Paris, Paris.
32. Huettig, F., Quinlan, P. T., McDonald, S. A., and Altmann, G. T. M. (2006). Models of high-dimensional semantic space predict language-mediated eye movements in the visual world. Acta Psychol. (Amst.) 121, 65-80.
33. Humphreys, G. W., and Müller, H. J. (1993). SEarch via recursive rejection (SERR): a connectionist model of visual search. Cogn. Psychol. 25, 43-110.
34. Itti, L., and Koch, C. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40, 1489-1506.
35. Jackendoff, R. (2002). Foundations of Language: Brain, Meaning, Grammar, Evolution. New York: Oxford University Press.
36. Johnson, E., McQueen, J. M., and Huettig, F. (2011). Toddlers' language-mediated visual search: they need not have the words for it. Q. J. Exp. Psychol. 64, 1672-1682.
37. Johnson, E. K., and Huettig, F. (2011). Eye movements during language-mediated visual search reveal a strong link between overt visual attention and lexical processing in 36-month-olds. Psychol. Res. 75, 35-42.
38. Kahneman, D., Treisman, A., and Gibbs, B. (1992). The reviewing of object files: object-specific integration of information. Cogn. Psychol. 24, 175-219.
39. Kanwisher, N. (1987). Repetition blindness: type recognition without token individuation. Cognition 27, 117-143.
40. Knoeferle, P., and Crocker, M. W. (2007). The influence of recent scene events on spoken comprehension: evidence from eye-movements. J. Mem. Lang. 57,519-543.
41. Konkle, T., Brady, T. F., Alvarez, G. A., and Oliva, A. (2010a). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. J. Exp. Psychol. Gen. 139,558-578.
42. Konkle, T., Brady, T. F., Alvarez, G. A., and Oliva, A. (2010b). Scene memory is more detailed than you think: the role of categories in visual long-term memory. Psychol. Sci. 21, 1551-1556.
43. Kukona, A., and Tabor, W (2011). Impulse processing: a dynamical systems model of the visual world paradigm. Cogn. Sci. 35, 1009-1051.
44. Lamme, V. A. F., and Roelfsema, P. R. (2000). The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571-579.
45. Landauer, T. K., and Dumais, S. T. (1997). A solution to Plato's problem: the latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychol. Rev. 104,211-240.
46. Mani, N., and Plunkett, K. (2010). In the infant's mind's ear: evidence for implicit naming in 18-month-olds. Psychol. Sci. 21,908-913.
47. Marcus, G. (1998). Rethinking elimi-native connectionism. Cogn. Psychol. 37,243-282.
48. Marcus, G. (2001). The Algebraic Mind. Cambridge, MA: MIT Press.
49. Marslen-Wilson, W., and Welsh, A. (1978). Processing interactions and lexical access during word recognition in continuous speech. Cogn. Psychol. 10,29-63.
50. Marslen-Wilson, W. D. (1987). Functional parallelism in spoken word recognition. Cognition 25, 71-102.
51. Mayberry, M., Crocker, M. W., and Knoeferle, P. (2009). Learning to attend: a connectionist model of the coordinated interplay of utterance, visual context, and world knowledge. Cogn. Sci. 33, 449-496.
52. McClelland, J. L., and Elman, J. L. (1986). The TRACE model of speech perception. Cogn. Psychol. 18, 1-86.
53. McDonald, S. A. (2000). Environmental Determinants of Lexical Processing Effort. Unpublished doctoral dissertation, University of Edinburgh, Scotland. Available at: http://www.inf.ed.ac.uk/publications/thesis/online/IP000007.pdf [accessed December 10,2004].
54. McDonald, S. A., and Shillcock, R. C. (2003). Eye movements reveal the on-line computation of lexical probabilities. Psychol. Sci. 14, 648-652.
55. McMurray, B., Samelson, V. M., Lee, S. H., and Tomblin, J. B. (2010). Individual differences in online spoken word recognition: implications for SLI. Cogn. Psychol. 60, 1-39.
56. McMurray, B., Tanenhaus, M., and Aslin, R. (2002). Gradient effects of within-category phonetic variation on lexical access. Cognition 86, B33-B42.
57. Meyer, A. S., Belke, E., Telling, A. L., and Humphreys, G. W. (2007). Early activation of object names in visual search. Psychon. Bull. Rev. 14, 710-716.
58. Meyer, A. S., andDamian, M. E (2007). Activation of distractor names in the picture-picture interference paradigm. Mem. Cognit. 35, 494-503.
59. Miller, E. K., and Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167-202.
60. Mirman, D., and Magnuson, J. S. (2009). Dynamics of activation of semanti-cally similar concepts during spoken word recognition. Mem. Cognit. 37, 1026-1039.
61. Mishra, R. K., and Marmolejo-Ramos, E (2010). On the mental representations originating during the interaction between language and vision. Cogn. Process. 11,295-305.
62. Mishra, R. K., Singh, N., and Huettig, E (2011). "'Looking at nothing' is neither automatic nor an inevitable consequence of human cognitive architecture," in Paper Presented at the AMLaP 2011 Conference in Paris, Paris.
63. Moores, E., Laiti, L., and Chelazzi, L. (2003). Associative knowledge controls deployment of visual selective attention. Nat. Neurosci. 6, 182-189.
64. Morsella, E., and Miozzo, M. (2002). Evidence for a cascade model of lexical access in speech production. J. Exp. Psychol. Learn. Mem. Cogn. 28, 555-563.
65. Moss, H. E., McCormick, S. F., and Tyler, L. K. (1997). The time-course of semantic activation during spoken word recognition. Lang. Cogn. Process. 12, 695-731.
66. Myachykov, A., Thompson, D., Scheepers, C., and Garrod, S. (2011).Visual attention and structural choice in sentence production across languages. Lang. Linguist. Compass 5, 95-107.
67. Navarette, E., and Costa, A. (2005). Phonological activation of ignored pictures: further evidence for a cascade model of lexical access. J. Mem. Lang. 53, 359-377.
68. Nelson, D. L., McEvoy, C. L., and Schreiber, T. A. (1998). The University of South Florida Word Association, Rhyme, and Word Fragment Norms. Available at: http://www.usf.edu/FreeAssociation/
69. Noizet, G., and Pynte, J. (1976). Implicit labeling and readiness for pronunciation during the perceptual process. Perception 5,217-223.
70. Olivers, C. N. L. (2009). What drives memory-driven attentional capture? The effects of memory type, display type, and search type. J. Exp. Psychol. Hum. Percept. Perform. 35, 1275-1291.
71. Olivers, C. N. L. (2011). Long-term visual associations affect attentional guidance. Acta Psychol. (Amst.) 137, 243-247.
72. Olivers, C. N. L., Meijer, F., and Theeuwes, J. (2006). Feature-based memory-driven attentional capture: visual working memory content affects visual attention. J. Exp. Psychol. Hum. Percept. Perform. 32, 1243-1265.
73. O'Regan, J. K. (1992). Solving the "real" mysteries of visual perception: the world as an outside memory. Can. I Psychol. 46, 461-488.
74. O'Regan, J. K., and Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behav. Brain Sci. 24, 939-1011.
75. Palmer, J., Verghese, P., and Pavel, M. (2000). The psychophysics of visual search. Vision Res. 40, 1227-1268.
76. Posner,M. I. (1980). Orienting of attention, the VIIth Sir Frederic Bartlett Lecture. Q. J. Exp. Psychol. 32, 3-25.
77. Pylyshyn, Z. (2001). Visual indexes, preconceptual objects, and situated vision. Cognition 80,127-58.
78. Reynolds, J. H., and Desimone, R. (1999). The role of neural mechanisms of attention in solving the binding problem. Neuron 24,19-9.
79. Richardson, D. C., and Spivey, M. J. (2000). Representation, space and hollywood squares: looking at things that aren't there anymore. Cognition 76, 269-295.
80. Roy, D., and Mukherjee, N. (2005). Towards situated speech understanding: visual context priming of language models. Comput. Speech Lang. 19,227-248.
81. Salverda, A. P., Brown, M., and Tanen-haus, M. K. (2011). A goal-based perspective on eye movements in visual-world studies. Acta Psychol. (Amst.) 137, 172-180.
82. Salverda, A. P., Dahan, D., and McQueen, J. M. (2003). The role of prosodic boundaries in the resolution of lexical embedding in speech comprehension. Cognition 90,51-89.
83. Salverda, A. P., and Tanenhaus, M. K. (2010). Tracking the time course of orthographic information in spoken-word recognition. J. Exp. Psychol. Learn. Mem. Cogn. 36, 1108-1117.
84. Schreuder, R., Flores D'Arcais, G. B., and Glazenborg, G. (1984). Effects of perceptual and conceptual similarity in semantic priming. Psychol. Res. 45, 339-354.
85. Shatzman, K. B., and McQueen, J. M. (2006). Prosodic knowledge affects the recognition of newly acquired words. Psychol. Sci. 17, 372-377.
86. Soto, D., Heinke, D., Humphreys, G. W., and Blanco, M. J. (2005). Early, involuntary top-down guidance of attention from working memory. J. Exp. Psychol. Hum. Percept. Perform. 31,248-261.
87. Soto, D., and Humphreys, G. W. (2007). Automatic guidance of visual attention from verbal working memory. J. Exp. Psychol. Hum. Percept. Perform. 33, 730-757.
88. Spivey, M., and Geng, J. (2001). Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects. Psychol. Res. 65,235-241.
89. Spivey, M. J., Richardson, D. C., and Fit-neva, S. A. (2004)."Memory outside of the brain: oculomotor indexes to visual and linguistic information," in The Interface of Language, Vision, and Action: Eye Movements and the Visual World, eds J. Henderson and E Ferreira (New York: Psychology Press), 161-189.
90. Stephen, D. G., Mirman, D., Magnuson, J. S., and Dixon, J. A. (2009). Lévy-like diffusion in eye movements during spoken-language comprehension. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 79,056114.
91. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., and Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science 268, 1632-1634.
92. Theeuwes, J. (2010), Top-down and bottom-up control of visual selection. Acta Psychol. (Amst.) 135, 77-99.
93. Treisman, A. (1996). The binding problem. Curr. Opin. Neurobiol. 6, 171-178.
94. Treisman, A., and Gelade, G. (1980). A feature-integration theory of attention. Cogn. Psychol. 12, 97-136.
95. Treisman, A., and Sato, S. (1990). Conjunction search revisited. J. Exp. Psychol. Hum. Percept. Perform.16, 459-478.
96. van der Velde, F., and de Kamps, M. (2001). From knowing what to knowing where: modeling object-based attention with feedback disin-hibition of activation. J. Cogn. Neu-rosci. 13,479-491.
97. Vanduffel, W., Ekstrom, L. B., Roelf-sema, P. R., Arsenault, J. T., and Bonmassar, G. (2008). Bottom-up dependent gating of frontal signals in early visual cortex. Science 321, 414-417.
98. Vickery, T. J., King, L.-W., and Jiang, Y. (2005). Setting up the target template in visual search. J. Vis. 5, 81-92.
99. Wolfe, J. M. (1994). Guided Search 2.0. A revised model of visual search. Psychon. Bull. Rev. 1, 202-238.
100. Wolfe, J. M. (1998). "Visual search," in Attention, ed. H. Pashler (Hove: Psychological Press), 14-73.
101. Wolfe, J. M., Horowitz, T. S., Kenner, N., Hyle, M., and Vasan, N. (2004). How fast can you change your mind? Vision Res. 44, 1411-1426.
102. Wolfe, J. M., Klempen, N., and Dahlen, K. (2000). Post attentive vision. J. Exp. Psychol. Hum. Percept. Perform. 26, 693-716.
103. Yee, E., Huffstetler, S., and Thompson-Schill, S. L. (2011). Function follows form: activation of shape and function features during object identification. J. Exp. Psychol. Gen. 140, 348-363.
104. Yee, E., Overton, E., and Thompson-Schill, S. L. (2009). Looking for meaning: eye movements are sensitive to overlapping semantic features, not association. Psychon. Bull. Rev. 16, 869-874.
105. Yee, E., and Sedivy, J. C. (2001). "Using eye movements to track the spread of semantic activation during spoken word recognition," in Paper Presented at the 13th Annual CUNYSentence Processing Conference, Philadel-phia.
106. Yee, E., and Sedivy, J. C. (2006). Eye movements to pictures reveal transient semantic activation during spoken word recognition. J. Exp. Psychol. Learn. Mem. Cogn. 32, 1-14.
107. Zelinsky, G. J., and Murphy, G. L. (2000). Synchronizing visual and language processing: an effect of object name length on eye movements. Psychol. Sci. 11, 125-131