Computational modelling of visual attention

Nature Reviews Neuroscience

Volumn 2, Issue 3, 2001, Pages 194-203

  Itti, Laurent ^a   Koch, Christof ^b  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL; ATTENTION; BIOLOGICAL MODEL; COMPUTER SIMULATION; HUMAN; METABOLISM; NERVE CELL; PHYSIOLOGY; REVIEW; VISION; VISUAL CORTEX;

ANIMALS; ATTENTION; COMPUTER SIMULATION; HUMANS; MODELS, NEUROLOGICAL; NEURONS; VISUAL CORTEX; VISUAL PERCEPTION;

EID: 0035286497     PISSN: 1471003X     EISSN: 14710048     Source Type: Journal    
DOI: 10.1038/35058500     Document Type: Article

References (122)

1. James, W. The Principles of Psychology (Harvard Univ. Press, Cambridge, Massachusetts, 1980/1981).
2. Treisman, A. M. & Gelade, G. A feature-integration theory of attention. Cogn. Psychol. 12, 97-136 (1980).
3. Bergen, J. R. & Julesz, B. Parallel versus serial processing in rapid pattern discrimination. Nature 303, 696-698 (1983).
4. Treisman, A. Features and objects: The fourteenth Bartlett memorial lecture. Q. J. Exp. Psychol. A 40, 201-237 (1988).
5. Nakayama, K. & Mackeben, M. Sustained and transient components of focal visual attention. Vision Res. 29, 1631-1647 (1989).
6. Braun, J. & Sagi, D. Vision outside the focus of attention. Percept. Psychophys. 48, 45-58 (1990).
7. Hikosaka, O., Miyauchi, S. & Shimojo, S. Orienting a spatial attention — its reflexive, compensatory, and voluntary mechanisms. Brain Res. Cogn. Brain Res. 5, 1-9 (1996).
8. Braun, J. & Julesz, B. Withdrawing attention at little or no cost: detection and discrimination tasks. Percept. Psychophys. 60, 1-23 (1998).
9. Braun, J., Itti, L., Lee, D. K., Zenger, B. & Koch, C. in Visual Attention and Neural Circuits (eds Braun, J., Koch, C. & Davis, J.) (MIT, Cambridge, Massachusetts, in the press).
10. Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193-222 (1995).
11. Crick, F. & Koch, C. Constraints on cortical and thalamic projections: the no-strong-loops hypothesis. Nature 391, 245-250 (1998).
12. Hummel, J. E. & Biederman, I. Dynamic binding in a neural network for shape recognition. Psychol. Rev. 99, 480-517 (1992).
13. Reynolds, J. H. & Desimone, R. The role of neural mechanisms of attention in solving the binding problem. Neuron 24, 19-29 (1999).
14. Weichselgartner, E. & Sperling, G. Dynamics of automatic and controlled visual attention. Science 238, 778-780 (1987).
15. Miller, E. K. The prefrontal cortex and cognitive control. Nature Rev. Neurosci. 1, 59-65 (2000).
16. Hopfinger, J. B., Buonocore, M. H. & Mangun, G. R. The neural mechanisms of top-down attentional control. Nature Neurosci. 3, 284-291 (2000).
17. Corbetta, M., Kincade, J. M., Ollinger, J. M., McAvoy, M. P. & Shulman, G. L. Voluntary orienting is dissociated from target detection in human posterior pariet al cortex. Nature Neurosci. 3, 292-297 (2000).
18. Ungerleider, L. G. & Mishkin, M. in Analysis of Visual Behavior (eds Ingle, D. J., Goodale, M. A. & Mansfield, R. J. W.) 549-586 (MIT, Cambridge, Massachusetts, 1982).
19. Koch, C. & Ullman, S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4, 219-227 (1985).
20. Didday, R. L. & Arbib, M. A. Eye movements and visual perception: A "two visual system'' model. Int. J. Man-Machine Studies 7, 547-569 (1975).
21. Suder, K. & Worgotter, F. The control of low-level information flow in the visual system. Rev. Neurosci. 11, 127-146 (2000).
22. Pasupathy, A. & Connor, C. E. Responses to contour features in macaque area v4. J. Neurophysiol. 82, 2490-2502 (1999).
23. Braun, J. Shape-from-shading is independent of visual attention and may be a 'texton'. Spat. Vis. 7, 311-322 (1993).
24. Sun, J. & Perona, P. Early computation of shape and reflectance in the visual system. Nature 379, 165-168 (1996).
25. Logothetis, N. K., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552-563 (1995).
26. Bar, M. & Biederman, I. Localizing the cortical region mediating visual awareness of object identity. Proc. Natl Acad. Sci. USA 96, 1790-1793 (1999).
27. Vogels, R., Biederman, I., Bar, M. & Lorincz, A. Inferior temporal neurons show greater sensitivity to nonaccidental than metric differences. J. Cogn. Neurosci. (in the press).
28. Kreiman, G., Koch, C. & Fried, I. Category-specific visual responses of single neurons in the human medial temporal lobe. Nature Neurosci. 3, 946-953 (2000).
29. He, Z. J. & Nakayama, K. Perceiving textures: beyond filtering. Vision Res. 34, 151-162 (1994).
30. Treue, S. & Maunsell, J. H. Attentional modulation of visual motion processing in cortical areas MT and MST. Nature 382, 539-541 (1996).
31. Hubel, D. H. & Wiesel, T N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (Lond.) 160, 106-154 (1962).
32. DeSchepper, B. & Treisman, A. Visual memory for novel shapes: implicit coding without attention. J. Exp. Psychol. Learn. Mem. Cogn. 22, 27-47 (1996).
33. Lee, D. K., Itti, L., Koch, C. & Braun, J. Attention activates winner-take-all competition among visual filters. Nature Neurosci. 2, 375-381 (1999).
34. Yeshurun, Y & Carrasco, M. Attention improves or impairs visual performance by enhancing spatial resolution. Nature 396, 72-75 (1998).
35. Mack, A., Tang, B., Tuma, R., Kahn, S. & Rock, I. Perceptual organization and attention. Cogn. Psychol. 24, 475-501 (1992).
36. Moore, C. M. & Egeth, H. Perception without attention: evidence of grouping under conditions of inattention. J. exp. Psychol. Hum. Percept. Perform. 23, 339-352 (1997).
37. Motter, B. C. Neural correlates of attentive selection for color or luminance in extrastriate area V4. J. Neurosci. 14, 2178-2189 (1994).
38. Treue, S. & Trujillo, J. C. M. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399, 575-579 (1999).
39. Barcelo, F., Suwazono, S. & Knight, R. T. Prefrontal modulation of visual processing in humans. Nature Neurosci. 3, 399-403 (2000).
40. Moran, J. & Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782-784 (1985).
41. Niebur, E., Koch, C. & Rosin, C. An oscillation-based model for the neuronal basis of attention. Vision Res. 33, 2789-2802 (1993).
42. Chawla, D., Rees, G. & Friston, K. J. The physiological basis of attentional modulation in extrastriate visual areas. Nature Neurosci. 2, 671-676 (1999).
43. Reynolds, J. H., Pasternak, T. & Desimone, R. Attention increases sensitivity of V4 neurons. Neuron 26, 703-714 (2000).
44. Dosher, B. A. & Lu, Z. L. Mechanisms of perceptual attention in precuing of location. Vision Res. 40, 1269-1292 (2000).
45. Itti, L., Koch, C. & Braun, J. Revisiting spatial vision: towards a unifying model. J. Opt. Soc. Am. A 17, 1899-1917 (2000).
46. Carrasco, M., Penpeci-Talgar, C. & Eckstein, M. Spatial covert attention increases contrast sensitivity across the CSF: support for signal enhancement. Vision Res. 40, 1203-1215 (2000).
47. Deco, G. & Zihl, J. A neurodynamical model of visual attention: Feedback enhancement of spatial resolution in a hierarchical system. J. Comp. Neurosci. (in the press).
48. Daugman, J. G. Spatial visual channels in the Fourier plane. Vision Res. 24, 891-910 (1984).
49. Palmer, L. A., Jones, J. P. & Stepnoski, R. A. in The Neural Basis of VsualFunction (ed. Leventhal, A. G.) 246-265 (CRC, Boca Raton, Florida, 1991).
50. Zetzsche, C. et al. Investigation of a sensorimotor system for saccadic scene analysis: an integrated approach. Proc. 5tth Int. Conf. Simulation Adaptive Behav. 5, 120-126 (1998).
51. Reinagel, P. & Zador, A. M. Natural scene statistics at the centre of gaze. Network Comp. Neural Syst 10, 341-350 (1999).
52. Barth, E., Zetzsche, C. & Rentschler, I. Intrinsic two- dimensional features as textons. J. Opt. Soc. Am. A Opt. Image Sci. Vs. 15, 1723-1732 (1998).
53. Nothdurft, H. Salience from feature contrast: additivity across dimensions. Vision Res. 40, 1183-1201 (2000).
54. Wolfe, J. M. Visual search in continuous, naturalistic stimuli. Vision Res. 34, 1187-1195 (1994).
55. Braun, J. Vision and attention: the role of training. Nature 393, 424-425 (1998).
56. Ahissar, M. & Hochstein, S. The spread of attention and learning in feature search: effects of target distribution and task difficulty. Vision Res. 40, 1349-1364 (2000).
57. Sigman, M. & Gilbert, C. D. Learning to find a shape. Nature Neurosci. 3, 264-269 (2000).
58. Itti, L. & Koch, C. Feature combination strategies for saliency-based visual attention systems. J. Electronic Imaging (in the press).
59. Wolfe, J. in Attention (ed. Pashler, H.) 13-74 (University College London, London, 1996).
60. Carandini, M. & Heeger, D. J. Summation and division by neurons in primate visual cortex. Science 264, 1333-1336 (1994).
61. Nothdurft, H. C. Texture discrimination by cells in the cat lateral geniculate nucleus. Exp. Brain Res. 82, 48-66 (1990).
62. Allman, J., Miezin, F & McGuinness, E. Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annu. Rev. Neurosci. 8, 407-430 (1985).
63. Cannon, M. W. & Fullenkamp, S. C. Spatial interactions in apparent contrast: inhibitory effects among grating patterns of different spatial frequencies, spatial positions and orientations. Vision Res. 31, 1985-1998 (1991).
64. Sillito, A. M., Grieve, K. L., Jones, H. E., Cudeiro, J. & Davis, J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 378, 492-496 (1995).
65. Levitt, J. B. & Lund, J. S. Contrast dependence of contextual effects in primate visual cortex. Nature 387, 73-76 (1997).
66. Gilbert, C. D. & Wiesel, T. N. Columnar specificity of intrinsic horizontal and corticocortical connections in cat visual cortex. J. Neurosci. 9, 2432-2442 (1989).
67. Gilbert, C., Ito, M., Kapadia, M. & Westheimer, G. Interactions between attention, context and learning in primary visual cortex. Vision Res. 40, 1217-1226 (2000).
68. Ben-Av, M. B., Sagi, D. & Braun, J. Visual attention and perceptual grouping. Percept. Psychophys. 52, 277-294 (1992).
69. Grossberg, S. & Raizada, R. D. Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex. Vision Res. 40, 1413-1432 (2000).
70. Vinje, W. E. & Gallant, J. L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273-1276 (2000).
71. Itti, L., Koch, C. & Niebur, E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Patt. Anal. Mach. Intell. 20, 1254-1259 (1998).
72. Tsotsos, J. K. et al. Modeling visual-attention via selective tuning. Artif. Intell. 78, 507-545 (1995).
73. Milanese, R., Gil, S. & Pun, T. Attentive mechanisms for dynamic and static scene analysis. Opt. Eng. 34, 2428-2434 (1995).
74. Itti, L. & Koch, C. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40, 1489-1506 (2000).
75. Toet, A., Bijl, P., Kooi, F. L. & Valeton, J. M. A High- Resoluton Image Dataset for Testing Search and Detection Models (TNO-TM-98-A020) (TNO Human Factors Research Institute, Soesterberg, The Netherlands, 1998).
76. Hamker, F. H. in Proc. Neural Comp. Psychol. Workshop (NCPW'98) (eds von Heinke, D., Humphreys, G. W. & Olson, A.) 252-261 (Springer Verlag, London, 1999).
77. Laberge, D. & Buchsbaum, M. S. Positron emission tomographic measurements of pulvinar activity during an attention task. J. Neurosci. 10, 613-619 (1990).
78. Robinson, D. L. & Petersen, S. E. The pulvinar and visual salience. Trends Neurosci. 15, 127-132 (1992).
79. Kustov, A. A. & Robinson, D. L. Shared neural control of attentional shifts and eye movements. Nature 384, 74-77 (1996).
80. Gottlieb, J. P., Kusunoki, M. & Goldberg, M. E. The representation of visual salience in monkey pariet al cortex. Nature 391, 481-484 (1998).
81. Colby, C. L. & Goldberg, M. E. Space and attention in pariet al cortex. Annu. Rev. Neurosci. 22, 319-349 (1999).
82. Thompson, K. G. & Schall, J. D. Antecedents and correlates of visual detection and awareness in macaque prefrontal cortex. Vision Res. 40, 1523-1538 (2000).
83. Andersen, R. A., Bracewell, R. M., Barash, S., Gnadt, J. W. & Fogassi, L. Eye position effects on visual, memory, and saccade-related activity in areas lip and 7a of macaque. J. Neurosci. 10, 1176-1196 (1990).
84. Pouget, A. & Sejnowski, T. J. Spatial transformations in the pariet al cortex using basis functions. J. Cogn. Neurosci. 9, 222-237 (1997).
85. Blaser, E., Sperling, G. & Lu, Z. L. Measuring the amplification of attention. Proc. Natl Acad. Sci. USA 96, 11681-11686 (1999).
86. Brefczynski, J. A. & DeYoe, E. A. A physiological correlate of the 'spotlight' of visual attention. Nature Neurosci. 2, 370-374 (1999).
87. Amari, S. & Arbib, M. A. in Systems Neuroscience (ed. Metzler, J.) 119-165 (Academic, New York, 1977).
88. Posner, M. I. & Cohen, Y. in Attention and Performance Vol. X (eds Bouma, H. & Bouwhuis, D.) 531-556 (Erlbaum, Hillsdale, New Jersey, 1984).
89. Kwak, H. W. & Egeth, H. Consequences of allocating attention to locations and to other attributes. Percept. Psychophys. 51, 455-464 (1992).
90. Klein, R. M. Inhibition of return. Trends Cogn. Sci. 4, 138-147 (2000).
91. Shimojo, S., Tanaka, Y. & Watanabe, K. Stimulus-driven facilitation and inhibition of visual information processing in environmental and retinotopic representations of space. Brain Res. Cogn. Brain Res. 5, 11-21 (1996).
92. Kingstone, A. & Pratt, J. Inhibition of return is composed of attentional and oculomotor processes. Percept. Psychophys. 61, 1045-1054 (1999).
93. Taylor, T. L. & Klein, R. M. Visual and motor effects in inhibition of return. J. Exp. Psychol. Hum. Percept. Perform. 26, 1639-1656 (2000).
94. Tipper, S. P., Driver, J. & Weaver, B. Object-centred inhibition of return of visual attention. Q. J. Exp. Psychol. A 43, 289-298 (1991).
95. Gibson, B. S. & Egeth, H. Inhibition of return to object-based and environment-based locations. Percept. Psychophys. 55, 323-339 (1994).
96. Ro, T. & Rafal, R. D. Components of reflexive visual orienting to moving objects. Percept. Psychophys. 61, 826-836 (1999).
97. Becker, L. & Egeth, H. Mixed reference frames for dynamic inhibition of return. J. Exp. Psychol. Hum. Percept Perform. 26, 1167-1177 (2000).
98. Horowitz, T. S. & Wolfe, J. M. Visual search has no memory. Nature 394, 575-577 (1998).
99. Mozer, M & Sitton, S. in Attention (ed. Pashler, H.) 341-393 (University College London, London, 1996)
100. Guigon, E., Grandguillaume, P, Otto, I., Boutkhil, L. & Burnod, Y. Neural network models of cortical functions based on the computational properties of the cerebral cortex. J. Physiol. (Paris) 88, 291-308 (1994).
101. Schill, K., Umkehrer, E., Beinlich, S., Krieger, G. & Zetzsche, C. Scene analysis with saccadic eye movements: top-down and bottom-up modeling. J. Electronic Imaging (in the press).
102. Rybak, I. A., Gusakova, V I., Golovan, A. V, Podladchikova, L. N. & Shevtsova, N. A. A model of attention-guided visual perception and recognition. Vision Res. 38, 2387-2400 (1998).
103. Deco, G. & Schurmann, B. A hierarchical neural system with attentional top-down enhancement of the spatial resolution for object recognition. Vision Res. 40, 2845-2859 (2000).
104. Stark, L. W. & Choi, Y. S. in Visual Attention and Cognition (eds Zangemeister, W. H., Stiehl, H. S. & Freska, C.) 3-69 (Elsevier Science B. V, Amsterdam, 1996).
105. Stark, L. W. et al. Representation of human vision in the brain: how does human perception recognize images? J. Electronic Imaging (in the press).
106. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nature Neurosci. 2, 1019-1025 (1999).
107. Riesenhuber, M. & Poggio, T. Models of object recognition. Nature Neurosci. S3, 1199-1204 (2000).
108. O'Craven, K. M., Downing, P. E. & Kanwisher, N. fmri evidence for objects as the units of attentional selection. Nature 401, 584-587 (1999).
109. Roelfsema, P R., Lamme, V A. & Spekreijse, H. Object- based attention in the primary visual cortex of the macaque monkey. Nature 395, 376-381 (1998).
110. Abrams, R. A. & Law, M. B. Object-based visual attention with endogenous orienting. Percept. Psychophys. 62, 818-833 (2000).
111. Webster, M. J. & Ungerleider, L. G. in The Attentive Brain (ed. Parasuraman, R.) 19-34 (MIT, Cambridge, Massachusetts, 1998).
112. Shepherd, M., Findlay, J. M. & Hockey, R. J. The relationship between eye movements and spatial attention. Q. J. Exp. Psychol. 38, 475-491 (1986).
113. Sheliga, B. M., Riggio, L. & Rizzolatti, G. Orienting of attention and eye movements. Exp. Brain Res. 98, 507-522 (1994).
114. Hoffman, J. E. & Subramaniam, B. The role of visual attention in saccadic eye movements. Percept Psychophys. 57, 787-795 (1995).
115. Kowler, E., Anderson, E., Dosher, B. & Blaser, E. The role of attention in the programming of saccades. Vision Res. 35, 1897-1916 (1995).
116. Schall, J. D., Hanes, D. P. & Taylor, T. L. Neural control of behavior: countermanding eye movements. Psychol. Res. 63, 299-307 (2000).
117. Corbetta, M. Frontopariet al cortical networks for directing attention and the eye to visual locations: identical, independent, or overlapping neural systems? Proc. Natl Acad. Sci. USA 95, 831-838 (1998).
118. Nobre, A. C., Gitelman, D. R., Dias, E. C. & Mesulam, M. M. Covert visual spatial orienting and saccades: overlapping neural systems. Neuroimage 11, 210-216 (2000).
119. Dominey, P. F. & Arbib, M. A. A cortico-subcortical model for generation of spatially accurate sequential saccades. Cereb. Cortex 2, 153-175 (1992).
120. Motter, B. C. & Belky, E. J. The guidance of eye movements during active visual search. Vision Res. 38, 1805-1815 (1998).
121. Gilchrist, I. D., Heywood, C. A. & Findlay, J. M. Saccade selection in visual search: evidence for spatial frequency specific between-item interactions. Vision Res. 39, 1373-1383 (1999).
122. Wolfe, J. M. & Gancarz, G. in Basic and Clinical Applications of Vision Science (ed. Lakshminarayanan, V.) 189-192 (Kluwer Academic, Dordrecht, The Netherlands, 1996).