Predictive coding as a model of the V1 saliency map hypothesis

Neural Networks

Volumn 26, Issue , 2012, Pages 7-28

  Spratling, M W ^a  

^a KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

Attention; Contour integration; Neural network model; Saliency map; Texture segmentation; Visual search

Indexed keywords

ATTENTION; CONTOUR INTEGRATION; NEURAL NETWORK MODEL; SALIENCY MAP; TEXTURE SEGMENTATION; VISUAL SEARCH;

CODES (SYMBOLS); COMPUTER PROGRAMMING; NEURAL NETWORKS; TEXTURES;

IMAGE SEGMENTATION;

ARTICLE; ARTIFICIAL NEURAL NETWORK; ATTENTION; CODING; CONTRAST; FEEDBACK SYSTEM; HYPOTHESIS; IMAGE PROCESSING; LUMINANCE; MATHEMATICAL MODEL; NERVE CELL; ORIENTATION; PERCEPTION; PHYSICAL PARAMETERS; PREDICTION; PRIORITY JOURNAL; PSYCHOPHYSIOLOGY; RECEPTIVE FIELD; SALIENCY MAP HYPOTHESIS; SIMULATION; STRIATE CORTEX; TEXTURE SEGMENTATION;

ANIMALS; ATTENTION; COMPUTER SIMULATION; FUNCTIONAL LATERALITY; HUMANS; INHIBITION (PSYCHOLOGY); MODELS, NEUROLOGICAL; NEURAL PATHWAYS; PHOTIC STIMULATION; PSYCHOPHYSICS; REACTION TIME; RETINA; VISUAL CORTEX; VISUAL FIELDS; VISUAL PERCEPTION;

EID: 84855970749     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2011.10.002     Document Type: Article

References (147)

1. Attneave F. Informational aspects of visual perception. Psychology Review 1954, 61:183-193.
2. Baker C.L., Mareschal I. Processing of second-order stimuli in the visual cortex. Progress Brain Research 2001, 134:171-191.
3. Baldi P., Itti L. Of bits and wows: a bayesian theory of surprise with applications to attention. Neural Network 2010, 23:649-666.
4. Bar M. Visual objects in context. Natural Review Neuroscience 2004, 5:617-629.
5. Barlow H. Redundancy reduction revisited. Network 2001, 12:241-253.
6. Barlow H.B. What is the computational goal of the neocortex?. Large-scale neuronal theories of the brain 1994, 1-22. MIT Press, Cambridge, MA, chapter 1. C. Koch, J.L. Davis (Eds.).
7. Bauer B., Jolicoeur P., Cowan W.B. Visual search for colour targets that are or are not linearly separable from distractors. Vision Research 1996, 36:1439-1466.
8. Ben-Shahar O., Zucker S.W. Geometrical computations explain projection patterns of long range horizontal connections in visual cortex. Neural Computers 2004, 16:445-476.
9. Ben-Shahar O., Zucker S.W. Sensitivity to curvatures revealed in orientation-based texture segmentation. Vision Research 2004, 44:257-277.
10. Bergen J.R., Landy M.S. Computational modelling of visual texture segregation. Computational modelling of visual processing 1991, MIT Press, Cambridge, MA. M.S. Landy, M.J. Anthony (Eds.).
11. Biederman I., Mezzanotte R.J., Rabinowitz J.C. Scene perception: detecting and judging objects undergoing relational violations. Cognitive Psychology 1982, 14:143-177.
12. Bosking W., Zhang Y.B.S., Fitzpatrick D. Orientation selectivity and the arrangement of horizontal connections in the tree shrew striate cortex. Journal of Neuroscience 1997, 17:2112-2127.
13. Boyce S.J., Pollatsek A., Rayner K. Effect of background information on object recognition. Journal of Experimental Psycology: Human Perception & Performance 1989, 15:556-566.
14. Bruce N.D.B., Tsotsos J.K. Saliency, attention, and visual search: an information theoretic approach. Journal of Visualization 2009, 9:1-24.
15. Bubic A., von Cramon D.Y., Schubotz R.I. Prediction, cognition and the brain. Frontiers Humaines Neuroscience 2010, 4:1-15.
16. Carrasco M., McLean T.L., Katz S.M., Frieder K.S. Feature asymmetries in visual search: effects of display duration, target eccentricity, orientation and spatial frequency. Vision Research 1998, 38:347-374.
17. Casagrande V.A., Sáry G., Royal D., Ruiz O. On the impact of attention and motor planning on the lateral geniculate nucleus. Progress in brain research 2005, Vol. 149:11-29. Elsevier. V.A. Casagrande, R.W. Guillery, S.M. Sherman (Eds.).
18. Chun M.M. Contextual cueing of visual attention. Trends Cognitive Science 2002, 4:170-178.
19. Chun M.M., Jiang Y. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cognitive Psycholology 1998, 36:28-71.
20. Dakin S.C., Baruch N.J. Context influences contour integration. Journal of Visualization 2009, 9:1-13.
21. Daugman J.G. Two-dimensional spectral analysis of cortical receptive field profiles. Vision Research 1980, 20:847-856.
22. Daugman J.G. Complete discrete 2-D Gabor transformations by neural networks for image analysis and compression. IEEE Transition Acoustics 1988, 36:1169-1179.
23. Donk M. Prioritizing selection of new elements: on the time course of the preview effect. Visualization Cognitives 2005, 17:1373-1385.
24. Donk M. The preview benefit: visual marking, feature-based inhibition, temporal segregation, or onset capture?. Visualization Cognitives 2006, 14:736-748.
25. Donk M., Theeuwes J. Visual marking beside the mark: prioritizing selection by abrupt onsets. Perception Psychophysics 2001, 63:891-900.
26. Duncan J., Humphreys G.W. Visual search and stimulus similarity. Psychology Review 1989, 96:433-458.
27. D'Zmura M. Colour in visual search. Vision Research 1991, 31:951-966.
28. Egner T., Monti J.M., Summerfield C. Expectation and surprise determine neural population responses in the ventral visual stream. Journal of Neuroscience 2010, 30:16601-16608.
29. Enns J.T., Rensink R.A. Sensitivity to three-dimensional orientation in visual search. Psychology Science 1990, 1:323-326.
30. Fahle M. Parallel perception of vernier offests, curvature, and chevrons in humans. Vision Research 1991, 31:2149-2184.
31. Field D., Hayes A., Hess R. Contour integration in the human visual system: evidence for a local association field. Vision Research 1993, 33:173-193.
32. Field D.J. Relations between the statistics of natural images and the response properties of cortical cells. Journal of Optimization Society American A Optimization Image Science Visualization 1987, 4:2379-2394.
33. Field D.J. What is the goal of sensory coding?. Neural Computers 1994, 6:559-601.
34. Foster D.H., Ward P.A. Asymmetries in oriented-line detection indicate two orthogonal filters in early vision. Proceedings R Society of London, B, Biology Science 1991, 243:75-81.
35. Friston K. The free-energy principle: a unified brain theory?. Natural Review Neuroscience 2010, 11:127-138.
36. Friston K.J. A theory of cortical responses. Philosophy Transactions R Society of London, B, Biological Science 2005, 360:815-836.
37. Friston K.J. The free-energy principle: a rough guide to the brain?. Trends Cognitive Science 2009, 13:293-301.
38. Friston K.J., Daunizeau J., Kilner J., Kiebel S.J. Action and behavior: a free-energy formulation. Biological Cybernetics 2010, 102:227-260.
39. Frith U. A curious effect with reversed letters explained by a theory of schema. Perception Psychophysics 1974, 16:113-116.
40. Gilchrist I.D., Harvey M. Refixation frequency and memory mechanisms in visual search. Current Biology 2000, 10:1209-1212.
41. Giora E., Casco C. Region- and edge-based configurational effects in texture segmentation. Vision Research 2007, 47:879-886.
42. Hess R., Field D. Integration of contours: new insight. Trends Cognitive Science 1999, 3:480-486.
43. Hess R., Hayes A., Field D.J. Contour integration and cortical processing. Journal of Physiology Paris 2003, 97:105-119.
44. Hinton G.E. Training products of experts by minimizing contrastive divergence. Neural Computers 2002, 14:1711-1800.
45. Hinton G.E., Dayan P., Frey B.J., Neal R.M. The wake-sleep algorithm for unsupervised neural networks. Science 1995, 268:1158-1161.
46. Hoyer P.O. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research 2004, 5:1457-1469.
47. Hoyer P.O., Hyvärinen A. Independent component analysis applied to feature extraction from colour and stereo images. Network: Computation in Neural Systems 2000, 11:191-210.
48. Itti L., Baldi P. Bayesian surprise attracts human attention. Vision Research 2009, 49:1295-1306. Visual Attention: Psychophysics, electrophysiology and neuroimaging.
49. Itti L., Koch C. Computational modelling of visual attention. Nature Reviews Neuroscience 2001, 2:194-202.
50. Jingling L., Zhaoping L. Change detection is easier tat texture border bars when they are parallel to the border: evidence for V1 mechanisms of bottom-up salience. Perception 2008, 37:197-206.
51. Jones J.P., Palmer L.A. An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. Jounal of Neurophysiology 1987, 58:1233-1258.
52. Jonides J., Yantis S. Uniqueness of abrupt visual onset in capturing attention. Perception Psychophysics 1988, 43:346-354.
53. Kapadia M.K., Ito M., Gilbert C.D., Westheimer G. Improvement in visual sensitivity by changes in local context: parallel studies in human observers and in V1 of alert monkeys. Neuron 1995, 15:843-856.
54. Kapadia M.K., Westheimer G., Gilbert C.D. Spatial distribution of contextual interactions in primary visual cortex and in visual perception. Journal of Neurophysiology 2000, 84:2048-2062.
55. Keysers C., Xiao D.K., Földiák P., Perrett D.I. The speed of sight. Journal of Cognitive Neuroscience 2001, 13:90-101.
56. Klein R.M. Inhibition of return. Trends Cognitive Science 2000, 4:138-147.
57. Koch C., Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Humaines Neurobiology 1985, 4:219-227.
58. Koene A.R., Zhaoping L. Feature-specific interactions in salience from combined feature contrasts: evidence for a bottom-up saliency map in V1. Journal of Visualization 2007, 7:1-14.
59. Landy M.S., Bergen J.R. Texture segregation and orientation gradient. Vision Research 1991, 31:679-691.
60. Laughlin S. Coding efficiency and visual processing. Vision: coding and efficiency 1990, 25-31. Cambridge University Press, chapter 2. C. Blakemore (Ed.).
61. Lee D.D., Seung H.S. Learning the parts of objects by non-negative matrix factorization. Nature 1999, 401:788-791.
62. Lee T.S. Image representation using 2D Gabor wavelets. IEEE Transactions on Pattern Analysis Machine Intelligence 1996, 18:959-971.
63. Lee T.S., Mumford D. Hierarchical Bayesian inference in the visual cortex. Journal of Optimization Society American A Optimization Image Science Visualization 2003, 20:1434-1448.
64. Li F.F., VanRullen R., Koch C., Perona P. Rapid natural scene categorization in the near absence of attention. Proceedings on National Academic Science USA 2002, 99:8378-8383.
65. Li W., Gilbert C.D. Global contour saliency and local colinear interactions. Journal of Neurophysiology 2002, 88:2846-2856.
66. Li Z. Contextual influences in V1 as a basis for pop out and asymmetry in visual search. Proceedings on National Academic Science USA 1999, 96:10530-10535.
67. Li Z. Visual segmentation by contextual influences via intracortical interactions in primary visual cortex. Network 1999, 10:187-212.
68. Li Z. Pre-attentive segmentation in the primary visual cortex. Spatial Visualization 2000, 13:25-50.
69. Li Z. A saliency map in primary visual cortex. Trends Cognitive Science 2002, 6:9-16.
70. Loftus G.R., Mackworth N.H. Cognitive determinants of fixation location during picture viewing. Journal of Experimental Psycology: Human Perception & Performance 1978, 4:565-572.
71. Lubow R.E., Kaplan O. Visual search as a function of type of prior experience with target and distractor. Journal of Experimental Psycology: Human Perception & Performance 1997, 23:14-24.
72. Malik J., Perona P. Preattentive texture discrimination with early vision mechanisms. Journal of Optimization Society American A Optimization Image Science Visualization 1990, 7:923-932.
73. Maljkovic V., Nakayama K. Priming of pop-out: I. role of features. Memoirs Cognitive 1994, 22:657-672.
74. Marcelja S. Mathematical description of the responses of simple cortical cells. Journal of Optimization Society American A Optimization Image Science Visualization 1980, 70:1297-1300.
75. Mirpour K., Arcizet F., Ong W.S., Bisley J.W. Been there, seen that: a neural mechanism for performing efficient visual search. Journal of Neurophysiology 2009, 102:3481-3491.
76. Moraglia G. Display organization and the detection of horizontal line segments. Perception and Psychophysics 1989, 45:265-272.
77. Mounts J.R.W., Tomaselli R.G. Competition for representation is mediated by relative attentional salience. Acta Psychology 2005, 118:261-275.
78. Mumford D. On the computational architecture of the neocortex II: the role of cortico-cortical loops. Biological Cybernetics 1992, 66:241-251.
79. Mundhenk T.N., Itti L. Computational modeling and exploration of contour integration for visual saliency. Biological Cybernetics 2005, 93:188-212.
80. Nagy A.L., Sanchez R.R. Chromaticity and luminance as coding dimensions in visual search. Human Factors 1992, 34:601-614.
81. Nakayama K., Silverman G.H. Serial and parallel processing of visual feature conjunctions. Nature 1986, 320:264-265.
82. Navalpakkam, V., & Itti, L. 2006. An integrated model of top-down and bottom-up attention for optimizing detection speed. In Proceedings IEEE Computing Society Conference Computing Visualization Pattern Recognition (pp. 2049-2056).
83. Nothdurft H.-C. Sensitivity for structure gradient in texture discrimination tasks. Vision Research 1985, 25:1957-1968.
84. Nothdurft H.-C. Texture segmentation and pop-out from orientation contrast. Vision Research 1991, 31:1073-1078.
85. Nothdurft H.-C. Feature analysis and the role of similarity in preattentive vision. Perception and Psychophysics 1992, 52:355-375.
86. Nothdurft H.-C. Salience from feature contrast: variations with texture density. Vision Research 2000, 40:3181-3200.
87. Nothdurft H.-C. Salience-controlled visual search: are the brightest and the least bright targets found by different processes?. Visualization Cognitive 2006, 13:700-732.
88. Olivers C.N.L., Humphreys G.W., Braithwaite J.J. The preview search task: evidence for visual marking. Visua Cognitive 2006, 14:716-735.
89. Olshausen B.A., Field D.J. Emergence of simple-cell receptive properties by learning sparse code for natural images. Nature 1996, 381:607-609.
90. Olshausen B.A., Field D.J. Natural image statistics and efficient coding. Network 1996, 7:333-339.
91. Olshausen B.A., Field D.J. Sparse coding with an overcomplete basis set: a strategy employed by V1?. Vision Research 1997, 37:3311-3325.
92. Olshausen B.A., Field D.J. How close are we to understanding V1?. Neural Computers 2005, 17:1665-1699.
93. Olson R.K., Attneave F. What variables produce similarity grouping?. American Journal of Psychology 1970, 83:1-21.
94. Parent P., Zucker S. Trace inference, curvature consistency, and curve detection. IEEE Transactions on Pattern Analysis Machine Intelligence 1989, 11:823-839.
95. Pashler H., Dobkins K., Huang L.Q. Is contrast just another feature for visual selective attention?. Vision Research 2004, 44:1403-1410.
96. Popple A.V. Context effects on texture border localization bias. Vision Research 2003, 43:739-743.
97. Posner M.I., Cohen Y. Components of visual orienting. Attention and performance X 1984, 531-556. Lawrence Erlbaum Associates, Hillsdale, NJ. H. Bouma, D. Bouwhuis (Eds.).
98. Rao R.P.N., Ballard D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience 1999, 2:79-87.
99. Reicher G.M., Snyder C.R.R., Richards J.T. Familiarity of background characters in visual scanning. Journal of Experimental Psycology: Human Perception & Performance 1976, 2:522-530.
100. Rensink R., O'Regan J., Clark J. To see or not to see: the need for attention to perceive changes in scenes. Psychology Science 1997, 38:368-373.
101. Rensink R., O'Regan J., Clark J. On the failure to detect changes in scenes across brief interruptions. Visualization Cognitive 2000, 37:127-145.
102. Riesenhuber M., Poggio T. Hierarchical models of object recognition in cortex. Natural Neuroscience 1999, 2:1019-1025.
103. Saiki J., Koike T., Takahashi K., Inoue T. Visual search asymmetry with uncertain targets. Journal of Experimental Psycology: Human Perception & Performance 2005, 31:1274-1287.
104. Serre T., Oliva A., Poggio T. A feedforward architecture accounts for rapid categorization. Proceedings of National Academic Science USA 2007, 104:6424-6429.
105. Serre T., Wolf L., Bileschi S., Riesenhuber M., Poggio T. Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis Machine Intelligence 2007, 29:411-426.
106. Shen J., Reingold E.M. Visual search asymmetry: the influence of stimulus familiarity and low-level features. Perception Psychophysics 2001, 63:464-475.
107. Shipp S. The brain circuitry of attention. Trends Cognitive Science 2004, 8:223-230.
108. Smith M.A., Kelly R.C., Lee T.S. Dynamics of response to perceptual pop-out stimuli in macaque V1. Journal of Neurophysiology 2007, 98:3436-3449.
109. Spratling M.W. Predictive coding as a model of biased competition in visual selective attention. Vision Research 2008, 48:1391-1408.
110. Spratling M.W. Reconciling predictive coding and biased competition models of cortical function. Frontier Computing Neuroscience 2008, 2:1-8.
111. Spratling M.W. Predictive coding as a model of response properties in cortical area V1. Journal of Neuroscience 2010, 30:3531-3543.
112. Spratling M.W. A single functional model accounts for the distinct properties of suppression in cortical area V1. Vision Research 2011, 51:563-576.
113. Spratling, M.W. Unsupervised learning of generative and discriminative weights encoding elementary image components in a predictive coding model of cortical function. Neural Computers, (in press).
114. Srinivasan M.V., Laughlin S.B., Dubs A. Predictive coding: a fresh view of inhibition in the retina. Proceeding R Society of London, B, Biological Science 1982, 216:427-459.
115. Summerfield C., Egner T. Expectation and attention in visual cognition. Trends Cognitive Science 2009, 13:403-409.
116. Theeuwes J., Kramer A.F., Atchley P. Visual marking of old objects. Psychon Bulletin Review 1998, 5:130-134.
117. Thorpe S.J., Guyonneau R., Guilbaud N., Allegraud J.M., VanRullen R. Spikenet: real-time visual processing with one spike per neuron. Neurocomputing 2004, 58-60:857-864.
118. Treisman A., Gormican S. Feature analysis in early vision: evidence from search asymmetries. Psychology Review 1988, 95:15-48.
119. Treisman A., Souther S. Search asymmetry: a diagnostic for preattentive processing of separable features. Journal of Experimental Psycology: General 1985, 114:285-310.
120. Treisman A.M., Gelade G. A feature-integration theory of attention. Cognitive Psychology 1980, 12:97-136.
121. Underwood G., Foulsham T. Visual saliency and semantic incongruency influence eye movements when inspecting pictures. Quarterly Journal of Experimental Psychology 2006, 18:1931-1949.
122. Underwood G., Templeman E., Lamming L., Foulsham T. Is attention necessary for object identification? evidence from eye movements during the inspection of real-world scenes. Conscious Cognitive 2008, 17:159-170.
123. Van Hooser S.D. Similarity and diversity in visual cortex: is there a unifying theory of cortical computation?. Neuroscientist 2007, 13:639-656.
124. VanRullen R., Thorpe S.J. Is it a bird? is it a plane? ultra-rapid visual categorisation of natural and artifactual objects. Perception 2001, 30:655-668.
125. Wallis G., Rolls E.T. Invariant face and object recognition in the visual system. Progress in Neurobiology 1997, 51:167-194.
126. Wan X., Lleras A. The effect of feature discriminability on the intertrial inhibition of focused attention. Visualization Cognitive 2010, 18:920-944.
127. Wang Q., Cavanagh P., Green M. Familiarity and pop-out in visual search. Perception Psychophysics 1994, 56:495-500.
128. Watson D., Humphreys G.W. Visual marking: prioritizing selection for new objects by top-down attentional inhibition of old objects. Psychological Review 1997, 104:90-122.
129. Watt R., Ledgeway T., Dakin S.C. Families of models for gabor paths demonstrate the importance of spatial adjacency. Journal of Visualization 2008, 8:1-19.
130. Wilder M.H., Mozer M.C., Wickens C.D. An integrative, experience-based theory of attentional control. Journal of Visualization 2011, 11.
131. Wolfe J.M. Guided search 2.0: a revised model of visual search. Psychon Bulletin Review 1994, 1:202-238.
132. Wolfe J.M. Visual search. Attention 1998, 13-73. University College London Press, London, UK. H. Pashler (Ed.).
133. Wolfe J.M. Asymmetries in visual search: an introduction. Perception and Psychophysics 2001, 63:381-389.
134. Wolfe J.M. Moving towards solutions to some enduring controversies in visual search. Trends Cognitive Science 2003, 7:70-76.
135. Wolfe J.M., Butcher S.J., Lee C., Hyle M. Changing your mind: on the contributions of top-down and bottom-up guidance in visual search for feature singletons. Journal of Experimental Psychology: Human Perception & Performance 2003, 29:483-502.
136. Wolfe J.M., Friedman-Hill S.R. On the role of symmetry in visual search. Psychology Science 1992, 3:194-198.
137. Wolfe J.M., Friedman-Hill S.R., Stewart M.I., O'Connell K.M. The role of categorization in visual search for orientation. Journal of Experimental Psychology: Human Perception & Performance 1992, 18:34-49.
138. Wolfe J.M., Horowitz T.S. What attributes guide the deployment of visual attention and how do they do it?. Nature Review Neuroscience 2004, 5:1-7.
139. Wolfson S.S., Landy M.S. Discrimination of orientation-defined texture edges. Vision Research 1995, 35:2863-2877.
140. Yantis S., Jonides J. Abrupt visual onsets and selective attention: evidence from visual search. Journal of Experimental Psychology: Human Perception & Performance 1984, 10:601-621.
141. Yen S., Finkel L. Extraction of perceptually salient contours by striate cortical networks. Vision Research 1998, 38:719-741.
142. Yuille A., Kersten D. Vision as Bayesian inference: analysis by synthesis?. Trends Cognitive Science 2006, 10:301-308.
143. Zhaoping L. V1 mechanisms and some figure-ground and border effects. Journal of Physiology Paris 2003, 97:503-515.
144. Zhaoping L. Attention capture by eye of origin singletons even without awareness-a hallmark of a bottom-up saliency map in the primary visual cortex. Journal of Visualization 2008, 8:1-18.
145. Zhaoping L., Guyader N., Lewis A. Relative contributions of 2D and 3D cues in a texture segmentation task, implications for the roles of striate and extrastriate cortex in attentional selection. Journal of Visualization 2009, 9:1-22.
146. Zhaoping L., May K.A. Psychophysical tests of the hypothesis of a bottom-up saliency map in primary visual cortex. PLoS Computing Biology 2007, 3:e62.
147. Zhaoping L., May K.A., Koene A. Some fingerprints of V1 mechanisms in the bottom up saliency for visual selection. Computational modelling in behavioural neuroscience: closing the gap between neurophysiology and behaviour 2009, 137-164. Psychology Press, London, UK, chapter 7. D. Heinke, E. Mavritsaki (Eds.).