Object segmentation controls image reconstruction from natural scenes

PLoS Biology

Volumn 15, Issue 8, 2017, Pages

  Neri, Peter ^a  

^a PSL Research University   (France)

Author keywords

[No Author keywords available]

Indexed keywords

EXTRACTION; IMAGE RECONSTRUCTION; NOISE; VISUAL CORTEX; ALGORITHM; BIOLOGICAL MODEL; BRAIN MAPPING; COMPARATIVE STUDY; ELECTROENCEPHALOGRAPHY; FEMALE; HUMAN; MALE; NERVE CELL; NERVE CELL NETWORK; PHYSIOLOGY; RETINA; RETINA NERVE CELL; SPATIAL BEHAVIOR; VISION; VISUAL FIELD; VISUAL SYSTEM;

BIOLOGICAL MARKER;

ALGORITHMS; BIOMARKERS; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; FEMALE; HUMANS; MALE; MODELS, NEUROLOGICAL; NERVE NET; NEURONS; RETINA; RETINAL NEURONS; SPATIAL PROCESSING; VISION, OCULAR; VISUAL CORTEX; VISUAL FIELDS; VISUAL PATHWAYS; VISUAL PERCEPTION;

EID: 85028658613     PISSN: 15449173     EISSN: 15457885     Source Type: Journal    
DOI: 10.1371/journal.pbio.1002611     Document Type: Article

References (109)

1. Hubel DH, The visual cortex of the brain. Sci Am. 1963;209:54–62. 14075682
2. Marr DC, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: Freeman; 1982.
3. Itti L, Koch C, Computational modelling of visual attention. Nat Rev Neurosci. 2001;2(3):194–203. doi: 10.1038/35058500 11256080
4. Morgan MJ, Features and the 'primal sketch'. Vision Res. 2011;51:738–753. doi: 10.1016/j.visres.2010.08.002 20696182
5. Arbelaez P, Maire M, Fowlkes C, Malik J, Contour Detection and Hierarchical Image Segmentation. IEEE Trans Patt Anal Mach Intell. 2011;33(5):898–916. doi: 10.1109/TPAMI.2010.161 20733228
6. Hou X, Yuille AL, Koch C. Boundary Detection Benchmarking: Beyond F-Measures. In: CVPR. IEEE Computer Society; 2013. p. 2123–2130.
7. Farid H, Simoncelli EP, Differentiation of discrete multidimensional signals. IEEE Trans Image Process. 2004;13(4):496–508. 15376584
8. Neri P, The elementary operations of human vision are not reducible to template matching. PLoS Comput Biol. 2015;11(11):e1004499. doi: 10.1371/journal.pcbi.1004499 26556758
9. Neri P, Global properties of natural scenes shape local properties of human edge detectors. Front Psychol. 2011;2:172. doi: 10.3389/fpsyg.2011.00172 21886631
10. Neri P, Semantic control of feature extraction from natural scenes. J Neurosci. 2014;34(6):2374–2388. doi: 10.1523/JNEUROSCI.1755-13.2014 24501376
11. Carandini M, Heeger DJ, Normalization as a canonical neural computation. Nat Rev Neurosci. 2011;13:51–62. doi: 10.1038/nrn3136 22108672
12. Yu C, Levi DM, Surround modulation in human vision unmasked by masking experiments. Nat Neurosci. 2000;3:724–728. doi: 10.1038/76687 10862706
13. Paltoglou AE, Neri P, Attentional control of sensory tuning in human visual perception. J Neurophysiol. 2012;107:1260–1274. doi: 10.1152/jn.00776.2011 22131380
14. Levi DM, Crowding–an essential bottleneck for object recognition: a mini-review. Vision Res. 2008;48:635–654. doi: 10.1016/j.visres.2007.12.009 18226828
15. Carandini M, Demb JB, Mante V, Tolhurst DJ, Dan Y, Olshausen BA, et al. Do we know what the early visual system does?J Neurosci. 2005;25:10577–10597. doi: 10.1523/JNEUROSCI.3726-05.2005 16291931
16. Ullman S, High-level vision. Cambridge, MA: MIT Press; 1996.
17. Simoncelli EP, Olshausen BA, Natural image statistics and neural representation. Annu Rev Neurosci. 2001;24:1193–1216. doi: 10.1146/annurev.neuro.24.1.1193 11520932
18. Felsen G, Dan Y, A natural approach to studying vision. Nat Neurosci. 2005;8:1643–1646. doi: 10.1038/nn1608 16306891
19. Geisler WS, Visual perception and the statistical properties of natural scenes. Annu Rev Psychol. 2008;59:167–192. doi: 10.1146/annurev.psych.58.110405.085632 17705683
20. Bar M, Visual objects in context. Nat Rev Neurosci. 2004;5(8):617–629. doi: 10.1038/nrn1476 15263892
21. Cox DD, Do we understand high-level vision?Curr Opin Neurobiol. 2014;25:187–193. doi: 10.1016/j.conb.2014.01.016 24552691
22. Arbelaez P, Hariharan B, Gu C, Gupta S, Bourdev L, Malik J. Semantic segmentation using regions and parts. In: CVPR; 2012.
23. Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z, Du D, et al. Conditional Random Fields as Recurrent Neural Networks. In: International Conference on Computer Vision (ICCV); 2015.
24. David SV, Vinje WE, Gallant JL, Natural stimulus statistics alter the receptive field structure of v1 neurons. J Neurosci. 2004;24:6991–7006. doi: 10.1523/JNEUROSCI.1422-04.2004 15295035
25. Gilbert CD, Li W, Top-down influences on visual processing. Nat Rev Neurosci. 2013;14(5):350–363. doi: 10.1038/nrn3476 23595013
26. Shamma S, Fritz J, Adaptive auditory computations. Curr Opin Neurobiol. 2014;25:164–168. doi: 10.1016/j.conb.2014.01.011 24525107
27. Olshausen BA, Bower J, Springer20 years of learning about vision: Questions answered, Questions unanswered, and Questions not yet asked. 20 Years of Computational Neuroscience (2012; p. 243–270.
28. Lamme VA, Roelfsema PR, The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 2000;23(11):571–579. 11074267
29. Herzog MH, Clarke AM, Why vision is not both hierarchical and feedforward. Front Comput Neurosci. 2014;8:135. doi: 10.3389/fncom.2014.00135 25374535
30. Firestone C, Scholl BJ, Cognition does not affect perception: Evaluating the evidence for 'top-down' effects. Behav Brain Sci. 2015; p. 1–77.
31. Murray RF, Classification images: A review. J Vis. 2011;11 (5):1–25. doi: 10.1167/11.5.2 21536726
32. Elder JH, Velisavljević L, Cue dynamics underlying rapid detection of animals in natural scenes. J Vis. 2009;9(7):7. doi: 10.1167/9.7.7 19761322
33. Bookstein FL, Principal Warps: Thin-Plate Splines and the Decomposition of Deformations. IEEE Trans Pattern Anal Mach Intell. 1989;11(6):567–585.
34. Ahumada AJ, Classification image weights and internal noise level estimation. J Vis. 2002;2:121–131. doi: 10.1167/2.1.8 12678600
35. Green DM, Swets JA, Signal Detection Theory and Psychophysics. New York: Wiley; 1966.
36. Burgess AE, Colborne B, Visual signal detection. IV. Observer inconsistency. J Opt Soc Am A. 1988;5:617–627. 3404312
37. Neri P, How inherently noisy is human sensory processing?Psychon Bull Rev. 2010;17:802–808. doi: 10.3758/PBR.17.6.802 21169572
38. Awwad Shiekh Hasan B, Joosten E, Neri P, Estimation of internal noise using double passes: does it matter how the second pass is delivered?Vision Res. 2012;69:1–9. doi: 10.1016/j.visres.2012.06.014 22835631
39. Green DM, Consistency of auditory detection judgments. Psychol Rev. 1964;71:392–407. 14208857
40. Diependaele K, Brysbaert M, Neri P, How noisy is lexical decision?Front Psychol. 2012;3:348. doi: 10.3389/fpsyg.2012.00348 23015793
41. Harel J, Koch C, Perona P. Graph-Based Visual Saliency? NIPS. 2006.
42. Shi J, Malik J, Normalized Cuts and Image Segmentation. IEEE Trans Patt Anal Mach Intell. 1997;22:888–905.
43. Sun X, Shang K, Ming D, Tian J, Ma J, A Biologically-Inspired Framework for Contour Detection Using Superpixel-Based Candidates and Hierarchical Visual Cues. Sensors (Basel). 2015;15(10):26654–26674.
44. Luck SJ, An Introduction to the Event-Related Potential Technique. MIT Press; 2005.
45. Rousselet GA, Does Filtering Preclude Us from Studying ERP Time-Courses?Front Psychol. 2012;3:131. doi: 10.3389/fpsyg.2012.00131 22586415
46. Luck SJ, Hillyard SA, Spatial filtering during visual search: evidence from human electrophysiology. J Exp Psychol Hum Percept Perform. 1994;20(5):1000–1014. 7964526
47. Cumming G, The new statistics: why and how. Psychol Sci. 2014;25(1):7–29. doi: 10.1177/0956797613504966 24220629
48. Wasserstein RL, Lazar NA, The ASA's Statement on p-Values: Context, Process, and Purpose. The American Statistician. 2016;70(2):129–133.
49. Rauss K, Pourtois G, What is Bottom-Up and What is Top-Down in Predictive Coding?Front Psychol. 2013;4:276. doi: 10.3389/fpsyg.2013.00276 23730295
50. Nyström M, Holmqvist K, Semantic override of low-level features in image viewing - both initially and overall. J Eye Movement Research. 2008;2(2):1–11.
51. Velisavljević L, Elder JH, Visual short-term memory of local information in briefly viewed natural scenes: configural and non-configural factors. J Vis. 2008;8(16):1–17.
52. Valentine T, Upside-down faces: a review of the effect of inversion upon face recognition. Br J Psychol. 1988;79 (Pt 4):471–491.
53. Williford JR, von der Heydt R, Figure-Ground Organization in Visual Cortex for Natural Scenes. eNeuro. 2016;3(6).
54. Gaspar CM, Bennett PJ, Sekuler AB, The effects of face inversion and contrast-reversal on efficiency and internal noise. Vision Res. 2008;48:1084–1095. doi: 10.1016/j.visres.2007.12.014 18314157
55. Hupe JM, James AC, Payne BR, Lomber SG, Girard P, Bullier J, Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature. 1998;394:784–787. doi: 10.1038/29537 9723617
56. Rieger JW, Kochy N, Schalk F, Gruschow M, Heinze HJ, Speed limits: orientation and semantic context interactions constrain natural scene discrimination dynamics. J Exp Psychol Hum Percept Perform. 2008;34(1):56–76. doi: 10.1037/0096-1523.34.1.56 18248140
57. Van Rullen R, Four common conceptual fallacies in mapping the time course of recognition. Front Psychol. 2011;2:365. doi: 10.3389/fpsyg.2011.00365 22162973
58. Eriksen CW, The use of a visual mask may seriously confound your experiment. Percept Psychophys. 1980;28(1):89–92. 7413417
59. Hermens F, Herzog MH, Francis G, Combining simultaneous with temporal masking. J Exp Psychol Hum Percept Perform. 2009;35(4):977–988. doi: 10.1037/a0014252 19653743
60. Wascher E, The timing of stimulus localisation and the Simon effect: an ERP study. Exp Brain Res. 2005;163(4):430–439. doi: 10.1007/s00221-004-2198-1 15711792
61. Jolicoeur P, Brisson B, Robitaille N, Dissociation of the N2pc and sustained posterior contralateral negativity in a choice response task. Brain Res. 2008;1215:160–172. doi: 10.1016/j.brainres.2008.03.059 18482718
62. Ratcliff R, Philiastides MG, Sajda P, Quality of evidence for perceptual decision making is indexed by trial-to-trial variability of the EEG. Proc Natl Acad Sci USA. 2009;106(16):6539–6544. doi: 10.1073/pnas.0812589106 19342495
63. Rauss K, Schwartz S, Pourtois G, Top-down effects on early visual processing in humans: a predictive coding framework. Neurosci Biobehav Rev. 2011;35(5):1237–1253. doi: 10.1016/j.neubiorev.2010.12.011 21185860
64. Mazza V, Turatto M, Umilta C, Eimer M, Attentional selection and identification of visual objects are reflected by distinct electrophysiological responses. Exp Brain Res. 2007;181(3):531–536. doi: 10.1007/s00221-007-1002-4 17602216
65. Casiraghi M, Fortier-Gauthier U, Sessa P, Dell'Acqua R, Jolicœur P, N1pc reversal following repeated eccentric visual stimulation. Psychophysiology. 2013;50(4):351–364. doi: 10.1111/psyp.12021 23317174
66. Hickey C, Di Lollo V, McDonald JJ, Electrophysiological indices of target and distractor processing in visual search. J Cogn Neurosci. 2009;21(4):760–775. doi: 10.1162/jocn.2009.21039 18564048
67. Dowdall JR, Luczak A, Tata MS, Temporal variability of the N2pc during efficient and inefficient visual search. Neuropsychologia. 2012;50(10):2442–2453. doi: 10.1016/j.neuropsychologia.2012.06.015 22743179
68. Hesse JK, Tsao DY, Consistency of Border-Ownership Cells across Artificial Stimuli, Natural Stimuli, and Stimuli with Ambiguous Contours. J Neurosci. 2016;36(44):11338–11349. doi: 10.1523/JNEUROSCI.1857-16.2016 27807174
69. Lutfi RA, Gilbertson L, Effect of decision weights and internal noise on the growth of d' with N. J Acoust Soc Am. 2011;130(5):L329–333.
70. Brunelli R, Poggio T, Template matching: matched spatial filters and beyond. Pattern Recognition. 1997;30:751–768.
71. Pritchett LM, Murray RF, Classification images reveal decision variables and strategies in forced choice tasks. Proc Natl Acad Sci USA. 2015;112(23):7321–7326. doi: 10.1073/pnas.1422169112 26015584
72. Li HH, Barbot A, Carrasco M, Saccade Preparation Reshapes Sensory Tuning. Curr Biol. 2016;26(12):1564–1570. doi: 10.1016/j.cub.2016.04.028 27265397
73. Gold J, Bennett PJ, Sekuler AB, Signal but not noise changes with perceptual learning. Nature. 1999;402:176–178. doi: 10.1038/46027 10647007
74. Maniglia M, Pavan A, Aedo-Jury F, Trotter Y, The spatial range of peripheral collinear facilitation. Sci Rep. 2015;5:15530. doi: 10.1038/srep15530 26502834
75. He ZJ, Nakayama K, Surfaces versus features in visual search. Nature. 1992;359(6392):231–233. doi: 10.1038/359231a0 1528263
76. Manassi M, Sayim B, Herzog MH, Grouping, pooling, and when bigger is better in visual crowding. J Vis. 2012;12(10):13. doi: 10.1167/12.10.13 23019118
77. Pelli DG, Uncertainty explains many aspects of visual contrast detection and discrimination. J Opt Soc Am A. 1985;2:1508–1532. 4045584
78. Levi DM, Visual crowding. Curr Biol. 2011;21:R678–679. doi: 10.1016/j.cub.2011.07.025 21959149
79. Parkhurst DJ, Niebur E, Texture contrast attracts overt visual attention in natural scenes. Eur J Neurosci. 2004;19(3):783–789. 14984430
80. Anderson NC, Ort E, Kruijne W, Meeter M, Donk M, It depends on when you look at it: Salience influences eye movements in natural scene viewing and search early in time. J Vis. 2015;15(5):9. doi: 10.1167/15.5.9 26067527
81. Pelli DG, Palomares M, Majaj NJ, Crowding is unlike ordinary masking: distinguishing feature integration from detection. J Vis. 2004;4:1136–1169. doi: 10.1167/4.12.12 15669917
82. Thorpe S, Fize D, Marlot C, Speed of processing in the human visual system. Nature. 1996;381(6582):520–522. doi: 10.1038/381520a0 8632824
83. Liu H, Agam Y, Madsen JR, Kreiman G, Timing, timing, timing: fast decoding of object information from intracranial field potentials in human visual cortex. Neuron. 2009;62(2):281–290. doi: 10.1016/j.neuron.2009.02.025 19409272
84. Scholte HS, Jolij J, Fahrenfort JJ, Lamme VA, Feedforward and recurrent processing in scene segmentation: electroencephalography and functional magnetic resonance imaging. J Cogn Neurosci. 2008;20(11):2097–2109. doi: 10.1162/jocn.2008.20142 18416684
85. Roelfsema PR, Tolboom M, Khayat PS, Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron. 2007;56(5):785–792. doi: 10.1016/j.neuron.2007.10.006 18054856
86. Kubilius J, Wagemans J, Op de Beeck HP, A conceptual framework of computations in mid-level vision. Front Comput Neurosci. 2014;8:158. doi: 10.3389/fncom.2014.00158 25566044
87. Bieniek MM, Bennett PJ, Sekuler AB, Rousselet GA, A robust and representative lower bound on object processing speed in humans. Eur J Neurosci. 2016;44(2):1804–1814. doi: 10.1111/ejn.13100 26469359
88. Foxe JJ, Simpson GV, Flow of activation from V1 to frontal cortex in humans. A framework for defining "early" visual processing. Exp Brain Res. 2002;142(1):139–150. doi: 10.1007/s00221-001-0906-7 11797091
89. Bullier J, Feedback connections and conscious vision. Trends Cogn Sci (Regul Ed). 2001;5(9):369–370.
90. Watt RJ, Scanning from coarse to fine spatial scales in the human visual system after the onset of a stimulus. J Opt Soc Am A. 1987;4:2006–2021. 3430211
91. McSorley E, Findlay JM, Are spatial frequencies integrated from coarse to fine?Perception. 2002;31(8):955–967. doi: 10.1068/p3257 12269589
92. Joubert OR, Rousselet GA, Fize D, Fabre-Thorpe M, Processing scene context: fast categorization and object interference. Vision Res. 2007;47(26):3286–3297. doi: 10.1016/j.visres.2007.09.013 17967472
93. Schneider D, Beste C, Wascher E, On the time course of bottom-up and top-down processes in selective visual attention: an EEG study. Psychophysiology. 2012;49(11):1492–1503. doi: 10.1111/j.1469-8986.2012.01462.x 22978270
94. Katsuki F, Constantinidis C, Bottom-up and top-down attention: different processes and overlapping neural systems. Neuroscientist. 2014;20(5):509–521. doi: 10.1177/1073858413514136 24362813
95. Van Essen D, Olshausen BA, Anderson CH, Gallant JTL. Pattern recognition, attention, and information bottlenecks in the primate visual system. In: SPIE. vol. 1473; 1991.
96. Whitney D, Haberman J, Sweeney TD, Werner JS, Chalupa LM, From textures to crowds: multiple levels of summary statistical perception. The New Visual Neurosciences (Cambridge: MIT Press. 2014; p. 695–710.
97. Motter BC, Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. J Neurophysiol. 1993;70:909–919. 8229178
98. Peelen MV, Fei-Fei L, Kastner S, Neural mechanisms of rapid natural scene categorization in human visual cortex. Nature. 2009;460(7251):94–97. doi: 10.1038/nature08103 19506558
99. Epshtein B, Lifshitz I, Ullman S, Image interpretation by a single bottom-up top-down cycle. Proc Natl Acad Sci USA. 2008;105(38):14298–14303. doi: 10.1073/pnas.0800968105 18796607
100. Shi WB X, Tsotsos JK. Early recurrence improves edge detection. In: British Machine Vision Conference; 2013.
101. Ito M, Gilbert CD, Attention modulates contextual influences in the primary visual cortex of alert monkeys. Neuron. 1999;22(3):593–604. 10197538
102. Katsuki F, Constantinidis C, Early involvement of prefrontal cortex in visual bottom-up attention. Nat Neurosci. 2012;15(8):1160–1166. doi: 10.1038/nn.3164 22820465
103. Ullman S, Object recognition and segmentation by a fragment-based hierarchy. Trends Cogn Sci (Regul Ed). 2007;11(2):58–64.
104. Biederman I, Perceiving real-world scenes. Science. 1972;177:77–80. 5041781
105. Li FF, VanRullen R, Koch C, Perona P, Rapid natural scene categorization in the near absence of attention. Proc Natl Acad Sci USA. 2002;99:9596–9601. doi: 10.1073/pnas.092277599 12077298
106. Kriegeskorte N, Deep Neural Networks: A New Framework for Modeling Biological Vision and Brain Information Processing. Annual Review of Vision Science. 2015;1:417–446. doi: 10.1146/annurev-vision-082114-035447 28532370
107. DiCarlo JJ, Zoccolan D, Rust NC, How does the brain solve visual object recognition?Neuron. 2012;73(3):415–434. doi: 10.1016/j.neuron.2012.01.010 22325196
108. Carandini M, From circuits to behavior: a bridge too far?Nat Neurosci. 2012;15(4):507–509. doi: 10.1038/nn.3043 22449960
109. Strasburger H, Rentschler I, Juttner M, Peripheral vision and pattern recognition: a review. J Vis. 2011;11(5):13. doi: 10.1167/11.5.13 22207654