Saliency and saccade encoding in the frontal eye field during natural scene search

Cerebral Cortex

Volumn 24, Issue 12, 2014, Pages 3232-3245

  Fernandes, Hugo L ^a,b,c   Stevenson, Ian H ^d   Phillips, Adam N ^e,f   Segraves, Mark A ^f   Kording, Konrad P ^a,f,g  

^a REHABILITATION INSTITUTE OF CHICAGO   (United States)

^b INSTITUTO GULBENKIAN DE CIÊNCIA   (Portugal)

^c INSTITUTO DE TECNOLOGIA QUÍMICA E BIOLÓGICA   (Portugal)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e TAMAGAWA UNIVERSITY   (Japan)

^f NORTHWESTERN UNIVERSITY   (United States)

^g Northwestern University   (United States)

Author keywords

Attention; Fixation choice; Generalized linear models; Natural stimuli; Neural data analysis; Visuomotor integration

Indexed keywords

ACTION POTENTIAL; ANIMAL CELL; ANIMAL EXPERIMENT; ARTICLE; BOTTOM UP VISUAL SALIENCY; BRIGHTNESS; CELL ACTIVITY; COLOR VISION; CONTROLLED STUDY; EYE POSITION; FEMALE; FRONTAL EYE FIELD; MOTOR ACTIVITY; NERVE CELL; NERVE POTENTIAL; NONHUMAN; PRIORITY JOURNAL; RECEPTIVE FIELD; RHESUS MONKEY; SACCADIC EYE MOVEMENT; SPIKE WAVE; VISUAL ORIENTATION; VISUAL STIMULATION; ANIMAL; ATTENTION; BIOLOGICAL MODEL; CYTOLOGY; MEMORY; PHOTOSTIMULATION; PHYSIOLOGY; PREFRONTAL CORTEX; REACTION TIME; RECEIVER OPERATING CHARACTERISTIC; VISION; VISUAL FIELD;

ACTION POTENTIALS; ANIMALS; ATTENTION; FEMALE; MACACA MULATTA; MEMORY; MODELS, NEUROLOGICAL; NEURONS; PHOTIC STIMULATION; PREFRONTAL CORTEX; REACTION TIME; ROC CURVE; SACCADES; VISUAL FIELDS; VISUAL PERCEPTION;

EID: 84920771575     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bht179     Document Type: Article

References (109)

1. Ahrens MB, Paninski L, Sahani M. 2008. Inferring input nonlinearities in neural encoding models. Network. 19:35-67.
2. Arcizet F, Mirpour K, Bisley JW. 2011. A pure salience response in posterior parietal cortex. Cereb Cortex. 21:2498-2506.
3. Bell AJ, Sejnowski TJ. 1997. The "independent components" of natural scenes are edge filters. Vision Res. 37:3327-3338.
4. Bengio Y, Grandvalet Y. 2004. No unbiased estimator of the variance of k-fold cross-validation. J Mach Learn Res. 5:1089-1105.
5. Berg DJ, Boehnke SE, Marino RA, Munoz DP, Itti L. 2009. Free viewing of dynamic stimuli by humans and monkeys. J Vis. 9:1-15.
6. Betsch BY, Einhäuser W, Körding KP, König P. 2004. The world from a cat's perspective-statistics of natural videos. Biological cybernetics. 90:41-50.
7. Bichot NP, Schall JD, Thompson KG. 1996. Visual feature selectivity in frontal eye fields induced by experience in mature macaques. Nature. 381:697-699.
8. Bichot NP, Schall JD. 1999. Effects of similarity and history on neural mechanisms of visual selection. Nature Neurosci. 2:549-554.
9. Bizzi E. 1968. Discharge of frontal eye field neurons during saccadic and following eye movements in unanesthetized monkeys. Exp Brain Res. 6:69-80.
10. Bizzi E, Schiller PH. 1970. Single unit activity in the frontal eye fields of unanesthetized monkeys during head and eye movement. Exp Brain Res. 10:151-158.
11. Borji A, Sihite D, Itti L. 2012. Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans Image Processing. 22:55-69.
12. Bruce CJ, Goldberg ME. 1985. Primate frontal eye fields. I. Single neurons discharging before saccades. J Neurophysiol. 53:603.
13. Burman DD, Segraves MA. 1994. Primate frontal eye field activity during natural scanning eye movements. J Neurophysiol. 71:1266-1271.
14. Burrows BE, Moore T. 2009. Influence and limitations of popout in the selection of salient visual stimuli by area V4 neurons. J Neurosci. 29:15169-15177.
15. Cave KR, Wolfe JM. 1990. Modeling the role of parallel processing in visual search. Cognit Psychol. 22:225-271.
16. Constantinidis C, Steinmetz MA. 2005. Posterior parietal cortex automatically encodes the location of salient stimuli. J Neurosci. 25:233-238.
17. Crist CF, Yamasaki D S G, Komatsu H, Wurtz RH. 1988. A grid system and a microsyringe for single cell recording. J Neurosci Methods. 26:117-122.
18. Dias EC, Kiesau M, Segraves MA. 1995. Acute activation and inactivation of macaque frontal eye field with GABA-related drugs. J Neurophysiol. 74:2744-2748.
19. Dias EC, Segraves MA. 1999. Muscimol-induced inactivation of monkey frontal eye field: effects on visually and memory-guided saccades. J Neurophysiol. 81:2191-2214.
20. Ehinger KA, Hidalgo-Sotelo B, Torralba A, Oliva A. 2009. Modelling search for people in 900 scenes: a combined source model of eye guidance. Vis Cogn. 17:945-978.
21. Einhäuser W, Kruse W, Hoffmann KP, König P. 2006. Differences of monkey and human overt attention under natural conditions. Vision Res. 46:1194-1209.
22. Einhäuser W, Rutishauser U, Koch C. 2008. Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. J Vis. 8:1-19.
23. Elazary L, Itti L. 2008. Interesting objects are visually salient. J Vis. 8:1-15.
24. Everling S, Munoz DP. 2000. Neuronal correlates for preparatory set associated with pro-saccades and anti-saccades in the primate frontal eye field. J Neurosci. 20:387-400.
25. Fecteau JH, Munoz DP. 2006. Salience, relevance, and firing: a priority map for target selection. Trends Cogn Sci. 10:382-390.
26. Ferrier D. 1875. Experiments on the brains of monkeys. Phil Trans London (Croonian Lecture). 165:433-488.
27. Foulsham T, Barton J J S, Kingstone A, Dewhurst R, Underwood G. 2011. Modeling eye movements in visual agnosia with a saliency map approach: bottom-up guidance or top-down strategy? Neural Netw. 24:665-677.
28. Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT. 1982. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci. 2:1527.
29. Gottlieb JP, Kusunoki M, Goldberg ME. 1998. The representation of visual salience in monkey parietal cortex. Nature. 391:481-484.
30. Harris KD, Csicsvari J, Hirase H, Dragoi G, Buzsáki G. 2003. Organization of cell assemblies in the hippocampus. Nature. 424:552-556.
31. Haslinger R, Pipa G, Lima B, Singer W, Brown EN, Neuenschwander S. 2012. Context matters: the illusive simplicity of Macaque V1 receptive fields. PLOS One. 7:e39699.
32. Hatsopoulos NG, Xu Q, Amit Y. 2007. Encoding of movement fragments in the motor cortex. J Neurosci. 27:5105.
33. Hays AV, Richmond BJ, Optican LM. 1982. Unix-based multipleprocess system, for real-time data acquisition and control. WESCON Conf Proc. 2:1-10.
34. Heinzl H, Mittlböck M. 2003. Pseudo R-squared measures for Poisson regression models with over- or underdispersion. Comput Stat Data Anal. 44:253-271.
35. Howard IS, Ingram JN, Kording KP, Wolpert DM. 2009. Statistics of natural movements are reflected in motor errors. J Neurophysiol. 102:1902-1910.
36. Hyvarinen A, Hurri J, Vayrynen J. 2003. Bubbles: a unifying framework for low-level statistical properties of natural image sequences. J Opt Soc Am AOpt Image Sci Vis. 20:1237-1252.
37. Ingram JN, Körding KP, Howard IS, Wolpert DM. 2008. The statistics of natural hand movements. Exp Brain Res. 188:223-236.
38. Itti L, Baldi P. 2006. Bayesian surprise attracts human attention. Adv Neural Inform Process Syst. 18:547.
39. Itti L, Koch C. 2001. Computational modelling of visual attention. Nat Rev Neurosci. 2:194-203.
40. Itti L, Koch C. 2000. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res. 40:1489-1506.
41. Judge SJ, Richmond BJ, Chu FC. 1980. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res. 20:535-538.
42. Kayser C, Kording KP, Konig P. 2004. Processing of complex stimuli and natural scenes in the visual cortex. Curr Opin Neurobiol. 14:468-473.
43. Kayser C, Nielsen KJ, Logothetis NK. 2006. Fixations in natural scenes: interaction of image structure and image content. Vision Res. 46:2535-2545.
44. Kayser C, Salazar R, König P. 2003. Responses to natural scenes in cat V1. J Neurophysiol. 90:1910-1920.
45. Koch C, Ullman S. 1985. Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol. 4:219-227.
46. Land MF, Hayhoe M. 2001. In what ways do eye movements contribute to everyday activities? Vision Res. 41:3559-3565.
47. Lewicki MS. 2002. Efficient coding of natural sounds. Nat Neurosci. 5:356-363.
48. MacEvoy SP, Hanks TD, Paradiso MA. 2008. Macaque V1 activity during natural vision: effects of natural scenes and saccades. J Neurophysiol. 99:460-472.
49. Mazer JA, Gallant JL. 2003. Goal-related activity in V4 during free viewing visual search. Evidence for a ventral stream visual salience map. Neuron. 40:1241-1250.
50. McPeek RM, Keller EL. 2002. Saccade target selection in the superior colliculus during a visual search task. J Neurophysiol. 88:2019-2034.
51. Melloni L, van Leeuwen S, Alink A, Müller NG. 2012. Interaction between bottom-up saliency and top-down control: how saliency maps are created in the human brain. Cerebr Cortex. 22:2943-2952.
52. Mohler CW, Goldberg ME, Wurtz RH. 1973. Visual receptive fields of frontal eye field neurons. Brain Res. 61:385-389.
53. Murthy A, Thompson KG, Schall JD. 2001. Dynamic dissociation of visual selection from saccade programming in frontal eye field. J Neurophysiol. 86:2634-2637.
54. Oliva A, Torralba A, Castelhano MS, Henderson JM. 2003. Top-down control of visual attention in object detection. In Proceedings of the 2003 International Conference on Image Processing, Vol 1, pp. 253-256.
55. Olshausen B, Field D. 1996. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature. 381:607-609.
56. O'Shea J, Muggleton NG, Cowey A, Walsh V. 2004. Timing of target discrimination in human frontal eye fields. J Cogn Neurosci. 16:1060-1067.
57. Pearl J. 1988. Probabilistic reasoning in intelligent systems: networks of plausible inference. San Francisco: Morgan Kaufmann.
58. Peng X, Sereno ME, Silva AK, Lehky SR, Sereno AB. 2008. Shape selectivity in primate frontal eye field. J Neurophysiol. 100:796-814.
59. Phillips A, Segraves M. 2010. Predictive activity in Macaque frontal eye field neurons during natural scene searching. J Neurophysiol. 103:1238.
60. Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chichilnisky E, Simoncelli EP. 2008. Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature. 454:995-999.
61. Ray S, Pouget P, Schall JD. 2009. Functional distinction between visuomovement and movement neurons in macaque frontal eye field during saccade countermanding. J Neurophysiol. 102:3091-3100.
62. Robinson DA. 1963. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans Bio-Med Electron BME. 10:137-145.
63. Rolls ET, Tovee MJ. 1995. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. J Neurophysiol. 73:713-726.
64. Saleh M, Takahashi K, Amit Y, Hatsopoulos NG. 2010. Encoding of coordinated grasp trajectories in primary motor cortex. J Neurosci. 30:17079.
65. Sato T, Murthy A, Thompson KG, Schall JD. 2001. Search efficiency but not response interference affects visual selection in frontal eye field. Neuron. 30:583-591.
66. Sato TR, Schall JD. 2003. Effects of stimulus-response compatibility on neural selection in frontal eye field. Neuron. 38:637-648.
67. Schall JD. 1991. Neuronal activity related to visually guided saccades in the frontal eye fields of rhesus monkeys: comparison with supplementary eye fields. J Neurophysiol. 66:559-579.
68. Schall JD, Hanes DP. 1993. Neural basis of saccade target selection in frontal eye field during visual search. Nature. 366:467-469.
69. Schall JD, Hanes DP, Thompson KG, King DJ. 1995. Saccade target selection in frontal eye field of macaque. I. Visual and premovement activation. J Neurosci. 15:6905-6918.
70. Schall JD, Thompson KG. 1999. Neural selection and control of visually guided eye movements. Annu Rev Neurosci. 22:241-259.
71. Schiller PH, True SD, Conway JL. 1980. Deficits in eye movements following frontal eye field and superior colliculus ablations. J Neurophysiol. 44:1175-1189.
72. Schölkopf B, Smola AJ. 2002. Learning with kernels: support vector machines, regularization, optimization, and beyond. Cambridge, MA: MIT Press.
73. Schwartz O, Simoncelli EP. 2001. Natural signal statistics and sensory gain control. Nat Neurosci. 4:819-825.
74. Segraves MA. 1992. Activity of monkey frontal eye field neurons projecting to oculomotor regions of the pons. J Neurophysiol. 68:1967-1985.
75. Segraves MA, Goldberg ME. 1987. Functional properties of corticotectal neurons in the monkey's frontal eye field. J Neurophysiol. 58:1387-1419.
76. Segraves MA, Park K. 1993. The relationship of monkey frontal eye field activity to saccade dynamics. J Neurophysiol. 69:1880-1889.
77. Serences JT, Yantis S. 2006. Selective visual attention and perceptual coherence. Trends Cogn Sci. 10:38-45.
78. Serre T, Wolf L, Bileschi S, Riesenhuber M, Poggio T. 2007. Robust object recognition with cortex-like mechanisms. IEEE Trans Pattern Anal Mach Intelligence. 411-426.
79. Sharpee T, Rust NC, Bialek W. 2004. Analyzing neural responses to natural signals: maximally informative dimensions. Neural Comput. 16:223-250.
80. Smith EC, Lewicki MS. 2006. Efficient auditory coding. Nature. 439:978-982.
81. Smyth D, Willmore B, Baker GE, Thompson ID, Tolhurst DJ. 2003. The receptive-field organization of simple cells in primary visual cortex of ferrets under natural scene stimulation. J Neurosci. 23:4746-4759.
82. Soltani A, Koch C. 2010. Visual saliency computations: mechanisms, constraints, and the effect of feedback. J Neurosci. 30:12831-12843.
83. Sommer MA, Tehovnik EJ. 1997. Reversible inactivation of macaque frontal eye field. Exp Brain Res. 116:229-249.
84. Sommer MA, Wurtz RH. 2000. Composition and topographic organization of signals sent from the frontal eye field to the superior colliculus. J Neurophysiol. 83:1979-2001.
85. Sommer MA, Wurtz RH. 2001. Frontal eye field sends delay activity related to movement, memory, and vision to the superior colliculus. J Neurophysiol. 85:1673-1685.
86. Srivastava A, Lee AB, Simoncelli EP, Zhu SC. 2003. On advances in statistical modeling of natural images. J Math Imaging Vis. 18:17-33.
87. Suzuki H, Azuma M. 1977. Prefrontal neuronal activity during gazing at a light spot in the monkey. Brain Res. 126:497-508.
88. Theunissen FE, David SV, Singh NC, Hsu A, Vinje WE, Gallant JL. 2001. Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Network. 12:289-316.
89. Theunissen FE, Sen K, Doupe AJ. 2000. Spectral-temporal receptive fields of nonlinear auditory neurons obtained using natural sounds. J Neurosci. 20:2315-2331.
90. Thompson KG, Bichot NP. 2005. A visual salience map in the primate frontal eye field. Progr Brain Res. 147:249-262.
91. Thompson KG, Bichot NP, Sato TR. 2005. Frontal eye field activity before visual search errors reveals the integration of bottom-up and top-down salience. J Neurophysiol. 93:337.
92. Thompson KG, Bichot NP, Schall JD. 1997. Dissociation of visual discrimination from saccade programming in macaque frontal eye field. J Neurophysiol. 77:1046-1050.
93. Thompson KG, Hanes DP, Bichot NP, Schall JD. 1996. Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol. 76:4040-4055.
94. Torralba A, Oliva A, Castelhano MS, Henderson JM. 2006. Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. Psychol Rev. 113:766.
95. Treisman A. 1988. Features and objects: the fourteenth Bartlett memorial lecture. Q J Exp Psychol A. 40:201-237.
96. Truccolo W, Eden UT, Fellows MR, Donoghue JP, Brown EN. 2005. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J Neurophysiol. 93:1074.
97. Tseng PH, Carmi R, Cameron I G M, Munoz DP, Itti L. 2009. Quantifying center bias of observers in free viewing of dynamic natural scenes. J Vis.: 1-16.
98. Turano KA, Geruschat DR, Baker FH. 2003. Oculomotor strategies for the direction of gaze tested with a real-world activity. Vis Res. 43:333-346.
99. Van Hateren JH, van der Schaaf A. 1998. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc R Soc Lond B Biol Sci. 265.
100. Vinje WE, Gallant JL. 2002. Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1. J Neurosci. 22:2904-2915.
101. Vogel J, Schiele B. 2007. Semantic modeling of natural scenes for content-based image retrieval. Int J Comput Vis. 72:133-157.
102. Wainwright MJ, Schwartz O, Simoncelli EP. 2002. Natural image statistics and divisive normalization: modeling nonlinearities and adaptation in cortical neurons. Stat Theor Brain. 203-222.
103. Walther D, Koch C. 2006. Modeling attention to salient proto-objects. Neural Netw. 19:1395-1407.
104. Wardak C, Vanduffel W, Orban GA. 2010. Searching for a salient target involves frontal regions. Cereb Cortex. 20:2464-2477.
105. Weliky M, Fiser J, Hunt RH, Wagner DN. 2003. Coding of natural scenes in primary visual cortex. Neuron. 37:703-718.
106. Willmore B, Watters PA, Tolhurst DJ. 2000. A comparison of naturalimage-based models of simple-cell coding. Perception. 29:1017-1040.
107. Yarbus AL. 1967. Eye movements and vision. New York: Plenum Press.
108. Zhao Q, Koch C. 2011. Learning a saliency map using fixated locations in natural scenes. J Vis. 11:1-15.
109. Zhou H, Desimone R. 2011. Feature-based attention in the frontal eye field and area V4 during visual search. Neuron. 70:1205-1217.