Attention, Learning, and the Value of Information

Neuron

Volumn 76, Issue 2, 2012, Pages 281-295

  Gottlieb, Jacqueline ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATION; BEHAVIORAL RESEARCH; BRAIN NERVE CELL; DECISION MAKING; EXECUTIVE FUNCTION; EYE MOVEMENT CONTROL; HUMAN; LEARNING STYLE; MENTAL FUNCTION; NEUROFEEDBACK; NEUROMODULATION; NONHUMAN; PRIORITY JOURNAL; RECEPTIVE FIELD; REINFORCEMENT; REVIEW; REWARD; SELECTIVE ATTENTION; STATE DEPENDENT LEARNING; VISUAL INFORMATION; VISUAL THRESHOLD;

ANIMALS; ARTIFICIAL INTELLIGENCE; ATTENTION; BRAIN; EYE MOVEMENTS; HUMANS; LEARNING; MENTAL PROCESSES; NEURONS;

EID: 84867711788     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2012.09.034     Document Type: Review

References (108)

1. Acuña D.E., Schrater P. Structure learning in human sequential decision-making. PLoS Comput. Biol. 2010, 6:e1001003.
2. Albright T.D. On the perception of probable things: neural substrates of associative memory, imagery, and perception. Neuron 2012, 74:227-245.
3. Balan P.F., Gottlieb J. Integration of exogenous input into a dynamic salience map revealed by perturbing attention. J. Neurosci. 2006, 26:9239-9249.
4. Balan P.F., Gottlieb J. Functional significance of nonspatial information in monkey lateral intraparietal area. J. Neurosci. 2009, 29:8166-8176.
5. Balan P.F., Oristaglio J., Schneider D.M., Gottlieb J. Neuronal correlates of the set-size effect in monkey lateral intraparietal area. PLoS Biol. 2008, 6:e158.
6. Ballard D.H., Hayhoe M.M. Modelling the role of task in the control of gaze. Vis. Cogn. 2009, 17:1185-1204.
7. Baluch F., Itti L. Mechanisms of top-down attention. Trends Neurosci. 2011, 34:210-224.
8. Bavelier D., Green C.S., Pouget A., Schrater P. Brain plasticitiy through the life span: learning to learn and action video games. Annu. Rev. Neurosci. 2012, 21:391-416.
9. Beierholm U.R., Dayan P. Pavlovian-instrumental interaction in 'observing behavior'. PLoS Comput. Biol. 2010, 6:e1000903. 10.1371/journal.pcbi.1000903.
10. Bernacchia A., Seo H., Lee D., Wang X.J. A reservoir of time constants for memory traces in cortical neurons. Nat. Neurosci. 2011, 14:366-372.
11. Bisley J.W., Goldberg M.E. Neuronal activity in the lateral intraparietal area and spatial attention. Science 2003, 299:81-86.
12. Bisley J.W., Goldberg M.E. Attention, intention, and priority in the parietal lobe. Annu. Rev. Neurosci. 2010, 33:1-21.
13. Boehnke S.E., Berg D.J., Marino R.A., Baldi P.F., Itti L., Munoz D.P. Visual adaptation and novelty responses in the superior colliculus. Eur. J. Neurosci. 2011, 34:766-779.
14. Braun D.A., Mehring C., Wolpert D.M. Structure learning in action. Behav. Brain Res. 2010, 206:157-165.
15. Brockmole J.R., Henderson J.M. Object appearance, disappearance, and attention prioritization in real-world scenes. Psychon. Bull. Rev. 2005, 12:1061-1067.
16. Brockmole J.R., Henderson J.M. Prioritization of new objects in real-world scenes: evidence from eye movements. J. Exp. Psychol. Hum. Percept. Perform. 2005, 31:857-868.
17. Bromberg-Martin E.S., Hikosaka O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 2009, 63:119-126.
18. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362:933-942.
19. Damaraju E., Huang Y.M., Barrett L.F., Pessoa L. Affective learning enhances activity and functional connectivity in early visual cortex. Neuropsychologia 2009, 47:2480-2487.
20. Daw N.D., Gershman S.J., Seymour B., Dayan P., Dolan R.J. Model-based influences on humans' choices and striatal prediction errors. Neuron 2011, 69:1204-1215.
21. Dayan P., Daw N.D. Decision theory, reinforcement learning, and the brain. Cogn. Affect. Behav. Neurosci. 2008, 8:429-453.
22. Dayan P., Kakade S., Montague P.R. Learning and selective attention. Nat. Neurosci. 2000, 3(Suppl):1218-1223.
23. Della Libera C., Chelazzi L. Learning to attend and to ignore is a matter of gains and losses. Psychol. Sci. 2009, 20:778-784.
24. den Ouden H.E., Daunizeau J., Roiser J., Friston K.J., Stephan K.E. Striatal prediction error modulates cortical coupling. J. Neurosci. 2010, 30:3210-3219.
25. Fetsch C.R., Pouget A., DeAngelis G.C., Angelaki D.E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 2012, 15:146-154.
26. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
27. Flagel S.B., Clark J.J., Robinson T.E., Mayo L., Czuj A., Willuhn I., Akers C.A., Clinton S.M., Phillips P.E., Akil H. A selective role for dopamine in stimulus-reward learning. Nature 2011, 469:53-57.
28. Freedman D.J., Assad J.A. A proposed common neural mechanism for categorization and perceptual decisions. Nat. Neurosci. 2011, 14:143-146.
29. Friston K. Is the free-energy principle neurocentric?. Nat. Rev. Neurosci. 2010, 11:605-606.
30. Gershman S.J., Niv Y. Learning latent structure: carving nature at its joints. Curr. Opin. Neurobiol. 2010, 20:251-256.
31. Glimcher P.W. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl. Acad. Sci. USA 2011, 108(Suppl 3):15647-15654.
32. Gold J.I., Shadlen M.N. The neural basis of decision making. Annu. Rev. Neurosci. 2007, 30:535-574.
33. Gottlieb J., Balan P.F. Attention as a decision in information space. Trends Cogn. Sci. 2010, 14:240-248.
34. Gottlieb J.P., Kusunoki M., Goldberg M.E. The representation of visual salience in monkey parietal cortex. Nature 1998, 391:481-484.
35. Gottlieb J., Snyder L.H. Spatial and non-spatial functions of the parietal cortex. Curr. Opin. Neurobiol. 2010, 20:731-740.
36. Hayhoe M., Ballard D. Eye movements in natural behavior. Trends Cogn. Sci. 2005, 9:188-194.
37. Hayhoe M.M., McKinney T., Chajka K., Pelz J.B. Predictive eye movements in natural vision. Exp. Brain Res. 2012, 217:125-136.
38. Hickey C., Chelazzi L., Theeuwes J. Reward changes salience in human vision via the anterior cingulate. J. Neurosci. 2010, 30:11096-11103.
39. Hickey C., Chelazzi L., Theeuwes J. Reward guides vision when it's your thing: trait reward-seeking in reward-mediated visual priming. PLoS ONE 2010, 5:e14087.
40. Selective Attention to Conditioned Stimuli in Human Discrimination Learning: Untangling the Effects of Outcome Prediction, Valence, Arousal, and Uncertainty 2010, Oxford University Press, Oxford. L. Hogarth, A. Dickinson, T. Duka (Eds.).
41. Holland P.C., Maddux J.-M. Brain systems of attention in associative learning. Attention and Associative Learning 2010, Oxford University Press, Oxford. C.J. Mitchell, M.E. Le Pelley (Eds.).
42. Johnson A., Varberg Z., Benhardus J., Maahs A., Schrater P. The hippocampus and exploration: dynamically evolving behavior and neural representations. Front. Hum. Neurosci. 2012, 6:216.
43. Kable J.W., Glimcher P.W. The neurobiology of decision: consensus and controversy. Neuron 2009, 63:733-745.
44. Karacan H., Hayhoe M.M. Is attention drawn to changes in familiar scenes?. Vis. Cogn. 2008, 16:356-374.
45. Kepecs A., Uchida N., Zariwala H.A., Mainen Z.F. Neural correlates, computation and behavioural impact of decision confidence. Nature 2008, 455:227-231.
46. Kolling N., Behrens T.E., Mars R.B., Rushworth M.F. Neural mechanisms of foraging. Science 2012, 336:95-98.
47. Krajbich I., Armel C., Rangel A. Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 2010, 13:1292-1298.
48. Land M.F. Vision, eye movements, and natural behavior. Vis. Neurosci. 2009, 26:51-62.
49. Le Pelley M.E. Attention and Associative Learning 2010, Oxford University Press, Oxford.
50. Lo C.C., Wang X.J. Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat. Neurosci. 2006, 9:956-963.
51. Louie K., Grattan L.E., Glimcher P.W. Reward value-based gain control: divisive normalization in parietal cortex. J. Neurosci. 2011, 31:10627-10639.
52. Ma W.J., Beck J.M., Pouget A. Spiking networks for Bayesian inference and choice. Curr. Opin. Neurobiol. 2008, 18:217-222.
53. Maddux J.M., Holland P.C. Dissociations between medial prefrontal cortical subregions in the modulation of learning and action. Behav. Neurosci. 2011, 125:383-395.
54. Maddux J.M., Kerfoot E.C., Chatterjee S., Holland P.C. Dissociation of attention in learning and action: effects of lesions of the amygdala central nucleus, medial prefrontal cortex, and posterior parietal cortex. Behav. Neurosci. 2007, 121:63-79.
55. Maunsell J.H., Treue S. Feature-based attention in visual cortex. Trends Neurosci. 2006, 29:317-322.
56. McCoy A.N., Platt M.L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat. Neurosci. 2005, 8:1220-1227.
57. Mirabella G., Bertini G., Samengo I., Kilavik B.E., Frilli D., Della Libera C., Chelazzi L. Neurons in area V4 of the macaque translate attended visual features into behaviorally relevant categories. Neuron 2007, 54:303-318.
58. Mirpour K., Arcizet F., Ong W.S., Bisley J.W. Been there, seen that: a neural mechanism for performing efficient visual search. J. Neurophysiol. 2009, 102:3481-3491.
59. Moore T., Armstrong K.M. Selective gating of visual signals by microstimulation of frontal cortex. Nature 2003, 421:370-373.
60. Morris G., Nevet A., Arkadir D., Vaadia E., Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 2006, 9:1057-1063.
61. Nassar M.R., Rumsey K.M., Wilson R.C., Parikh K., Heasly B., Gold J.I. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat. Neurosci. 2012, 15:1040-1046.
62. Niv Y., Schoenbaum G. Dialogues on prediction errors. Trends Cogn. Sci. 2008, 12:265-272.
63. Noudoost B., Moore T. Control of visual cortical signals by prefrontal dopamine. Nature 2011, 474:372-375.
64. O'Neill M., Schultz W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 2010, 68:789-800.
65. Oristaglio J., Schneider D.M., Balan P.F., Gottlieb J. Integration of visuospatial and effector information during symbolically cued limb movements in monkey lateral intraparietal area. J. Neurosci. 2006, 26:8310-8319.
66. Oudeyer P.Y., Kaplan F., Hafner V.V. Instrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 2007, 11:265-286.
67. Payzan-LeNestour E., Bossaerts P. Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings. PLoS Comput. Biol. 2011, 7:e1001048.
68. Pearce J.M., Mackintosh N.J. Two Theories of Attention: A Review and a Possible Integration 2010, Oxford University Press, New York.
69. Peck C.J., Jangraw D.C., Suzuki M., Efem R., Gottlieb J. Reward modulates attention independently of action value in posterior parietal cortex. J. Neurosci. 2009, 29:11182-11191.
70. Preuschoff K., Bossaerts P. Adding prediction risk to the theory of reward learning. Ann. N Y Acad. Sci. 2007, 1104:135-146.
71. Preuschoff K., Bossaerts P., Quartz S.R. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 2006, 51:381-390.
72. Preuschoff K., Quartz S.R., Bossaerts P. Human insula activation reflects risk prediction errors as well as risk. J. Neurosci. 2008, 28:2745-2752.
73. Purcell B.A., Schall J.D., Logan G.D., Palmeri T.J. From salience to saccades: multiple-alternative gated stochastic accumulator model of visual search. J. Neurosci. 2012, 32:3433-3446.
74. Redgrave P., Gurney K. The short-latency dopamine signal: a role in discovering novel actions?. Nat. Rev. Neurosci. 2006, 7:967-975.
75. Reynolds J.H., Heeger D.J. The normalization model of attention. Neuron 2009, 61:168-185.
76. Rigotti M., Ben Dayan Rubin D., Wang X.J., Fusi S. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Front. Comput. Neurosci. 2010, 4:24.
77. Rossi A.F., Pessoa L., Desimone R., Ungerleider L.G. The prefrontal cortex and the executive control of attention. Exp. Brain Res. 2009, 192:489-497.
78. Rothkopf C.A., Ballard D. Credit assignment in multiple goal embodied visuomotor behavior. Front. Psychol. 2010, 1:173.
79. Rothkopf C.A., Ballard D.H., Hayhoe M.M. Task and context determine where you look. J. Vis. 2007, 7:11-20.
80. Rushworth M.F., Noonan M.P., Boorman E.D., Walton M.E., Behrens T.E. Frontal cortex and reward-guided learning and decision-making. Neuron 2011, 70:1054-1069.
81. Saalmann Y.B., Kastner S. Cognitive and perceptual functions of the visual thalamus. Neuron 2011, 71:209-223.
82. Schall J.D., Purcell B.A., Heitz R.P., Logan G.D., Palmeri T.J. Neural mechanisms of saccade target selection: gated accumulator model of the visual-motor cascade. Eur. J. Neurosci. 2011, 33:1991-2002.
83. Schultz W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 2006, 57:87-115.
84. Schultz W., Preuschoff K., Camerer C., Hsu M., Fiorillo C.D., Tobler P.N., Bossaerts P. Explicit neural signals reflecting reward uncertainty. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2008, 363:3801-3811.
85. Snyder L.H., Batista A.P., Andersen R.A. Intention-related activity in the posterior parietal cortex: a review. Vision Res. 2000, 40:1433-1441.
86. So N., Stuphorn V. Supplementary eye field encodes reward prediction error. J. Neurosci. 2012, 32:2950-2963.
87. Sprague N., Ballard D.H. Modeling embodied visual behaviors. ACM Trans. Appl. Percept. 2005, 1:1-26.
88. Stanford T.R., Shankar S., Massoglia D.P., Costello M.G., Salinas E. Perceptual decision making in less than 30 milliseconds. Nat. Neurosci. 2010, 13:379-385.
89. Sugrue L.P., Corrado G.S., Newsome W.T. Matching behavior and the representation of value in the parietal cortex. Science 2004, 304:1782-1787.
90. Sugrue L.P., Corrado G.S., Newsome W.T. Choosing the greater of two goods: neural currencies for valuation and decision making. Nat. Rev. Neurosci. 2005, 6:363-375.
91. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
92. Takahashi Y.K., Roesch M.R., Wilson R.C., Toreson K., O'Donnell P., Niv Y., Schoenbaum G. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 2011, 14:1590-1597.
93. Tatler B.W., Hayhoe M.M., Land M.F., Ballard D.H. Eye guidance in natural vision: reinterpreting salience. J. Vis. 2011, 11:5-25.
94. Thompson K.G., Bichot N.P. A visual salience map in the primate frontal eye field. Prog. Brain Res. 2005, 147:251-262.
95. Thompson K.G., Biscoe K.L., Sato T.R. Neuronal basis of covert spatial attention in the frontal eye field. J. Neurosci. 2005, 25:9479-9487.
96. Tobler P.N., Christopoulos G.I., O'Doherty J.P., Dolan R.J., Schultz W. Risk-dependent reward value signal in human prefrontal cortex. Proc. Natl. Acad. Sci. USA 2009, 106:7185-7190.
97. Vilares I., Kording K. Bayesian models: the structure of the world, uncertainty, behavior, and the brain. Ann. N Y Acad. Sci. 2011, 1224:22-39.
98. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn. Sci. 2005, 9:585-594.
99. Waelti P., Dickinson A., Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412:43-48.
100. Walton M.E., Behrens T.E., Noonan M.P., Rushworth M.F. Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Ann. N Y Acad. Sci. 2011, 1239:14-24.
101. Wardak C., Olivier E., Duhamel J.R. A deficit in covert attention after parietal cortex inactivation in the monkey. Neuron 2004, 42:501-508.
102. Wardak C., Ibos G., Duhamel J.R., Olivier E. Contribution of the monkey frontal eye field to covert visual attention. J. Neurosci. 2006, 26:4228-4235.
103. Wilson R.C., Niv Y. Inferring relevance in a changing world. Front. Hum. Neurosci. 2011, 5:189.
104. Wittmann B.C., Daw N.D., Seymour B., Dolan R.J. Striatal activity underlies novelty-based choice in humans. Neuron 2008, 58:967-973.
105. Yakushijin R., Jacobs R.A. Are people successful at learning sequences of actions on a perceptual matching task?. Cogn. Sci. 2011, 35:939-962.
106. Yang T., Shadlen M.N. Probabilistic reasoning by neurons. Nature 2007, 447:1075-1080.
107. Yang H., Chen X., Zelinsky G.J. A new look at novelty effects: guiding search away from old distractors. Atten. Percept. Psychophys. 2009, 71:554-564.
108. Yu A.J., Dayan P. Uncertainty, neuromodulation, and attention. Neuron 2005, 46:681-692.