Reinforcement learning can account for associative and perceptual learning on a visual-decision task

Nature Neuroscience

Volumn 12, Issue 5, 2009, Pages 655-663

  Law, Chi Tat ^a   Gold, Joshua I ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; DECISION MAKING; ERROR; FEEDBACK SYSTEM; LEARNING; MONKEY; MOTION; NONHUMAN; PERCEPTION; PERCEPTIVE DISCRIMINATION; PREDICTION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; SENSORY NERVE CELL; STATE DEPENDENT LEARNING; TASK PERFORMANCE; TEMPORAL CORTEX; VISION;

ANIMALS; ASSOCIATION LEARNING; COMPUTER SIMULATION; DECISION MAKING; FEEDBACK; HAPLORHINI; MODELS, NEUROLOGICAL; MOTION PERCEPTION; NEURAL NETWORKS (COMPUTER); NEURAL PATHWAYS; NEUROPSYCHOLOGICAL TESTS; PHOTIC STIMULATION; PSYCHOMOTOR PERFORMANCE; REINFORCEMENT (PSYCHOLOGY); REWARD; VISUAL PERCEPTION;

EID: 67349117811     PISSN: 10976256     EISSN: None     Source Type: Journal    
DOI: 10.1038/nn.2304     Document Type: Article

References (50)

1. Gibson, E.J. Perceptual learning. Annu. Rev. Psychol. 14, 29-56 (1963).
2. Gilbert, C.D., Sigman, M. & Crist, R.E. The neural basis of perceptual learning. Neuron 31, 681-697 (2001).
3. Mukai, I. et al. Activations in visual and attention-related areas predict and correlate with the degree of perceptual learning. J. Neurosci. 27, 11401-11411 (2007).
4. Law, C.T. & Gold, J.I. Neural correlates of perceptual learning in a sensory-motor, but not a sensory, cortical area. Nat. Neurosci. 11, 505-513 (2008).
5. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction (MIT Press, Cambridge, Massachusetts, 1998).
6. Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87-115 (2006).
7. Schultz, W. Multiple dopamine functions at dlfferenttime courses. Annu.Rev. Neurosci. 30, 259-288 (2007).
8. Berridge, K.C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl.) 191, 391-431 (2007).
9. Redgrave, P., Gurney, K. & Reynolds, J. What is reinforced by phasic dopamine signals? Brain Res. Rev. 58, 322-339 (2008).
10. Bao, S., Chan, V.T. & Merzenich, M.M. Cortical remodeling induced by activity of ventral tegmental dopamine neurons. Nature 412, 79-83 (2001).
11. Seitz, A. & Watanabe, T. A unified model for perceptual learning. Trends Cogn. Sci. 9, 329-334 (2005).
12. Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral Intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475-9489 (2002).
13. Hanks, T.D., Ditterich, J. & Shadlen, M.N. Microstimulation of macaque area LIP affects decision-making in a motion discrimination task. Nat. Neurosci. 9, 682-689 (2006).
14. Piatt, M.L. & Glimcher, P.W. Neural correlates of decision variables In parietal cortex. Nature 400, 233-238 (1999).
15. Andersen, R.A. & Buneo, C.A. Intentional maps in posterior parietal cortex. Annu. Rev. Neurosci. 25, 189-220 (2002).
16. Colby, C.L. & Goldberg, M.E. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22, 319-349 (1999).
17. Gold, J.I. & Shadlen, M.N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535-574 (2007).
18. Salzman, C.D., Britten, K.H. & Newsome, W.T. Cortical microstimulation influences perceptual judgments of motion direction. Nature 346, 174-177 (1990).
19. Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745-4765 (1992).
20. Williams, R.J. Simple statistical gradient-following algorithms for connectionlst reinforcement learning. Mach. Learn. 8, 229-256 (1992).
21. Shadlen, M.N., Britten, K.H., Newsome, W.T. & Movshon, J.A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486-1510 (1996).
22. Dayan, P. & Daw, N.D. Decision theory, reinforcement learning and the brain. Cogn. Affect. Behav. Neurosci. 8, 429-453 (2008).
23. Zohary, E., Shadlen, M.N. & Newsome, W.T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140-143 (1994).
24. Bair, W., Zohary, E. & Newsome, W.T. Correlated firing In macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676-1697 (2001).
25. Barto, A.G. & Anadan, P. Pattern recognizing stochastic learning automata. IEEE Trans. Syst. Man Cybern. 28, 360-374 (1985).
26. Loewenstein, Y. & Seung, H.S. Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc. Natl. Acad. Sci. USA 103, 15224-15229 (2006).
27. Mazurek, M.E., Roitman, J.D., Ditterich, J. & Shadlen, M.N. A role for neural integrators In perceptual decision making. Cereb. Cortex 13, 1257-1269 (2003).
28. Kepecs, A., Uchida, N., Zariwala, H.A. & Mainen, Z.F. Neural correlates, computation and behavioral impact of decision confidence. Nature 455, 227-231 (2008).
29. Gold, J.I. & Shadlen, M.N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5, 10-16 (2001).
30. Kiani, R., Hanks, T.D. & Shadlen, M.N. Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J. Neurosci. 28, 3017-3029 (2008).
31. Spiegelhalter, D.J. & Lauritzen, S.L. Sequential updating of conditional probabilities on directed graphical structures. Networks 20, 579-605 (1990).
32. Royer, S. & Pare, D. Conservation of total synaptic weight through balanced synaptic depression and potentiation. Nature422, 518-522 (2003).
33. Connolly, P.M., Bennur, S. & Gold, J. I. Correlates of perceptual learning in an oculomotor decision variable. J. Neurosci. 29, 2136-2150 (2009).
34. Purushothaman, G. & Bradley, D.C. Neural population code for fine perceptual decisions in area MT. Nat. Neurosci. 8, 99-106 (2005).
35. Britten, K.H., Newsome, W.T., Shadlen, M.N., Celebrini, S. & Movshon, J.A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87-100 (1996).
36. Sompolinsky, H., Yoon, H., Kang, K. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E 64, 051904 (2001).
37. Smith, M.A. & Kohn, A. Spatial and temporal scales of neuronal correlation in primary visual cortex. J. Neurosci. 28, 12591-12603 (2008).
38. Gold, J.I. & Shadlen, M.N. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J. Neurosci. 23, 632-651 (2003).
39. Jazayeri, M. & Movshon, J.A. A new perceptual illusion reveals mechanisms of sensory decoding. Nature 446, 912-915 (2007).
40. Hol, K. & Treue, S. Different populations of neurons contribute to the detection and discrimination of visual motion. Vision Res. 41, 685-689 (2001).
41. Ball, K. & Sekuler, R. Direction-specific improvement in motion discrimination. Vision Res. 27, 953-965 (1987).
42. Fahle, M. Perceptual learning: specificity versus generalization. Curr. Opin. Neurobiol. 15, 154-160 (2005).
43. Ahissar, M. & Hochstein, S. Task difficulty and the specificity of perceptual learning. Nature 387, 401-406 (1997).
44. Newsome, W.T. & Pare, E.B. A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci. 8, 2201-2211 (1988).
45. Celebrini, S. & Newsome, W.T. Neuronal and psychophysical sensitivity to motion signals in extrastriate area MST of the macaque monkey. J. Neurosci. 14, 4109-4124 (1994).
46. Shamir, M. & Sompolinsky, H. Correlation codes in neuronal networks. in Advances in Neural Information Processing Systemsleus. Dietterich, T.G., Becker, S. & Ghahramanl, Z.) (MIT Press, Cambridge, Massachusetts, 2002).
47. Yoon, H. & Sompolinsky, H. The effect of correlations on the fisher information of population codes, in Advances in Neural Information Processing Systems (eds. Kearns, M.S., Solla, S.A. & Cohn, D.A.) 167-173 (MIT Press, Cambridge, Massachusetts, 1998).
48. van Veen, V. & Carter, C.S. Error detection, correction, and prevention In the brain: a brief review of data and theories. Clin. EEG Neurosci. 37, 330-335 (2006).
49. Dosher, B.A. & Lu, Z.L. Mechanisms of perceptual learning. Vision Res. 39, 3197-3221 (1999).
50. Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. Responses of neurons In macaque MT to stochastic motion signals. Vis. Neurosci. 10, 1157-1169 (1993).