Reward-dependent learning in neuronal networks for planning and decision making

Progress in Brain Research

Volumn 126, Issue , 2000, Pages 217-229

  Dehaene, Stanislas ^a   Changeux, Jean Pierre ^b  

^a DIF   (France)

^b INSTITUT PASTEUR   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ANTICIPATION; COGNITION; CONFERENCE PAPER; DECISION MAKING; HUMAN; HUMAN EXPERIMENT; LEARNING; NERVE CELL NETWORK; PLANNING; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; REWARD; SYNAPTIC TRANSMISSION; WISCONSIN CARD SORTING TEST;

EID: 0034589621     PISSN: 00796123     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S0079-6123(00)26016-0     Document Type: Conference Paper

References (44)

1. Carter, C.S., Braver, T.S., Barch, D., Botvinick, M.M., Noll, D. and Cohen, J.D. (1998) Anterior cingulate cortex, error detection, and the online monitoring of performance. Science, 280: 747-749.
2. Changeux, J.P. (1981) The acetylcholine receptor: An allosteric membrane protein. Harvey Lectures, 75: 85-254.
3. Changeux, J.P. and Dehaene, S. (1989) Neuronal models of cognitive functions. Cognition, 33: 63-109.
4. Cohen, J.D., Dunbar, K. and McClelland, J. (1990) On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychol. Rev., 97: 332-361.
5. Cohen, J.D., Perlstein, W.M., Braver, T.S., Nystrom, L.E., Noll, D.C., Jonides, J. and Smith, E.E. (1997) Temporal dynamics of brain activation during a working memory task. Nature, 386: 604-608.
6. Corbetta, M., Miezin, F.M., Dobmeyer, S., Smulman, G.L. and Petersen, S.E. (1991) Selective and divided attention during visual discriminations of shape color and speed: Functional anatomy by positron emission tomography. J. Neurosci., 11: 2383-2402.
7. Damasio, A.R. (1994) Descartes' Error: Emotion, Reason, and the Human Brain. NY: G.P. Putnam, New York.
8. Dehaene, S. and Changeux, J.P. (1989) A simple model of prefrontal cortex function in delayed-response tasks. J. Cogn. Neurosci., 1: 244-261.
9. Dehaene, S. and Changeux, J.P. (1991) The Wisconsin Card Sorting Test: Theoretical analysis and modelling in a neuronal network. Cereb. Cort., 1: 62-79.
10. Dehaene, S. and Changeux, J.P. (1993) Development of elementary numerical abilities: A neuronal model. J. Cogn. Neurosci., 5: 390-407.
11. Dehaene, S. and Changeux, J.P. (1996) Neuronal models of prefrontal cortical functions. Ann. NY Acad. Sci., 769: 305-319.
12. Dehaene, S. and Changeux, J.P. (1997) A hierarchical neuronal network for planning behavior. Proc. Natl. Acad. Sci. USA, 94: 13293-13298.
13. Dehaene, S., Kerszberg, M. and Changeux, J.P. (1998) A neuronal model of a global workspace in effortful cognitive tasks. Proc. Natl. Acad. Sci. USA, 95: 14529-14534.
14. Dehaene, S., Posner, M.I. and Tucker, D.M. (1994) Localization of a neural system for error detection and compensation. Psycholog. Sci., 5: 303-305.
15. Edelman, G.M. (1987) Neural Darwinism. New York: Basic Books.
16. Edelman, G.M. (1993) Neural Darwinism: Selection and reentrant signaling in higher brain function. Neuron, 10: 115-125.
17. Eslinger, P.J. and Damasio, A.R. (1985) Severe disturbance of higher cognition after bilateral frontal lobe ablation: Patient EVR. Neurology, 35: 1731-1741.
18. Friston, K.J., Tononi, G., Reeke, G.N., Sporns, O. and Edelman, G.M. (1994) Value-dependent selection in the brain: Simulation in a synthetic neural model. Neuroscience, 59: 229-243.
19. Fuster, J.M. (1989) The Prefrontal Cortex. Raven: New York.
20. Kimberg, D.Y. and Farah, M.J. (1993) A unified account of cognitive impairments following frontal lobe damage: the role of working memory in complex organized behavior. J. Exp. Psychol.: Gen., 122: 411-428.
21. Konishi, S., Nakajima, K., Uchida, I., Kameyama, M., Nakahara, K., Sekihara, K. and Miyashita, Y. (1998) Transient activation of inferior prefrontal cortex during cognitive set shifting. Nature Neurosci., 1: 80-84.
22. Levine, D.S. and Prueitt, P.S. (1989) Modelling some effects of frontal lobe damage - Novelty and perseveration. Neur Net., 2: 103-116.
23. Luria, A.R. (1966) The Higher Cortical Functions in Man. New York: Basic Books.
24. Montague, P.R., Dayan, P., Person, C. and Sejnowski, T. (1995) Bee foraging in uncertain environments using predictive hebbian learning. Nature, 377: 725-728.
25. Montague, P.R., Dayan, P. and Sejnowski, T.J. (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16: 1936-1947.
26. Morris, R.G., Downes, J.J., Evenden, J.L., Sahakian, B.J., Heald, A. and Robbins, T.W. (1988) Planning and spatial working memory in Parkinson's disease. J. Neurol., Neurosurg. Psychiatry, 51: 757-766.
27. Niki, H. and Watanabe, M. (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res., 171: 213-224.
28. Owen, A.M., James, M., Leigh, P.N., Summers, B.A., Quinn, N.P., Marsden, C.D. and Robbins, T.W. (1992) Fronto-striatal cognitive deficits at different stages of Parkinson's disease. Brain, 115: 1727-1751.
29. Owen, A.M., Sahakian, B.J., Hodges, J.R., Summers, B.A., Polkey, C.E. and Robbins, T. W. (1995) Dopamine-dependent fronto-striatal planning deficits in early Parkinson's disease. Neuropsychology, 9: 126-140.
30. Pardo, J.V., Pardo, P.J., Janer, K.W. and Raichle, M.E. (1990) The anterior cingulate cortex mediates processing selection in the Stroop attentional conflict paradigm. Proc. Natl. Acad. Sci. USA, 87: 256-259.
31. Paus, T., Koski, L., Caramanos, Z. and Westbury, C. (1998) Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: A review of 107 PET activation studies. NeuroReport, 9: R37-R47.
32. Pennartz, C.M. (1996) The ascending neuromodulatory systems in learning by reinforcement: Comparing computational conjectures with experimental findings. Brain Res. Rev., 21: 219-245.
33. Platt, M.L. and Glimcher, P.W. (1999) Neural correlates of decision variables in parietal cortex. Nature, 400: 233-238.
34. Posner, M.I. and Dehaene, S. (1994) Attentional networks. Trends Neurosci., 17: 75-79.
35. Raichle, M.E., Fiez, J.A., Videen, T.O., MacLeod, A.K., Pardo, J.V., Fox, P.T. and Petersen, S.E. (1994) Practice-related changes in human brain functional anatomy during nonmotor learning. Cereb. Corr., 4: 8-26.
36. Redgrave, P., Prescott, T.J. and Gurney, K. (1999) Is the short-latency dopamine response too short to signal reward error? Trends Neurosci., 2: 146-151.
37. Schultz, W., Apicella, P. and Ljungberg, T. (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci., 13: 900-913.
38. Schultz, W., Dayan, P. and Montague, P.R. (1997) A neural substrate of prediction and reward. Science, 275:1593-1599.
39. Shallice, T. (1982) Specific impairments of planning. Philosoph. Trans. R. Soc. (Lond.), Ser. B, 298: 199-209.
40. Shallice, T. (1988) From Neuropsychology to Mental Structure. Cambridge University Press.
41. Spanagel, R. and Weiss, F. (1999) The dopamine hypothesis of reward: Past and current status. Trends Neurosci., 22: 521-527.
42. Sutton, R.S. and Barto, A.G. (1998) Reinforcement Learning: an Introduction. MIT Press: Cambridge, Mass.
43. Watanabe, M. (1996) Reward expectancy in primate prefrontal neurons. Nature, 382: 629-632.
44. Williams, S.M. and Goldman-Rakic, P.S. (1993) Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb. Cort., 3: 199-222.