Learning of sequential movements by neural network model with dopamine-like reinforcement signal

Experimental Brain Research

Volumn 121, Issue 3, 1998, Pages 350-354

  Suri, Roland E ^a,b   Schultz, Wolfram ^a  

^a UNIVERSITY OF FRIBOURG   (Switzerland)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Basal ganglia; Eligibility; Synaptic plasticity; Teaching signal; Temporal difference

Indexed keywords

DOPAMINE;

ALGORITHM; ARTICLE; LEARNING; MOVEMENT (PHYSIOLOGY); NERVE CELL NETWORK; PERFORMANCE; PRIORITY JOURNAL; REINFORCEMENT; SIGNAL TRANSDUCTION;

ALGORITHMS; ANIMALS; BASAL GANGLIA; DOPAMINE; HUMANS; LEARNING; MOVEMENT; NEURAL NETWORKS (COMPUTER); NEURONAL PLASTICITY; SIGNAL TRANSDUCTION; TIME FACTORS;

EID: 0031854385     PISSN: 00144819     EISSN: None     Source Type: Journal    
DOI: 10.1007/s002210050467     Document Type: Article

References (31)

1. Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85:491-500
2. Barto AG (1995) Adaptive critics and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, Mass., pp 215-232
3. Barto AG, Sutton RS, Anderson CW (1983) Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans Systems Man Cyber SMC 13:834-846
4. Benecke R, Rothwell JC, Dick JPR, Day BL, Marsden CD (1987) Disturbance of sequential movements in Parkinson's disease. Brain 110:361-379
5. Calabresi P, Pisani A, Mercuri NB, Bernardi G (1992) Long-term potentiation in the striatum is unmasked by removing the voltage-dependent magnesium block of NMDA receptor channels. Eur J Neurosci 4:929-935
6. Calabresi P, Saiardi A, Pisani A, Baik JH, Centonze D, Mercuri NB, Bernardi G, Borrelli E (1997) Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J Neurosci 17:4536-4544
7. Dickinson A (1980) Contemporary animal learning theory. Cambridge University Press, Cambridge, Mass.
8. Dominey P, Arbib M, Joseph JP (1995) A model of corticostriatal plasticity for learning oculomotor associations and sequences. J Cogn Neurosci 7:311-336
9. Friston KJ, Tononi G, Reeke GN Jr, Sporns O, Edelman GM (1994) Value-dependant selection in the brain: simulation in a synthetic neural model. Neuroscience 59:229-243
10. Grossberg S, Levine DS (1987) Neural dynamics of attentionally modulated Pavlovian conditioning: conditioned reinforcement, inhibition and opponent processing. Psychobiology 15:195-240
11. Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61:814-832
12. Houk JC, Adams JL, Barto AG (1995) A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, Mass., pp 249-270
13. Kermadi I, Joseph JP (1995) Activity in the caudate nucleus during spatial sequencing. J Neurophysiol 74:911-933
14. Mirenowicz J, Schultz W (1994) Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72:1024-1027
15. Miyachi S, Hikosaka O, Miyashita K, Karadi Z, Rand MK (1997) Differential roles of monkey striatum in learning of sequential hand movement. Exp Brain Res 115:1-5
16. Montague PR, Sejnowski TJ (1994) The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learn Mem 1:1-33
17. Montague PR, Dayan P, Nowlan SJ, Pouget A, Sejnowski TJ (1993) Using aperiodic reinforcement for directed self-organization during development. In: Hanson SJ, Cowan JD, Giles CL (eds) Neural information processing systems 5. Morgan Kaufmann, San Mateo, pp 969-976
18. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive hebbian learning. J Neurosci 16:1936-1947
19. Mushiake H, Strick PL (1995) Pallidal neuron activity during sequential arm movements. J Neurophysiol 74:2754-2758
20. Niki H, Watanabe M (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171:213-224
21. Nishijo H, Ono T, Nishino H (1988) Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J Neurosci 8:3570-3583
22. Phillips JG, Bradshaw JL, Iansek R, Chiu E (1993) Motor functions of the basal ganglia. Psychol Res 55:175-181
23. Rescola RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In: Black AH, Prokasy WF (eds) Classical conditioning II. current research and theory. Appleton-Century-Crofts, New York, pp 64-99
24. Schultz W, Apicella P, Ljungberg T (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response taks. J Neurosci 13:900-913
25. Schultz W, Apicella P, Romo R, Scarnati E (1995) Context-dependent activity in primate striatum reflecting past and future behavioral events. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, Mass., pp 11-28
26. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599
27. Sutton RS, Barto AG (1990) Time derivative models of Pavlovian reinforcement. In: Gabriel M, Moore J (eds) Learning and compuational neuroscience: foundations of adaptive networks. MIT Press, Cambridge, Mass., pp 539-602
28. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press/Bradford Books, Cambridge, Mass.
29. Wickens J, Kötter R (1995) Cellular models of reinforcement. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, Mass., pp 187-214
30. Wickens JR, Begg AJ, Arbuthnott GW (1996) Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70:1-5
31. Watanabe M (1989) The appropriateness of behavioral responses coded in post-trial activity of primate prefrontal units. Neurosci Lett 101:113-117