Predictive reward signal of dopamine neurons

Journal of Neurophysiology

Volumn 80, Issue 1, 1998, Pages 1-27

  Schultz, Wolfram ^a  

^a UNIVERSITY OF FRIBOURG   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIOR; CONDITIONING; DOPAMINERGIC NERVE CELL; DOPAMINERGIC SYSTEM; DOPAMINERGIC TRANSMISSION; LEARNING; PRIORITY JOURNAL; REVIEW; REWARD;

EID: 0031867046     PISSN: 00223077     EISSN: None     Source Type: Journal    
DOI: 10.1152/jn.1998.80.1.1     Document Type: Review

References (302)

1. ABERCROMBIE, E. D., KEEFE, K. A., DIFRISCHIA, D. S., AND ZIGMOND, M. J. Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. J. Neurochem. 52: 1655-1658, 1989.
2. AHLENIUS, S. Effects of low and high doses of L-dopa on the tetrabenazine or α-methyltyrosine-induced suppression of behaviour in a successive discrimination task. Psychopharmacologia 39: 199-212, 1974.
3. ALEXANDER, G. E., DELONG, M. R., AND STRICK, P. L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9: 357-381, 1986.
4. AMALRIC, M. AND KOOB, G. F. Depletion of dopamine in the caudate nucleus but not in nucleus accumbens impairs reaction-time performance. J. Neurosci. 7: 2129-2134, 1987.
5. ANDEN, N. E., FUXE, K., HAMBERGER, B., AND HÖKFELT, T. A quantitative study on the nigro-neostriatal dopamine neurons. Acta Physiol. Scand. 67: 306-312, 1966.
6. ANGLADE, P., MOUATT-PRIGENT, A., AGID, Y., AND HIRSCH, E. C. Synaptic plasticity in the caudate nucleus of patients with Parkinson's disease. Neurodegeneration 5: 121-128, 1996.
7. AOSAKI, T., TSUBOKAWA, H., ISHIDA, A., WATANABE, K., GRAYBIEL, A. M., AND KIMURA, M. Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J. Neurosci. 14: 3969-3984, 1994.
8. APICELLA, P., LEGALLET, E., AND TROUCHE, E. Responses of tonically discharging neurons in monkey striatum to visual stimuli presented under passive conditions and during task performance. Neurosci. Lett. 203: 147-150, 1996.
9. APICELLA, P., LEGALLET, E., AND TROUCHE, E. Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioral states. Exp. Brain Res. 116: 456-466, 1997.
10. APICELLA, P., LJUNGBERG, T., SCARNATI, E., AND SCHULTZ, W. Responses to reward in monkey dorsal and ventral striatum. Exp. Brain Res. 85: 491-500, 1991a.
11. APICELLA, P., SCARNATI, E., LJUNGBERG, T. AND SCHULTZ, W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J. Neurophysiol. 68: 945-960, 1992.
12. APICELLA, P., SCARNATI, E., AND SCHULTZ, W. Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli. Exp. Brain Res. 84: 672-675, 1991b.
13. ARBIB, M. A. AND DOMINEY, P. F. Modeling the roles of the basal ganglia in timing and sequencing saccadic eye movements. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 149-162.
14. ARBUTHNOTT, G. W. AND INGHAM, C. A. The thorny problem of what dopamine does in psychiatric disease. Prog. Brain Res. 99: 341-350, 1993.
15. ASTON-JONES, G. AND BLOOM, F. E. Norepinephrine-containing locus coeruleus neurons in behaving rats exhibit pronounced responses to nonnoxious environmental stimuli. J. Neurosci. 1: 887-900, 1981.
16. ASTON-JONES, G., RAJKOWSKI, J., AND KUBIAK, P. Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience 80: 697-716, 1997.
17. ASTON-JONES, G., RAJKOWSKI, J., KUBIAK, P., AND ALEXINSKY, T. Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. J. Neurosci. 14: 4467-4480, 1994.
18. BALLARD, D. H. An Introduction to Neural Computing. Cambridge, MA: MIT Press, 1997.
19. BARTO, A. G. Adaptive critics and the basal ganglia. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 215-232.
20. BARTO, A. G. AND ANANDAN, P. Pattern-recognizing stochastic learning automata. IEEE Trasnact. Syst. Man Cybern. 15: 360-375, 1985.
21. BARTO, A. G., SUTTON, R. S., AND ANDERSON, C. W. Neuronlike adaptive elements that can solve difficult learning problems. IEEE Trans Syst. Man Cybernet. 13: 834-846, 1983.
22. BENINGER, R. J. The role of dopamine in locomotor activity and learning. Brain Res. Rev. 6: 173-196, 1983.
23. BENINGER, R. J. Dissociating the effects of altered dopaminergic function on performance and learning. Brain Res. Bull. 23: 365-371, 1989.
24. BENINGER R. J., CHENG, M., HAHN, B. L., HOFFMAN, D. C., AND MAZURSKI, E. J. Effects of extinction, pimozide, SCH 23390 and metoclopramide on food-rewarded operant responding of rats. Psychopharmacology 92: 343-349, 1987.
25. BENINGER, R. J. AND HAHN, B. L. Pimozide blocks establishment but not expression of amphetamine-produced environment-specific conditioning. Science 220: 1304-1306, 1983.
26. BERENDSE, H. W., GROENEWEGEN, H. J., AND LOHMAN, A. H. M. Compartmental distribution of ventral striatal neurons projecting to the mesencephalon in the rat. J. Neurosci. 12: 2079-2103, 1992.
27. BERGER, B., TROTTIER, S., VERNEY, C., GASPAR, P., AND ALVAREZ, C. Regional and laminar distribution of the dopamine and serotonin innervation in the macaque cerebral cortex: a radioautographic study. J. Comp. Neurol. 273:99-119, 1988.
28. 5 dopamine receptors in primate brain. J. Neurosci. 15: 7821-7836, 1995.
29. BERTHIER, N. E. AND MOORE, J. W. Activity of deep cerebellar nuclear cells during classical conditioning of nictitating membrane extension in rabbits. Exp. Brain Res. 83: 44-54, 1990.
30. BINDRA, D. Neuropsychological interpretation of the effects of drive and incentive-motivation on general activity and instrumental behavior. Psychol. Rev. 75: 1-22, 1968.
31. BLANTON, M. G. AND KRIEGSTEIN, A. R. Properties of amino-acid neuro-transmitter receptors of embryonic cortical neurons when activated by exogenous and endogenous agonists. J. Neurophysiol. 67: 1185-1200, 1992.
32. BOEIJINGA, P. H., MULDER, A. B., PENNARTZ, C.M.A., MANSHANDEN, I., AND LOPES DA SILVA, F. H. Responses of the nucleus accumbens following fornix/fimbria stimulation in the rat. Identification and long-term potentiation of mono- and polysynaptic pathways. Neuroscience 53: 1049-1058, 1993.
33. BOLAM, J. P., FRANCIS, C. M., AND HENDERSON, Z. Cholinergic input to dopamine neurons in the substantia nigra: a double immunocytochemical study. Neuroscience 41: 483-494, 1991.
34. BOLLES, R. C. Reinforcement, expectancy and learning. Psychol. Rev. 79: 394-409, 1972.
35. BOWMAN, E. M., AIGNER, T. G., AND RICHMOND, B. J. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J. Neurophysiol. 75: 1061-1073, 1996.
36. BOUYER, J. J., PARK, D. H., JOH, T. H., AND PICKEL, V. M. Chemical and structural analysis of the relation between cortical inputs and tyrosine hydroxylase-containing terminals in rat neostriatum. Brain Res. 302: 267-275, 1984.
37. 2) receptors: a study of the actions of dopamine on corticostriatal transmission. Neuroscience 10: 349-355, 1983.
38. BROZOSKI, T. J., BROWN, R. M., ROSVOLD, H. E., AND GOLDMAN, P. S. Cognitive deficit caused by regional depletion of dopamine in prefrontal cortex of rhesus monkey. Science 205: 929-932, 1979.
39. CAILLÉ, I., DUMARTIN, B., AND BLOCK, B. Ultrastructural localization of D1 dopamine receptor immunoreactivity in rat striatonigral neurons and its relation with dopaminergic innervation. Brain Res. 730: 17-31, 1996.
40. CALABRESI, P., MAJ, R., PISANI, A., MERCURI, N. B., AND BERNARDI, G. Long-term synaptic depression in the striatum: physiological and pharmacological characterization. J. Neurosci. 12: 4224-4233, 1992a.
41. CALABRESI, P., PISANI, A., MERCURI, N. B., AND BERNARDI, G. Long-term potentiation in the striatum is unmasked by removing the voltage-dependent magnesium block of NMDA receptor channels. Eur. J. Neurosci. 4: 929-935, 1992b.
42. CALABRESI, P., SAIARDI, A., PISANI, A., BAIK, J. H., CENTONZE, D., MERCURI, N. B., BERNARDI, G., AND BORELLI, E. Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J. Neurosci. 17: 4536-4544, 1997.
43. CANAVAN, A.G.M., PASSINGHAM, R. E., MARSDEN, C. D., QUINN, N., WYKE, M., AND POLKEY, C. E. The performance on learning tasks of patients in the early stages of Parkinson's disease. Neuropsychologia 27: 141-156, 1989.
44. CEPEDA, C., BUCHWALD, N. A., AND LEVINE, M. S. Neuromodulatory actions of dopamine in the neostriatum are dependent upon the excitatory aminao acid receptor subtypes activated. Proc. Natl. Acad. Sci. USA 90: 9576-9580, 1993.
45. + conductance in medium-sized neostriatal neurons: characterization using infrared videomicroscopy and whole cell patch-clamp recordings. J. Neurophysiol. 74: 1343-1348, 1995.
46. CEPEDA, C., COLWELL, C. S., ITRI, J. N., CHANDLER, S. H., AND LEVINE, M. S. Dopaminergic modulation of NMDA-induced whole cell currents in neostriatal neurons in slices: contribution of calcium conductances. J. Neurophysiol. 79: 82-94, 1998.
47. CHERGUI, K., SUAUD-CHAGNY, M. F., AND GONON, F. Nonlinear relationship between impulse flow, dopamine release and dopamine elimination in the rat brain in vivo. Neurocience 62: 641-645, 1994.
48. CHESSELET, M. F. Presynaptic regulation of neurotransmitter release in the brain: facts and hypothesis. Neuroscience 12: 347-375, 1984.
49. CHURCH, W. H., JUSTICE, J. B., JR., AND BYRD, L. D. Extracellular dopamine in rat striatum following uptake inhibition by cocaine, nomifensine and benztropine. Eur. J. Pharmacol. 139: 345-348, 1987.
50. CHURCH, W. H., JUSTICE, J. B., JR., ANDNEILL, D. B. Detecting behaviorally relevant changes in extracellular dopamine with microdialysis. Brain Res. 412: 397-399, 1987.
51. CLARKE, P.B.S., HOMMER, D. W., PERT, A., AND SKIRBOLL, L. R. Innervation of substantia nigra neurons by cholinergic afferents from pedunculopontine nucleus in the rat: neuroanatomical and electrophysiological evidence. Neuroscience 23: 1011-1019, 1987.
52. COLLE, W. M. AND WISE, R. A. Effects of nucleus accumbens amphetamine on lateral hypothalamus brain stimulation reward. Brain Res. 459: 356-360, 1980.
53. CONTRERAS-VIDAL, J. L. AND SCHULTZ, W. A neural network model of reward-related learning, motivation and orienting behavior. Soc. Neurosci. Abstr. 22: 2029, 1996.
54. CORBETT, D. AND WISE, R. A. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable microelectrode study. Brain Res. 185: 1-15, 1980.
55. CORVAJA, N., DOUCET, G., AND BOLAM, J. P. Ultrastructure and synaptic targets of the raphe-nigral projection in the rat. Neuroscience 55: 417-427, 1993.
56. DEHAENE, S. AND CHANGEUX, J.-P. The Wisconsin Card Sorting Test: theoretical analysis and modeling in a neuronal network. Cerebr. Cortex 1: 62-79, 1991.
57. DELANEY, K. AND GELPERIN, A. Post-ingestive food-aversion learning to amino acid deficient diets by the terrestrial slug Limax maximus. J. Comp. Physiol. [A] 159: 281-295, 1986.
58. DELONG, M. R., CRUTCHER, M. D., AND GEORGOPOULOS, A. P. Relations between movement and single cell discharge in the substantia nigra of the behaving monkey. J. Neurosci. 3: 1599-1606, 1983.
59. DESCARRIES, L., WATKINS, K. C., GARCIA, S., BOSLER, O., AND DOUCET, G. Dual character, asynaptic and synaptic, of the dopamine innervation in adult rat neostriatum: a quantitative autoradiogrphic and immunocytochemical analysis. J. Comp. Neurol. 375: 167-186, 1996.
60. DI CHIARA, G. The role of dopamine in drug abuse viewed from the perspective of its role in motivation. Drug Alcohol Depend. 38: 95-137, 1995.
61. DICKINSON, A. Contemporary Animal Learning Theory. Cambridge, UK: Cambridge Univ. Press, 1980.
62. DICKINSON, A. AND BALLEINE, B. Motivational control of goal-directed action. Anim. Learn. Behav. 22: 1-18, 1994.
63. DICKINSON, A., HALL, G., AND MACKINTOSH, N. J. Surprise and the attenuation of blocking. J. Exp. Psychol. Anim. Behav. Proc. 2: 313-322, 1976.
64. DOHERTY, M. D. AND GRATTON, A. High-speed chronoamperometric measurements of mesolimbic and nigrostriatal dopamine release associated with repeated daily stress. Brain Res. 586: 295-302, 1992.
65. DOMINEY, P., ARBIB, M., AND JOSEPH, J.-P. A model of corticostriatal plasticity for learning oculomotor associations and sequences. J. Cognit. Neurosci. 7: 311-336, 1995.
66. DOUCET, G., DESCARRIES, L., AND GARCIA, S. Quantification of the dopamine innervation in adult rat neostriatum. Neuroscience 19: 427-445, 1986.
67. DUGAST, C., SUAUD-CHAGNY, M. F., AND GONON, F. Continuous in vivo monitoring of evoked dopamine release in the rat nucleus accumbens by amperometry. Neuroscience 62: 647-654, 1994.
68. EDMONDS, D. E. AND GALLISTEL, C. R. Reward versus performance in self-stimulation: electrode-specific effects of α-methyl-p-tyrosine on reward in the rat. J. Comp. Physiol. Psychol. 91: 962-974, 1977.
69. ELLIOTT, R., SAHAKIAN, B. J., MATTHEWS, K., BANNERJEA, A., RIMMER, J., AND ROBBINS, T. W. Effects of methylphenidate on spatial working memory and planning in healthy young adults. Psychopharmacology 131: 196-206, 1997.
70. ETTENBERG, A. Dopamine, neuroleptics and reinforced behavior. Neurosci. Biobehav. Rev. 13: 105-111, 1989.
71. FAGG, A. H. Reinforcement learning for robotic reaching and grasping. In: New Perspectives in the Control of the Reach to Grasp Movement, edited by K.M.B. Bennet and U. Castiello. Amsterdam: North-Holland, 1993, p. 281-308.
72. FAGG, A. H. AND ARBIB, M. A. A model of primate visual-motor conditional learning. Adapt. Behav. 1: 3-37, 1992.
73. FIBIGER, H. C., LEPIANE, F. G., JAKUBOVIC, A., AND PHILLIPS, A. G. The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J. Neurosci. 7: 3888-3896, 1987.
74. FIBIGER, H. C. AND MILLER, J. J. An anatomical and electrophysiological investigation of the serotonergic projection from the dorsal raphé nucleus to the substantia nigra in the rat. Neuroscience 2: 975-987, 1977.
75. FIBIGER, H. C. AND PHILLIPS, A. G. Reward, motivation, cognition: psychobiology of mesotelencephalic dopamine systems. In: Handbook of Physiology. The Nervous System. Intrinsic Regulatory Systems of the Brain. Bethesda, MA: Am. Physiol. Soc., 1986, sect. 1, vol. IV, p. 647-675.
76. FILION, M., TREMBLAY, L., AND BÉDARD, P. J. Abnormal influences of passive limb movement on the activity of globus pallidus neurons in parkinsonian monkey. Brain Res. 444: 165-176, 1988.
77. FLAHERTY, A. W. AND GRAYBIEL, A. Two input systems for body representations in the primate striatal matrix: experimental evidence in the squirrel monkey. J. Neurosci. 13: 1120-1137, 1993.
78. FLAHERTY, A. W. AND GRAYBIEL, A. Input-output organization of the sensorimotor striatum in the squirrel monkey. J. Neurosci. 14: 599-610, 1994.
79. FLOWERS, K. AND DOWNING, A. C. Predictive control of eye movements in Parkinson disease. Ann. Neurol. 4: 63-66, 1978.
80. FOOTE, S. L., ASTON-JONES, G., AND BLOOM, F. E. Impulse activity of locus coeruleus neurons in awake rats and monkeys is a function of sensory stimulation and arousal. Proc. Natl. Acad. Sci. USA 77: 3033-3037, 1980.
81. FREUND, T. F., POWELL, J. F., AND SMITH, A. D. Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13: 1189-1215, 1984.
82. FREY, U., SCHROEDER, H., AND MATTHIES, H. Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of hippocampal slices. Brain Res. 522: 69-75, 1990.
83. FRISTON, K. J., TONONI, G., REEKE, G. N., JR., SPORNS, O., AND EDELMAN, G. M. Value-dependent selection in the brain: simulation in a synthetic neural model. Neuroscience 59: 229-243, 1994.
84. FUJITA, K. Species recognition by five macaque monkeys. Primates 28: 353-366, 1987.
85. FURIEZOS, G. AND WISE, R. A. Pimozide-induced extinction of intracranial self-stimulation: response patterns rule out motor or performance deficits. Brain Res. 103: 377-380, 1976.
86. FUTAMI, T., TAKAKUSAKI, K., AND KITAI, S. T. Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta. Neurosci. Res. 21: 331-342, 1995.
87. GALLISTEL, C. R. The Organization of Learning. Cambridge, MA: MIT Press, 1990.
88. GARCIA, C. E., PRETT, D. M., AND MORARI, M. Model predictive control: theory and practice - a survey. Automatica 25: 335-348, 1989.
89. GARCIA-MUNOZ, M., YOUNG, S. J., AND GROVES, P. Presynaptic long-term changes in excitability of the corticostriatal pathway. Neuroreport 3: 357-360, 1992.
90. GARIANO, R. F. AND GROVES, P. M. Burst firing in midbrain dopamine neurons by stimulation of the medial prefrontal and anterior cingulate cortices. Brain Res. 462: 194-198, 1988.
91. GARRIS, P. A., CIOLKOWSKI, E. L., PASTORE, P., AND WIGHTMAN, R. M. Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain. J. Neurosci. 14: 6084-6093, 1994a.
92. GARRIS, P. A., CIOLKOWSKI, E. L., AND WIGHTMAN, R. M. Heterogeneity of evoked dopamine overflow within the striatal and striatoamygdaloid regions. Neuroscience 59: 417-427, 1994b.
93. GARRIS, P. A. AND WIGHTMAN, R. M. Different kinetics govern dopaminergic transmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study. J. Neurosci. 14: 442-450, 1994.
94. GERFEN, C. R. The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems. Nature 311: 461-464, 1984.
95. GERFEN, C. R., ENGBER, T. M., MAHAN, L. C., SUSEL, Z., CHASE, T. N., MONSMA, F. J., JR., AND SIBLEY, D. R. D1 and D2 dopamine receptor-regulated gene expression of striatonigral and striatopallidal neurons. Science 250: 1429-1432, 1990.
96. GERMAN, D. C., DUBACH, M., ASKARI, S., SPECIALE, S. G., AND BOWDEN, D. M. 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP)-induced parkinsonian syndrome in macaca fascicularis: which midbrain dopaminergic neurons are lost? Neuroscience 24: 161-174, 1988.
97. GILBERT, P.F.C. AND THACH, W. T. Purkinje cell activity during motor learning. Brain Res. 128: 309-328, 1977.
98. GIROS, B., JABER, M., JONES, S. R., WIGHTMAN, R. M., AND CARON, M. G. Hyperlocomotion and indifference to cocaine and amphetamine in mice lacking the dopamine transporter. Nature 379: 606-612, 1996.
99. GOLDMAN-RAKIC, P. S., LERANTH, C., WILLIAMS, M. S., MONS, N., AND GEFFARD, M. Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc. Natl. Acad. Sci. USA 86: 9015-9019, 1989.
100. GONON, F. Nonlinear relationship between impulse flow and dopamine released by rat midbrain dopaminergic neurons as studied by in vivo electrochemistry. Neuroscience 24: 19-28, 1988.
101. GONON, F. Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. J. Neurosci. 17: 5972-5978, 1997.
102. GONZALES, C. AND CHESSELET, M.-F. Amygdalonigral pathway: An antero-grade study in the rat with Phaseolus vulgaris Leucoagglutinin (PHA-L). J. Comp. Neurol. 297: 182-200, 1990.
103. GRACE, A. A. Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: a hypothesis for the etiology of schizophrenia. Neuroscience 41: 1-24, 1991.
104. GRACE, A. A. AND BUNNEY, B. S. Opposing effects of striatonigral feedback pathways on midbrain dopamine cell activity. Brain Res. 333: 271-284, 1985.
105. GRAYBIEL, A. M., AOSAKI, T., FLAHERTY, A. W., AND KIMURA, M. The basal ganglia and adaptive motor control. Science 265: 1826-1831, 1994.
106. GROVES, P. M., GARCIA-MUNOZ, M., LINDER, J. C., MANLEY, M. S., MARTONE, M. E., AND YOUNG, S. J. Elements of the intrinsic organization and information processing in the neostriatum. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 51-96.
107. GULLAPALLI, V., BARTO, A. G., AND GRUPEN, R. A. Learning admittance mapping for force-guided assembly. In: Proceedings of the 1994 International Conference on Robotics and Automation. Los Alamitos, CA: Computer Society Press, 1994, p. 2633-2638.
108. HABER, S. N., LYND, E., KLEIN, C., AND GROENEWEGEN, H. J. Topographic organization of the ventral striatal efferent projections in the rhesus monkey: an autoradiographic tracing study. J. Comp. Neurol. 293: 282-298, 1990.
109. HABER, S., LYND-BALTA, E., AND MITCHELL, S. J. The organization of the descending ventral pallidal projections in the monkey. J. Comp. Neurol. 329: 111-128, 1993.
110. HAMMER, M. An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature 366: 59-63, 1993.
111. HAMMOND, C., SHIBAZAKI, T., AND ROUZAIRE-DUBOIS, B. Branched output neurons of the rat subthalamic nucleus: electrophysiological study of the synaptic effects on identified cells in the two main target nuclei, the entopeduncular nucleus and the substantia nigra. Neuroscience 9: 511-520, 1983.
112. HATTORI, T., FIBIGER, H. C., AND MCGEER, P. L. Demonstration of a pallido-nigral projection innervating dopaminergic neurons. J. Comp. Neurol. 162: 487-504, 1975.
113. HEDREEN, J. C. AND DELONG, M. R. Organization of striatopallidal, striatonigral and nigrostriatal projections in the macaque. J. Comp. Neurol. 304: 569-595, 1991.
114. 2+ conductance. J. Neurosci. 17: 3334-3342, 1997.
115. HERRERA-MARSCHITZ, M., You, Z. B., GOINY, M., MEANA, J. J., SILVEIRA, R., GODUKHIN, O. V., CHEN, Y., ESPINOZA, S., PETTERSSON, E., LOIDL, C. F., LUBEC, G., ANDERSSON, K., NYLANDER, I., TERENIUS, L., AND UNGERSTEDT, U. On the origin of extracellular glutamate levels monitored in the basal ganglia of the rat by in vivo microdialysis. J. Neurochem. 66: 1726-1735, 1996.
116. HERSCH, S. M., CILIAX, B. J., GUTEKUNST, C.-A., REES, H. D., HEILMAN, C. J., YUNG, K. K. L., BOLAM, J. P., INCE, E., YI, H. AND LEVEY, A. I. Electron microscopic analysis of D1 and D2 dopamine receptor proteins in the dorsal striatum and their synaptic relationships with motor corticostriatal afferents. J. Neurosci. 15: 5222-5237, 1995.
117. HEYM, J. TRULSON, M. E., AND JACOBS, B. L. Raphe unit activity in freely moving cats: effects of phasic auditory and visual stimuli. Brain Res. 232: 29-39, 1982.
118. HIKOSAKA, O., SAKAMOTO, M., AND USUI, S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J. Neurophysiol. 61: 814-832, 1989.
119. HOLLERMAN, J. R. AND SCHULTZ, W. Activity of dopamine neurons during learning in a familiar task context. Soc. Neurosci. Abstr. 22: 1388, 1996.
120. HOLLERMAN, J. R., TREMBLAY, L., AND SCHULTZ, W. Reward dependency of several types of neuronal activity in primate striatum. Soc. Neurosci. Abstr. 20: 780, 1994.
121. HOLSTEIN, G. R., PASIK, P., AND HAMORI, J. Synapses between GABA-immunoreactive axonal and dendritic elements in monkey substantia nigra. Neurosci. Lett. 66: 316-322, 1986.
122. HOOVER, J. E. AND STRICK, P. L. Multiple output channels in the basal ganglia. Science 259: 819-821, 1993.
123. HORVITZ, J. C., STEWART, T., AND JACOBS, B. L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 759: 251-258, 1997.
124. HOUK, J. C., ADAMS, J. L., AND BARTO, A. G. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 249-270.
125. HOUK, J. C., BUCKINGHAM, J. T., AND BARTO, A. G. Models of the cerebellum and motor learning. Behav. Brain Sci. 19: 368-383, 1996.
126. HRUPKA, B. J., LIN, Y. M., GIETZEN, D. W., AND ROGERS, Q. R. Small changes in essential amino acid concentrations alter diet selection in amino acid-deficient rats. J. Nutr. 127: 777-784, 1997.
127. HULL, C. L. Principles of Behavior. New York: Appleton-Century-Crofts, 1943.
128. INGHAM, C. A., HOOD, S. H., WEENINK, A., VAN MALDEGEM, B., AND ARBUTHNOTT, G. W. Morphological changes in the rat neostriatum after unilateral 6-hydroxydopamine injections into the nigrostriatal pathway. Exp. Brain Res. 93: 17-27, 1993.
129. ITO, M. Long-term depression. Annu. Rev. Neurosci. 12: 85-102, 1989.
130. JACOBS, B. L. AND FORNAL, C. A. 5-HT and motor control: a hypothesis. Trends Neurosci. 16: 346-352, 1993.
131. JIMENEZ-CASTELLANOS, J. AND GRAYBIEL, A. M. Evidence that histochemically distinct zones of the primate substantia nigra pars compacta are related to patterned distributions of nigrostriatal projection neurons and striatonigral fibers. Exp. Brain Res. 74: 227-238, 1989.
132. KALMAN, R. E. A new approach to linear filtering and prediction problems. J. Basic Eng. Trans. ASME 82: 35-45, 1960.
133. KAMIN, L. J. Selective association and conditioning. In: Fundamental Issues in Instrumental Learning, edited by N. J. Mackintosh and W. K. Honig. Halifax, Canada: Dalhousie University Press, 1969, p. 42-64.
134. KAWAGOE, K. T., GARRIS, P. A., WIEDEMANN, D. J., AND WIGHTMAN, R. M. Regulation of transient dopamine concentration gradients in the microenvironment surrounding nerve terminals in the rat striatum. Neuroscience 51: 55-64, 1992.
135. KAWAGUCHI, Y., WILSON, C. J., AND EMSON, P. C. Intracellular recording of identified neostriatal patch and matrix spiny cells in a slice preparation preserving cortical inputs. J. Neurophysiol. 62: 1052-1068, 1989.
136. KAWATO, M. AND GOMI, H. The cerebellum and VOR/OKR learning models. Trends Neurosci. 15: 445-453, 1992.
137. KISKIN, N. I., KRISHTAL, O. A., AND TSYNDRENKO, A. Y. Excitatory amino acid receptors in hippocampal neurons: kainate fails to desensitize them. Neurosci. Lett. 63: 225-230, 1986.
138. KLOPF, A. H. The Hedonistic Neuron: A Theory of Memory, Learning and Intelligence. Washington, DC: Hemisphere, 1982.
139. KNOWLTON, B. J., MANGELS, J. A., AND SQUIRE, L. R. A neostriatal habit learning system in humans. Science 273: 1399-1402, 1996.
140. KÜNZLE, H. An autoradiographic analysis of the efferent connections from premotor and adjacent prefrontal regions (areas 6 and 9) in Macaca fascicularis. Brain Behav. Evol. 15: 185-234, 1978.
141. LEMOAL, M. AND OLDS, M. E. Peripheral auditory input to the midbrain limbic area and related structures. Brain Res. 167: 1-17, 1979.
142. LEMOAL, M. AND SIMON, H. Mesocorticolimbic dopaminergic network: functional and regulatory roles. Physiol. Rev. 71: 155-234, 1991.
143. LEVEY, A. I., HERSCH, S. M., RYE, D. B., SUNAHARA, R. K., NIZNIK, H. B., KITT, C. A., PRICE, D. L., MAGGIO, R., BRANN, M. R., AND CILIAX, B. J. Localisation of D1 and D2 dopamine receptors in brain with subtype-specific antibodies. Proc. Natl. Acad. Sci. USA 90: 8861-8865, 1993.
144. LINDEN, A., BRACKE-TOLKMITT, R., LUTZENBERGER, W., CANAVAN, A. G. M., SCHOLZ, E., DIENER, H. C., AND BIRBAUMER, N. Slow cortical potentials in parkinsonian patients during the course of an associative learning task. J. Psychophysiol. 4: 145-162, 1990.
145. LJUNGBERG, T., APICELLA, P., AND SCHULTZ, W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res. 586: 337-341, 1991.
146. LJUNGBERG, T., APICELLA, P., AND SCHULTZ, W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67: 145-163, 1992.
147. LLINAS, R. AND WELSH, J. P. On the cerebellum and motor learning. Curr. Opin. Neurobiol. 3: 958-965, 1993.
148. LOHMAN, A.H.M. AND VAN WOERDEN-VERKLEY, I. Ascending connections to the forebrain in the tegu lizard. J. Comp. Neurol. 182: 555-594, 1978.
149. LOUILOT, A., LEMOAL, M. AND SIMON, H. Differential reactivity of dopaminergic neurons in the nucleus accumbens in response to different behavioral situations. An in vivo voltammetric study in free moving rats. Brain Res. 397: 395-400, 1986.
150. LOVIBOND, P. F. Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J. Exp. Psychol. Anim. Behav. Proc. 9: 225-247, 1983.
151. LOVINGER, D. M., TYLER, E. C., AND MERRITT, A. Short- and long-term synaptic depression in rat neostriatum. J. Neurophysiol. 70: 1937-1949, 1993.
152. LYND-BALTA, E. AND HABER, S. N. Primate striatonigral projections: a comparison of the sensorimotor-related striatum and the ventral striatum. J. Comp. Neurol. 345: 562-578, 1994.
153. MACKINTOSH, N. J. A theory of attention: variations in the associability of stimulus with reinforcement. Psychol. Rev. 82: 276-298, 1975.
154. MANZONI, O. J., MANABE, T., AND NICOLL, R. A. Release of adenosine by activation of NMDA receptors in the hippocampus. Science 265: 2098-2101, 1994.
155. MARR, D. A theory of cerebellar cortex. J. Physiol. (Lond.) 202: 437-470, 1969.
156. MARSHALL, J. F., O'DELL, S. J., NAVARRETE, R. AND ROSENSTEIN, A. J. Dopamine high-affinity transport site topography in rat brain: major differences between dorsal and ventral striatum. Neuroscience 37: 11-21, 1990.
157. MATSUMOTO, K., NAKAMURA, K., MIKAMI, A., AND KUBOTA, K. Response to unpredictable water delivery into the mouth of visually responsive neurons in the orbitofontal cortex of monkeys. Abstr. Satellite Symp. IBR Meeting in Honor of Prof. Kubota, Inuyama, Japan, P-14, 1995.
158. MATSUMURA, M., KOJIMA, J., GARDINER, T. W., AND HIKOSAKA, O. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67: 1615-1632, 1992.
159. MAUNSELL, J.H.R. AND GIBSON, J. R. Visual response latencies in striate cortex of the macaque monkey. J. Neurophysiol. 68: 1332-1344, 1992.
160. MAZZONI, P., ANDERSEN, R. A., AND JORDAN, M. I. A more biologically plausible learning rule than backpropagation applied to a network model of cortical area 7. Cereb. Cortex 1: 293-307, 1991.
161. MCCALLUM, A. K. Reinforcement learning with selective perception and hidden states (PhD thesis). Rochester, NY: Univ. Rochester, 1995.
162. MCCORMICK, D. A. AND THOMPSON, R. F. Neuronal responses of the rabbit cerebellum during acquisition and peformance of a classically conditioned nictitating membrane-eyelid response. J. Neurosci. 4: 2811-2822, 1984.
163. MCLAREN, I. The computational unit as an assembly of neurones: an implementation of an error correcting learning algorithm. In: The Computing Neuron, edited by R. Durbin, C. Miall, and G. Mitchison. Amsterdam: Addison-Wesley, 1989 p. 160-178.
164. MICHAEL, A. C., JUSTICE, J. B., JR., AND NEILL, D. B. In vivo voltammetric determination of the kinetics of dopamine metabolism in the rat. Neurosci. Lett. 56: 365-369, 1985.
165. MIDDLETON, F. A. AND STRICK, P. L. The temporal lobe is a target of output from the basal ganglia. Proc. Natl. Acad. Sci. USA 93: 8683-8687, 1996.
166. MILLER, E. K., LI, L., AND DESIMONE, R. Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J. Neurosci. 13: 1460-1478, 1993.
167. MILLER, J. D., SANGHERA, M. K., AND GERMAN, D. C. Mesencephalic dopaminergic unit activity in the behaviorally conditioned rat. Life Sci. 29: 1255-1263, 1981.
168. MILLER, R., WICKENS, J. R., AND BENINGER, R. J. Dopamine D-1 and D-2 receptors in relation to reward and performance: a case for the D-1 receptor as a primary site of therapeutic action of neuroleptic drugs. Prog. Neurobiol. 34: 143-183, 1990.
169. MIRENOWICZ, J. AND SCHULTZ, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72: 1024-1027, 1994.
170. MIRENOWICZ, J. AND SCHULTZ, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379: 449-451, 1996.
171. MITCHELL, S. J., RICHARDSON, R. T., BAKER, F. H., AND DELONG, M. R. The primate globus pallidus: neuronal activity related to direction of movement. Exp. Brain Res. 68: 491-505, 1987.
172. MOGENSON, G. J., TAKIGAWA, M., ROBERTSON, A., AND WU, M. Self-stimulation of the nucleus accumbens and ventral tegmental area of Tsai attenuated by microinjections of spiroperidol into the nucleus accumbens. Brain Res. 171: 247-259, 1979.
173. MONTAGUE, P. R., DAYAN, P., NOWLAN, S. J., POUGET, A., AND SEJNOWSKI, T. J. Using aperiodic reinforcement for directed self-organization during development. In: Neural Information Processing Systems 5, edited by S. J. Hanson, J. D. Cowan, and C. L. Giles. San Mateo, CA: Morgan Kaufmann, 1993, p. 969-976.
174. MONTAGUE, P. R., DAYAN, P., PERSON, C., AND SEJNOWSKI, T. J. Bee foraging in uncertain environments using predictive hebbian learning. Nature 377: 725-728, 1995.
175. MONTAGUE, P. R., DAYAN, P., AND SEJNOWSKI, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16: 1936-1947, 1996.
176. MONTAGUE, P. R. AND SEJNOWSKI, T. J. The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learn. Memory 1: 1-33, 1994.
177. MORA, F. AND MYERS, R. D. Brain self-stimulation: direct evidence for the involvement of dopamine in the prefrontal cortex. Science 197: 1387-1389, 1977.
178. MURPHY, B. L., ARNSTEN, A. F., GOLDMAN-RAKIC, P. S., AND ROTH, R. H. Increased dopamine turnover in the prefrontal cortex impairs spatial working memory performance in rats and monkeys. Proc. Natl. Acad. Sci. USA 93: 1325-1329, 1996.
179. NAKAMURA, K., MIKAMI, A., AND KUBOTA, K. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J. Neurophysiol. 67: 1447-1463, 1992.
180. NEDERGAARD, S., BOLAM, J. P., AND GREENFIELD, S. A. Facilitation of dendritic calcium conductance by 5-hydroxytryptamine in the substantia nigra. Nature 333: 174-177, 1988.
181. NIIJIMA, K. AND YOSHIDA, M. Activation of mesencephalic dopamine neurons by chemical stimulation of the nucleus tegmenti pedunculopontinus pars compacta. Brain Res. 451: 163-171, 1988.
182. NIKI, H. AND WATANABE, M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 171: 213-224, 1979.
183. NIRENBERG, M. J., VAUGHAN, R. A., UHL, G. R., KUHAR, M. J., AND PICKEL,
184. NISHIJO, H., ONO, T., AND NISHINO, H. Topographic distribution of modality-specific amygdalar neurons in alert monkey. J. Neurosci. 8: 3556-3569, 1988.
185. NISHINO, H., ONO, T., MURAMOTO, K. I., FUKUDA, M., AND SASAKI, K. Neuronal activity in the ventral tegmental area (VTA) during motivated bar press feeding behavior in the monkey. Brain Res. 413: 302-313, 1987.
186. OJAKANGAS, C. L. AND EBNER, T. J. Purkinje cell complex and simple spike changes during a voluntary arm movement learning task in the monkey. J. Neurophysiol. 68: 2222-2236, 1992.
187. OLDS, J. AND MILNER, P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J. Comp. Physiol. Psychol. 47: 419-427, 1954.
188. OORSCHOT, D. E. Total number of neurons in the neostriatal, pallidal, subthalamic, and substantia nigral nuclei of the rat basal ganglia: a stereological study using the Cavalieri and optical disector methods. J. Comp. Neurol. 366: 580-599, 1996.
189. OTMAKHOVA, N. A. AND LISMAN, J. E. D1/D5 dopamine recetor activation increases the magnitude of early long-term potentiation at CA1 hippocampal synapses. J. Neurosci. 16: 7478-7486, 1996.
190. PACKARD, M. G. AND WHITE, N. M. Dissociation of hippocampus and caudate nucleus memory systems by posttraining intracerebral injection of dopamine agonists. Behav. Neurosci. 105: 295-306, 1991.
191. PASTOR, M. A., ARTIEDA, J., JAHANSHAHI, M., AND OBESO, J. A. Time estimation and reproduction is abnormal in Parkinson's disease. Brain 115: 211-225, 1992.
192. PEARCE, J. M. AND HALL, G. A model for Pavlovian conditioning : variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87: 532-552, 1980.
193. PENNARTZ, C.M.A., AMEERUN, R. F., GROENEWEGEN, H. J., AND LOPES DA SILVA, F. H. Synaptic plasticity in an in vitro slice preparation of the rat nucleus accumbens. Eur. J. Neurosci. 5: 107-117, 1993.
194. PERCHERON, G., FRANCOIS, C., YELNIK, J., AND FENELON, G. The primate nigro-striato-pallido-nigral system. Not a mere loop. In: Neural Mechanisms in Disorders of Movement, edited by A. R. Crossman and M. A. Sambrook. London: John Libbey, 1989, p. 103-109.
195. PHILLIPS, A. G., BROOKE, S. M., AND FIBIGER, H. C. Effects of amphetamine isomers and neuroleptics on self-stimulation from the nucleus accumbens and dorsal noradrenergic bundle. Brain Res. 85: 13-22, 1975.
196. PHILLIPS, A. G., CARTER, D. A., AND FIBIGER, H. C. Dopaminergic substrates of intracranial self-stimulation in the caudate nucleus. Brain Res. 104: 221-232, 1976.
197. PHILLIPS, A. G. AND FIBIGER, H. C. The role of dopamine in mediating self-stimulation in the ventral tegmentum, nucleus accumbens, and medial prefrontal cortex. Can. J. Psychol. 32: 58-66, 1978.
198. PHILLIPS, A. G., MORA, F., AND ROLLS, E. T. Intracranial self-stimulation in orbitofrontal cortex and caudate nucleus of rhesus monkey: effects of apomorphine, pimozide, and spiroperidol. Psychopharmacology 62: 79-82, 1979.
199. PICKEL, V. M., BECKLEY, S. C., JOH, T. H., AND REIS, D. J. Ultrastructural immunocytochemical localization of tyrosine hydroxylase in the neostriatum. Brain Res. 225: 373-385, 1981.
200. PRICE, J. L. AND AMARAL, D. G. An autoradiographic study of the projections of the central nucleus of the monkey amygdala. J. Neurosci. 1: 1242-1259, 1981.
201. RAO, R.P.N. AND BALLARD, D. H. Dynamic model of visual recognition predicts neural response properties in the visual cortex. Neural Computat. 9: 721-763, 1997.
202. RASMUSSEN. K. AND JACOBS, B. L. Single unit activity of locus coeruleus neurons in the freely moving cat. II. Conditioning and pharmacological studies. Brain Res. 371: 335-344, 1986.
203. RASMUSSEN, K., MORILAK, D. A., AND JACOBS, B. L. Single unit activity of locus coeruleus neurons in the freely moving cat. I. During naturalistic behaviors and in response to simple and complex stimuli. Brain Res. 371: 324-334, 1986.
204. RESCORLA, R. A. AND WAGNER, A. R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II: Current Research and Theory, edited by A. H. Black and W. F. Prokasy. New York: Appleton Century Crofts, 1972, p. 64-99.
205. RICHARDSON, R. T. AND DELONG, M. R. Nucleus basalis of Meynert neuronal activity during a delayed response task in monkey. Brain Res. 399: 364-368, 1986.
206. RICHARDSON, R. T. AND DELONG, M. R. Context-dependent responses of primate nucleus basalis neurons in a go/no-go task. J. Neurosci. 10: 2528-2540, 1990.
207. RICHFIELD, E. K., PENNNEY, J. B., AND YOUNG, A. B. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience 30: 767-777, 1989.
208. ROBBINS, T. W. AND EVERITT, B. J. Functions of dopamine in the dorsal and ventral striatum. Semin. Neurosci. 4: 119-128, 1992.
209. ROBBINS, T. W. AND EVERITT, B. J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6: 228-236, 1996.
210. ROBINSON, T. E. AND BERRIDGE, K. C. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18: 247-291, 1993.
211. ROGAWSKI, M. A. New directions in neurotransmitter action: dopamine provides some important clues. Trends Neurosci. 10: 200-205, 1987.
212. ROGERS, Q. R. AND HARPER, A. E. Selection of a solution containing histidine by rats fed a histidine-imbalanced diet. J. Comp. Physiol. Psychol. 72: 66-71, 1970.
213. ROLLS, E. T., CRITCHLEY, H. D., MASON, R., AND WAKEMAN, E. A. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J. Neurophysiol. 75: 1970-1981, 1996.
214. ROMO, R., SCARNATI, E., AND SCHULTZ, W. Role of primate basal ganglia and frontal cortex in the internal generation of movements: comparisons in striatal neurons activated during stimulus-induced movement initiation and execution. Exp. Brain Res. 91: 385-395, 1992.
215. ROMO, R. AND SCHULTZ, W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J. Neurophysiol. 63: 592-606, 1990.
216. ROMPRÉ, P.-P. AND WISE, R. A. Behavioral evidence for midbrain dopamine depolarization inactivation. Brain Res. 477: 152-156, 1989.
217. ROSSI, D. J. AND SLATER, N. T. The developmental onset of NMDA receptor channel activity during neuronal migration. Neuropharmacology 32: 1239-1248, 1993.
218. RUMELHART, D. E., HINTON, G. E., AND WILLIAMS, R. J. Learning internal representations by error propagation. In: Parallel Distributed Processing I, edited by D. E. Rumelhart and J. L. McClelland. Cambridge, MA: MIT Press, 1986, p. 318-362.
219. SAH, P., HESTRIN, S., AND NICOLL, R. A. Tonic activation of NMDA receptors by ambient glutamate enhances excitability of neurons. Science 246: 815-818, 1989.
220. SALAMONE, J. D. The actions of neuroleptic drugs on appetitive instrumental behaviors. In: Handbook of Psychopharmacology, edited by L. L. Iversen, S. D. Iversen, and S. H. Snyder. New York: Plenum, 1987, vol. 19, p. 576-608.
221. SALAMONE, J. D. Complex motor and sensorimotor functions of striatal and accumbens dopamine: involvement in instrumental behavior processes. Psychopharmacology 107: 160-174, 1992.
222. SANDS, S. B. AND BARISH, M. E. A quantitative description of excitatory amino acid neurotransmitter responses on cultured ambryonic Yenopus spinal neurons. Brain Res. 502: 375-386, 1989.
223. SARA, S. J. AND SEGAL, M. Plasticity of sensory responses of locus coeruleus neurons in the behaving rat: implications for cognition. Prog. Brain Res. 88: 571-585, 1991.
224. SAWAGUCHI, T. AND GOLDMAN-RAKIC, P. S. D1 Dopamine receptors in prefrontal cortex: involvement in working memory. Science 251: 947-950, 1991.
225. SCARNATI, E., PROIA, A., CAMPANA, E., AND PACITTI, C. A microiontophoretic study on the nature of the putative synaptic neurotransmitter involved in the pedunculopontine-substantia nigra pars compacta excitatory pathway of the rat. Exp. Brain Res. 62: 470-478, 1986.
226. SCHULTZ, W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J. Neurophysiol. 56: 1439-1462, 1986.
227. SCHULTZ, W., APICELLA, P., AND LJUNGBERG, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13: 900-913, 1993.
228. SCHULTZ, W., APICELLA, P., ROMO, R., AND SCARNATI, E. Context-dependent activity in primate striatum reflecting past and future behavioral events. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995a, p. 11-28.
229. SCHULTZ, W., APICELLA, P., SCARNATI, E., AND LJUNGBERG, T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12: 4595-4610, 1992.
230. SCHULTZ, W., DAYAN, P., AND MONTAGUE, R. R. A neural substrate of prediction and reward. Science 275: 1593-1599, 1997.
231. SCHULTZ, W. AND ROMO, R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey. J. Neurophysiol. 57: 201-217, 1987.
232. SCHULTZ, W. AND ROMO, R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J. Neurophysiol. 63: 607-624, 1990.
233. SCHULTZ, W., ROMO, R., LJUNGBERG, T., MIRENOWICZ, J., HOLLERMAN, J. R., AND DICKINSON, A. Reward-related signals carried by dopamine neurons. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambrdige, MA: MIT Press, 1995b, p. 233-248.
234. SCHULTZ, W., RUFFIEUX, A., AND AEBISCHER, P. The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation. Exp. Brain Res. 51: 377-387, 1983.
235. SEARS, L. L. AND STEINMETZ, J. E. Dorsal accessory inferior olive activity diminishes during acquisition of the rabbit classically conditioned eyelid response. Brain Res. 545: 114-122, 1991.
236. SELEMON, L. D. AND GOLDMAN-RAKIC, P. S. Topographic intermingling of striatonigral and striatopallidal neurons in the rhesus monkey. J. Comp. Neurol. 297: 359-376, 1990.
237. SESACK, S. R., AOKI, C., AND PICKEL, V. M. Ultrastructural localization of D2 receptor-like immunoreactivity in midbrain dopamine neurons and their striatal targets. J. Neurosci. 14: 88-106, 1994.
238. SESACK, S. R. AND PICKEL, V. M. Prefrontal cortical efferents in the rat synapse on unlabelled neuron targets of catecholamine terminals in the nucleus accumbens septi and on dopamine neurons in the ventral tegmental area. J. Comp. Neurol. 320: 145-160, 1992.
239. SIMON, H., SCATTON, B., AND LEMOAL, M. Dopaminergic A10 neurons are involved in cognitive functions. Nature 286: 150-151, 1980.
240. SMITH, A. D. AND BOLAM, J. P. The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones. Trends Neurosci. 13: 259-265, 1990.
241. SMITH, I. D. AND GRACE, A. A. Role of the subthalamic nucleus in the regulation of nigral dopamine neuron activity. Synapse 12: 287-303, 1992.
242. SMITH, M. C. CS-US interval and US intensity in classical conditioning of the rabbit's nictitating membrane response. J. Comp. Physiol. Psychol. 66: 679-687, 1968.
243. SMITH, Y., BENNETT, B. D., BOLAM, J. P., PARENT, A., AND SADIKOT, A. F. Synaptic relationships between dopaminergic afferents and cortical or thalamic input in the sensorimotor territory of the striatum in monkey. J. Comp. Neurol. 344: 1-19, 1994.
244. SMITH, Y. AND BOLAM, J. P. The output neurones and the dopaminergic neurones of the substantia nigra receive a GABA-containing input from the globus pallidus in the rat. J. Comp. Neurol. 296: 47-64, 1990.
245. SMITH, Y. AND BOLAM, J. P. Convergence of synaptic inputs from the striatum and the globus pallidus onto identified nigrocollicular cells in the rat: a double anterograde labelling study. Neuroscience 44: 45-73, 1991.
246. SMITH, Y., HAZRATI, L.-N., AND PARENT, A. Efferent projections of the subthalmaic nucleus in the squirrel monkey as studied by the PHA-L anterograde tracing method. J. Comp. Neurol. 294: 306-323, 1990.
247. SOMOGYI, P., BOLAM, J. P., TOTTERDELL, S., AND SMITH, A. D. Monosynaptic input from the nucleus accumbens - ventral striatum region to retrogradely labelled nigrostriatal neurones. Brain Res. 217: 245-263, 1981.
248. SPRENGELMEYER, R., CANAVAN, A.G.M., LANGE, H. W., AND HÖMBERG, V. Associative learning in degenerative neostriatal disorders: contrasts in explicit and implicit remembering between Parkinson's and Huntington's disease patients. Mov. Disord. 10: 85-91, 1995.
249. SURMEIER, D. J., EBERWINE, J., WILSON, C. J., STEFANI, A., AND KITAI, S. T. Dopamine receptor subtypes colocalize in rat striatonigral neurons. Proc. Natl. Acad. Sci. USA 89: 10178-10182, 1992.
250. STAMFORD, J. A., KRUK, Z. L., PALIJ, P., AND MILLAR, J. Diffusion and uptake of dopamine in rat caudate and nucleus accumbens compared using fast cyclic voltammetry. Brain Res. 448: 381-385, 1988.
251. STEIN, L. Self-stimulation of the brain and the central stimulant action of amphetamine. Federation Proc. 23: 836-841, 1964.
252. STEIN, L., XUE, B. G., AND BELLUZZI, J. D. In vitro reinforcement of hippocampal bursting: a search for Skinner's atoms of behavior. J. Exp. Anal. Behav. 61: 155-168, 1994.
253. STEINFELS, G. F., HEYM, J., STRECKER, R. E., AND JACOBS, B. L. Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res. 258: 217-228, 1983.
254. SUAUD-CHAGNY, M. F., DUGAST, C., CHERGUI, K., MSGHINA, M., AND GONON, F. Uptake of dopamine released by impulse flow in the rat mesolimbic and striatal system in vivo. J. Neurochem. 65: 2603-2611, 1995.
255. SURI, R. E. AND SCHULTZ, W. A neural learning model based on the activity of primate dopamine neurons. Soc. Neurosci. Abstr. 22: 1389, 1996.
256. SUTTON, R. S. Learning to predict by the method of temporal difference. Machine Learn. 3: 9-44, 1988.
257. SUTTON, R. S. AND BARTO, A. G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88: 135-170, 1981.
258. A receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons. J. Neurosci. 15: 3092-3103, 1995.
259. TESAURO, G. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Comp. 6: 215-219, 1994.
260. THOMPSON, R. F. AND GLUCK, M. A. Brain substrates of basic associative learning and memory. In: Perspectives on Cognitive Neuroscience, edited by R. G. Lister and H. J. Weingartner. New York: Oxford Univ. Press, 1991, p. 25-45.
261. THORNDIKE, E. L. Animal Intelligence: Experimental Studies. New York: MacMillan, 1911.
262. THORPE, S. J., ROLLS, E. T., AND MADDISON, S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49: 93-115, 1983.
263. TOAN, D. L. AND SCHULTZ, W. Responses of rat pallidum cells to cortex stimulation and effects of altered dopaminergic activity. Neuroscience 15: 683-694, 1985.
264. TONG, Z.-Y., OVERTON, P. G., AND CLARK, D. Stimulation of the prefrontal cortex in the rat induces patterns of activity in midbrain dopaminergic neurons which resemble natural burst events. Synapse 22: 195-208, 1996.
265. TREMBLAY, L. AND SCHULTZ, W. Processing of reward-related information in primate orbitofrontal neurons. Soc. Neurosci. Abstr. 21: 952, 1995.
266. TRENT, F. AND TEPPER, J. M. Dorsal raphé stimulation modifies striatalevoked antidromic invasion of nigral dopamine neurons in vivo. Exp. Brain Res. 84: 620-630, 1991.
267. UNGERSTEDT, U. Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol. Scand. Suppl. 367: 95-117, 1971.
268. VANKOV, A., HERVÉ-MINVIELLE, A., AND SARA, S. J. Response to novelty and its rapid habituation in locus coeruleus neurons of the freely exploring rat. Eur. J. Neurosci. 7: 1180-1187, 1995.
269. VRIEZEN, E. R. AND MOSCOVITCH, M. Memory for temporal order and conditional associative-learning in patients with Parkinson's disease. Neuropsychologia 28: 1283-1293, 1990.
270. WALSH, J. P. Depression of excitatory synaptic input in rat striatal neurons. Brain Res. 608: 123-128, 1993.
271. WANG, Y., CUMMINGS, S. L., AND GIETZEN, D. W. Temporal-spatial pattern of c-fos expression in the rat brain in response to indispensable amino acid deficiency. I. The initial recognition phase. Mol. Brain Res. 40: 27-34, 1996.
272. WATANABE, M. The appropriateness of behavioral responses coded in posttrial activity of primate prefrontal units. Neurosci. Lett. 101: 113-117, 1989.
273. WATANABE, M. Prefrontal unit activity during associative learning in the monkey. Exp. Brain Res. 80: 296-309, 1990.
274. WATANABE, M. Reward expectancy in primate prefrontal neurons. Nature 382: 629-632, 1996.
275. WAUQUIER, A. The influence of psychoactive drugs on brain self-stimulation in rats: a review. In: Brain Stimulation Reward, edited by A. Wauquier and E. T. Rolls. New York: Elsevier, 1976, p. 123-170.
276. WHITE, N. M. Reward or reinforcement: what's the difference? Neurosci. Biobehav. Rev. 13: 181-186, 1989.
277. WHITE, N. W. AND MILNER, P. M. The psychobiology of reinforcers. Annu. Rev. Psychol. 43: 443-471, 1992.
278. WIGHTMAN, R. M. AND ZIMMERMAN, J. B. Control of dopamine extracellular concentration in rat striatum by impulse flow and uptake. Brain Res. Rev. 15: 135-144, 1990.
279. WICKENS, J. R., BEGG, A. J., AND ARBUTHNOTT, G. W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70: 1-5, 1996.
280. WICKENS, J. AND KÖTTER, R. Cellular models of reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 187-214.
281. WIDROW, G. AND HOFF, M. E. Adaptive switching circuits. IRE Western Electronic Show Conven., Conven. Rec. part 4: 96-104, 1960.
282. WIDROW, G. AND STERNS, S. D. Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
283. WILLIAMS, S. M. AND GOLDMAN-RAKIC, P. S. Characterization of the dopaminergic innervation of the primate frontal cortex using a dopamine-specific antibody. Cereb. Cortex 3: 199-222, 1993.
284. WILLIAMS, G. V. AND MILLAR, J. Concentration-dependent actions of stimulated dopamine release on neuronal activity in rat striatum. Neuroscience 39: 1-16, 1990.
285. WILLIAMS, G. V., ROLLS, E. T., LEONARD, C. M., AND STERN, C. Neuronal responses in the ventral striatum of the behaving monkey. Behav. Brain Res. 55: 243-252, 1993.
286. WILSON, C., NOMIKOS, G. G., COLLU, M., AND FIBIGER, H. C. Dopaminergic correlates of motivated behavior: importance of drive. J. Neurosci. 15: 5169-5178, 1995.
287. WILSON, C. J. The contribution of cortical neurons to the firing pattern of striatal spiny neurons. In: Models of Information Processing in the Basal Ganglia, edited by J. C. Houk, J. L. Davis, and D. G. Beiser. Cambridge, MA: MIT Press, 1995, p. 29-50.
288. WILSON, F.A.W. AND ROLLS, E. T. Neuronal responses related to the novelty and familiarity of visual stimuli in the substantia innominata, diagonal band of Broca and periventricular region of the primate forebrain. Exp. Brain Res. 80: 104-120, 1990a.
289. WILSON, F.A.W. AND ROLLS, E. T. Neuronal responses related to reinforcement in the primate basal forebrain. Brain Res. 509: 213-231, 1990b.
290. WILSON, F.A.W. AND ROLLS, E. T. Learning and memory is reflected in the responses of reinforcement-related neurons in the primate basal forebrain. J. Neurosci. 10: 1254-1267, 1990c.
291. WISE R. A. Neuroleptics and operant behavior: the anhedonia hypothesis. Behav. Brain Sci. 5: 39-87, 1982.
292. WISE, R. A. Neurobiology of addiction. Curr. Opin. Neurobiol. 6: 243-251, 1996.
293. WISE, R. A. AND COLLE, L. Pimozide attenuates free feeding: "best scores" analysis reveals a motivational deficit. Psychopharmacologia 84: 446-451, 1984.
294. WISE, R. A. AND HOFFMAN, D. C. Localization of drug reward mechanisms by intracranial injections. Synapse 10: 247-263, 1992.
295. WISE, R. A. AND ROMPRE, P.-P. Brain dopamine and reward. Annu. Rev. Psychol. 40: 191-225, 1989.
296. WISE, R. A., SPINDLER, J., DE WIT, H., AND GERBER, G. J. Neurolepticinduced "anhedonia" in rats: pimozide blocks reward quality of food. Science 201: 262-264, 1978.
297. WYNNE, B. AND GÜNTÜRKÜN, O. Dopaminergic innervation of the telencephalon of the pigeon (Columba liva): a study with antibodies against tyrosine hydroxylase and dopamine. J. Comp. Neurol. 357: 446-464. 1995.
298. 2+ currents in rat neostriatal cholinergic interneurons through a membrane-delimited, protein-kinase-C-insensitive pathway. J. Neurophysiol. 77: 1003-1015, 1997.
299. YIM, C. Y. AND MOGENSON, G. J. Response of nucleus accumbens neurons to amygdala stimulation and its modification by dopamine. Brain Res. 239: 401-415, 1982.
300. YOUNG, A.M.J., JOSEPH, M. H., AND GRAY, J. A. Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: a microdialysis study. Neuroscience 48: 871-876, 1992.
301. YOUNG, A.M.J., JOSFPH, M. H., AND GRAY, J. A. Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neuroscience 54: 5-9, 1993.
302. YUNG. K.K.L., BOLAM, J. P., SMITH, A. D., HERSCH, S. M., CILIAX, B. J., AND LEVEY, A. I. Immunocytochemical localization of D1 and D2 dopamine receptors in the basal ganglia of the rat: light and electron microscopy. Neuroscience 65: 709-730, 1995.