The uncertainty processing theory of motivation

Behavioural Brain Research

Volumn 208, Issue 2, 2010, Pages 291-310

  Anselme, Patrick ^a  

^a UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

Anticipation; Attention; Dopamine; Motivation; Seeking behaviour; Uncertainty

Indexed keywords

DOPAMINE;

ANIMAL BEHAVIOR; ANTICIPATION; ASSOCIATION; ATTENTION; BEHAVIOR; BRAIN DEPTH STIMULATION; BRAIN FUNCTION; COGNITION; DRIVE; HUMAN; HYPOTHALAMUS; LEARNING; MESOLIMBIC DOPAMINERGIC SYSTEM; MOTIVATION; NEUROSCIENCE; NEUROTRANSMITTER RELEASE; NONHUMAN; PRIORITY JOURNAL; PSYCHOLOGICAL THEORY; REVIEW;

ANIMALS; ATTENTION; COGNITION; EXPLORATORY BEHAVIOR; HUMANS; MODELS, PSYCHOLOGICAL; MOTIVATION; PROBLEM SOLVING; REINFORCEMENT (PSYCHOLOGY); UNCERTAINTY;

EID: 77249166485     PISSN: 01664328     EISSN: 18727549     Source Type: Journal    
DOI: 10.1016/j.bbr.2009.12.020     Document Type: Review

References (268)

1. Acquas E., Wilson C., and Fibiger H.C. Conditioned and unconditioned stimuli increase frontal cortical and hippocampal acetylcholine release: effects of novelty, habituation, and fear. J Neurosci 16 (1996) 3089-3096
2. Acquas E., Wilson C., and Fibiger H.C. Pharmacology of sensory stimulation-evoked increases in frontal cortical acetylcholine release. Neuroscience 85 (1998) 73-83
3. Adams C.D., and Dickinson A. Instrumental responding following reinforcer devaluation. Q J Exp Psychol B 33 (1981) 109-122
4. Andrade R. Cell excitation enhances muscarinic cholinergic responses in rat association cortex. Brain Res 548 (1991) 81-93
5. Anselme P. Some conceptual problems with the classical theory of behaviour. Behav Process 75 (2007) 259-275
6. Anselme P. Abnormal patterns of displacement activities: a review and reinterpretation. Behav Process 79 (2008) 48-58
7. Anselme P. The effect of exposure to drugs on the processing of natural rewards. Neurosci Biobehav Rev 33 (2009) 314-335
8. Apicella P., Ljungberg T., Scarnati E., and Schultz W. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85 (1991) 491-500
9. Archer J. The behavioural biology of aggression (1988), Cambridge University Press, Cambridge
10. Assadi S.M., Yücel M., and Pantelis C. Dopamine modulates neural networks involved in effort-based decision-making. Neurosci Biobehav Rev 33 (2009) 383-393
11. Baddeley A. Human memory: theory and practice (1990), Lawrence Erlbaum Associates, New York
12. Bakshi V.P., and Kelley A.E. Dopaminergic regulation of feeding behavior: I. Differential effects of haloperidol microinjection in three striatal subregions. Psychobiology 19 (1991) 223-232
13. Balasko M., and Cabanac M. Motivational conflict among water need, palatability, and cold discomfort in rats. Physiol Behav 65 (1998) 35-41
14. Balleine B.W. Instrumental performance following a shift in primary motivation depends on incentive learning. J Exp Psychol Anim B 18 (1992) 236-250
15. Balleine B.W., and Dickinson A. Instrumental performance following reinforcer devaluation depends upon incentive learning. Q J Exp Psychol B 43 (1991) 279-296
16. Balleine B., and Killcross S. Effects of ibotenic acid lesions of the nucleus accumbens on instrumental action. Behav Brain Res 65 (1994) 181-193
17. Balleine B.W., and Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37 (1998) 407-419
18. Balleine B., and Dickinson A. The role of incentive learning in instrumental outcome revaluation by sensori-specific satiety. Anim Learn Behav 26 (1998) 46-59
19. Baskerville K.A., Schweitzer J.B., and Herron P. Effects of cholinergic depletion on experience-dependent plasticity in the cortex of the rat. Neuroscience 80 (1997) 1159-1169
20. Baxter M.G., and Chiba A.A. Cognitive functions of the basal forebrain. Curr Opin Neurobiol 9 (1999) 178-183
21. Bear M.F., and Singer W. Modulation of visual cortical plasticity by acetylcholine and noradrenaline. Nature 320 (1986) 172-176
22. Beck R.C. Motivation theories and principles (1978), Prentice-Hall, New York
23. Berger S.P., Hall S., Mickalian J.D., Reid M.S., Crawford C.A., Deluchi K., et al. Haloperidol antagonism of cue-elicited cocaine craving. Lancet 347 (1996) 504-508
24. Berridge K.C. Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns. Neurosci Biobehav Rev 24 (2000) 173-198
25. Berridge K.C. Motivation concepts in behavioral neuroscience. Physiol Behav 81 (2004) 179-209
26. Berridge K.C. The debate over dopamine's role in reward: the case of incentive salience. Psychopharmacology 19 (2007) 391-431
27. Berridge K.C., and Valenstein E.S. What psychological process mediates feeding evoked by electrical stimulation of the lateral hypothalamus?. Behav Neurosci 105 (1991) 3-14
28. Berridge K.C., and Robinson T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?. Brain Res Rev 28 (1998) 309-369
29. Berridge K.C., Venier I.L., and Robinson T.E. Taste reactivity analysis of 6-hydroxydopamine-induced aphagia: implications for arousal and anhedonia hypotheses of dopamine function. Behav Neurosci 103 (1989) 36-45
30. Bickel W.K., DeGrandpre R.J., and Higgins S.T. The behavioral economics of concurrent drug reinforcers: a review and reanalysis of drug self-administration research. Psychopharmacology 118 (1995) 250-259
31. Bindra D. How adaptive behavior is produced: a perceptual-motivational alternative to response-reinforcement. Behav Brain Sci 1 (1978) 41-91
32. Birrell J., and Brown V. Medial frontal cortex mediates perceptual attentional set shifting in the rat. J Neurosci 20 (2000) 4320-4324
33. Bolles R.C. Reinforcement, expectancy, and learning. Psychol Rev 79 (1972) 394-409
34. Bossert J.M., Poles G.C., Wihbey K.A., Koya E., and Shaham Y. Differential effects of blockade of dopamine D1-family receptors in nucleus accumbens core or shell on reinstatement of heroin seeking induced by contextual and discrete cues. J Neurosci 27 (2007) 12655-12663
35. Bourdelais A., and Kalivas P. Amphetamine lowers extracellular GABA concentrations in the ventral pallidum. Brain Res 516 (1990) 132-136
36. Bowman E.M., and Brown V.J. Effects of excitotoxic lesions of the rat ventral striatum on the perception of reward cost. Exp Brain Res 123 (1998) 439-448
37. Braun J.J. Gustatory cortex: definition and function. In: Kolb B., and Tees R. (Eds). The cerebral cortex of the rat (1990), The MIT Press, Cambridge, MA 407-430
38. Brown V.J., and Bowman E.M. Discriminative cues indicating reward magnitude continue to determine reaction time of rats following lesions of the nucleus accumbens. Eur J Neurosci 7 (1995) 2479-2485
39. Buck R. Prime theory: an integrated view of motivation and emotion. Psychol Rev 92 (1985) 389-413
40. Cabanac M. Physiological role of pleasure. Science 17 (1971) 1103-1107
41. Cabanac M. Pleasure: the common currency. J Theor Biol 155 (1992) 173-200
42. Cabanac M. La Cinquième Influence ou la Dialectique du Plaisir. Québec: Presses Universitaires de l'Université Laval; 2003.
43. Cannon C.M., and Bseikri M.R. Is dopamine required for natural reward?. Physiol Behav 81 (2004) 741-748
44. Capaldi E.D., Viveiros D.M., and Davidson T.L. Deprivation stimulus intensity and incentive factors in the control of instrumental responding. J Exp Psychol Anim B 7 (1981) 140-149
45. Cardinal R.N., and Everitt B.J. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol 14 (2004) 156-162
46. Cardinal R.N., Parkinson J.A., Hall J., and Everitt B.J. Emotion and motivation: the role of amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26 (2002) 321-352
47. Cardinal R.N., Winstanley C.A., Robbins T.W., and Everitt B.J. Limbic corticostriatal systems and delayed reinforcement. Ann NY Acad Sci 1021 (2004) 33-50
48. Cézilly F. Histoire de l'écologie comportementale. In: Danchin E., Giraldeau L.-A., and Cézilly F. (Eds). Ecologie comportementale (2005), Dunod, Paris 3-26
49. Chatlosh D.L., and Wasserman E.A. Memory and expectancy in delayed discrimination procedures. In: Germezano I., and Wasserman E.A. (Eds). Learning and memory (1992), Lawrence Erlbaum Associates, Hillsale 61-79
50. Chiba A.A., Bushnell P.J., Oshiro W.M., and Gallagher M. Selective removal of ACh neurons in the basal forebrain alters cued target detection. Neuroreport 10 (1999) 3119-3123
51. Cohen S., and McFarland D.J. Time-sharing as a mechanism for the control of behaviour sequences during the courtship of the three-spined stickleback (Gasterosteus aculeatus). Anim Behav 27 (1979) 270-283
52. Colwill R.M., and Rescorla R.A. Postconditioning devaluation of a reinforcer affects instrumental responding. J Exp Psychol Anim B 11 (1985) 120-132
53. Colwill R.M., and Rescorla R.A. Associative structures in instrumental learning. In: Bower G.H. (Ed). The psychology of learning and motivation (1986), Academic Press, Orlando, FL 55-104
54. Cooper J.J., and Appleby M.C. The value of environmental resources to domestic hens: a comparison of the work-rate for food and for nests as a function of time. Anim Welfare 12 (2003) 39-52
55. Corbit L.H., and Balleine B.W. The role of prelimbic cortex in instrumental conditioning. Behav Brain Res 146 (2003) 145-157
56. Corbit L.H., Muir J.L., and Balleine B.W. Lesions of mediodorsal thalamus and anterior. J Neurosci 18 (2003) 1286-1294
57. Corwin R.L., Woolverton W.L., and Schuster C.R. Effects of cholecystokinin, d-amphetamine, and fenfluramine in rats trained to discriminate 3 from 22 h of food deprivation. J Pharmacol Exp Ther 253 (1990) 720-728
58. Cosgrove K.P., Hunter R.G., and Carroll M.E. Wheel-running attenuates intravenous cocaine self-administration in rats: sex differences. Pharmacol Biochem Behav 73 (2002) 663-671
59. Cousins M.S., Wei W., and Salamone J.D. Pharmacological characterization of performance on a concurrent lever pressing/choice procedure: effects of dopamine antagonist, cholinomimetic, sedative and stimulant drugs. Psychopharmacology 116 (1994) 529-537
60. Crawford S.S., and Colgan P.W. Motivational models of courtship in male threespine sticklebacks (Gasterosteus aculeatus). Behaviour 109 (1989) 285-302
61. Culshaw A.D., and Broom D.M. The imminence of behavioural change and startle responses of chicks. Behaviour 73 (1980) 64-76
62. Davidson T.L. Learning about deprivation intensity stimuli. Behav Neurosci 101 (1987) 198-208
63. Davidson T.L. The nature and function of interoceptive signals to feed: toward integration of physiological and learning perspectives. Psychol Rev 100 (1993) 640-657
64. Day J.C., and Fibiger H.C. Dopaminergic regulation of cortical acetylcholine release. Synapse 12 (1992) 281-286
65. de la Fuente-Fernandez R., Phillips A.G., Zamburlini M., Sossi V., Calne D.B., Ruth T.J., et al. Dopamine release in human ventral striatum and expectation of reward. Behav Brain Res 136 (2002) 359-363
66. Devan B.D., McDonald R.J., and White N.M. Effects of medial and lateral caudate-putamen lesions on place- and cue-guided behaviors in the water maze: relation to thigmotaxis. Behav Brain Res 100 (1999) 5-14
67. Devauges V., and Sara S.J. Activation of the noradrenergic system facilitates an attentional shift in the rat. Behav Brain Res 39 (1990) 19-28
68. Di Chiara G. The role of dopamine in drug abuse viewed from the perspective of its role in motivation. Drug Alcohol Depend 38 (1995) 95-137
69. Di Chiara G. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res 137 (2002) 75-114
70. Dickinson A., and Dawson G.R. Motivational control of instrumental performance: the role of prior experience of the reinforcer. Q J Exp Psychol B 40 (1988) 113-134
71. Dickinson A., and Balleine B. Motivational control of instrumental performance following a shift from thirst to hunger. Q J Exp Psychol B 42 (1990) 413-431
72. Dickinson A., and Balleine B. Motivational control of goal-directed action. Anim Learn Behav 22 (1994) 1-18
73. Dickinson A., Nicholas D.J., and Adams C.D. The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Q J Exp Psychol B 35 (1983) 35-51
74. Dickinson A., Smith J., and Mirenowicz J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci 114 (2000) 468-483
75. Dukas R., and Kamil A.C. Limited attention: the constraint underlying search image. Behav Ecol 12 (2001) 192-199
76. Duffy E. The concept of energy mobilization. Psychol Rev 58 (1951) 30-40
77. El-Ghundi M., George S.R., Drago J., Fletcher P.J., Fan T., Nguyen T., et al. Disruption of dopamine D1 receptor gene expression attenuates alcohol-seeking behavior. Eur J Pharmacol 353 (1998) 149-158
78. Epstein A.N. Instinct and motivation as explanations for complex behavior. In: Pfaff D.W. (Ed). The physiological mechanisms of motivation (1982), Spinger-Verlag, New York 25-58
79. Everitt B.J., and Robbins T.W. Arousal systems and attention. In: Gazzaniga M.S. (Ed). The cognitive neurosciences (1994), MIT Press, Cambridge, MA 703-720
80. Everitt B.J., and Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neurosci 11 (2005) 1481-1489
81. Fiorillo C.D., Tobler P.N., and Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299 (2003) 1898-1902
82. Floresco S.B., Blaha C.D., Yang C.R., and Phillips A.G. Dopamine D1, NMDA receptors mediate potentiation of basolateral amygdala-evoked firing of nucleus accumbens neurons. J Neurosci 21 (2001) 6370-6376
83. Forkman B. The foraging behaviour of Mongolian gerbils: a behavioural need or a need to know?. Behaviour 133 (1996) 129-143
84. Franken I.H.A. Drug craving and addiction: integrating psychological and neuropsychopharmacological approaches. Prog Neuro-Psychopharmacol Biol Psychiatry 27 (2003) 563-579
85. Franken I.H.A., Hendricks V.M., Stam C.J., and van der Brink W. A role for dopamine in the processing of drug cues in heroin dependent patients. Eur Neuropsychopharmacol 14 (2004) 503-508
86. Freire R., Appleby M.C., and Hugues B.O. The interaction between pre-laying behaviour and feeding in hens: implication for motivation. Behaviour 134 (1997) 1019-1030
87. Fuchs R.A., Evans K.A., Ledford C.C., Parker M.P., Case J.M., Metha R.H., et al. The role of the dorsomedial prefrontal cortex, basolateral amygdala, and dorsal hippocampus in contextual reinstatement of cocaine seeking in rats. Neuropsychopharmacology 30 (2005) 296-309
88. Gallistel C.R. Conditioning from an information processing perspective. Behav Process 62 (2003) 89-101
89. Gallistel C.R., and Gibbon J. Time, rate, and conditioning. Psychol Rev 107 (2000) 289-344
90. Ghiani C.A., Dazzi L., Maciocco E., Flore G., Maira G., and Biggo G. Antagonism by abecarnil of enhanced acetylcholine release in the rat brain during anticipation but not consumption of food. Pharmacol Biochem Behav 59 (1998) 657-662
91. Gilbert A.V., Pelchat R.J., and Adler N.T. Sexual and maternal behaviour at the postpartum oestrus: the role of experience in time-sharing. Anim Behav 32 (1984) 1045-1053
92. Gong A., and Gibson R.M. Reversal of female preference after exposure to a predator in the guppy, Poecilia reticulata. Anim Behav 52 (1996) 1007-1015
93. Grace A.A. The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. Addiction 95 (2000) S119-S128
94. Gratton A., and Wise R.A. Drug- and behavior-associated changes in dopamine-related electrochemical signals during intravenous cocaine self-administration in rats. J Neurosci 14 (1994) 4130-4146
95. Gray J.A., Young A.M.J., and Joseph M.H. Dopamine's role. Science 278 (1997) 1548-1549
96. Hammond L.J. The effects of contingencies upon appetitive conditioning of free-operant behavior. J Exp Anal Behav 34 (1980) 297-304
97. Harmer C.J., and Phillips G.D. Enhanced dopamine efflux in the amygdala by a predictive, but not by a non-predictive, stimulus: facilitation by prior repeated d-amphetamine. Neuroscience 90 (1999) 119-130
98. Hatfield T., Hans J.S., Conley M., Gallagher M., and Holland P. Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J Neurosci 16 (1996) 5256-5265
99. Hernandez L., and Hoebel B.G. Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis. Life Sci 42 (1988) 1705-1712
100. Himmelheber A.M., Bruno J.P., and Sarter M. Effects of intra-accumbens infusions of amphetamine or cis-flupenthixol on sustained attention performance in rats. Behav Brain Res 116 (2000) 123-133
101. Hinde R.A. Unitary drives. Br J Anim Behav 1 (1959) 130-141
102. Hinde R.A. Energy models of motivation. Sym Soc Exp Biol 14 (1960) 199-213
103. Hirst W., and Kalmar D. Characterizing attentional resources. J Exp Psychol Gen 116 (1987) 68-81
104. Ho Y.-J., Chang Y.-C., Liu T.-M., Tai M.-Y., Wong C.-S., and Tsai Y.-F. Striatal glutamate release during novelty exposure-induced hyperactivity in olfactory bulbectomized rats. Neurosci Lett 287 (2000) 117-120
105. Hogan J.A. The interaction of incubation and feeding in broody junglefowl hens. Anim Behav 38 (1989) 121-128
106. Hogan J.A. Energy models of motivation: a reconsideration. Appl Anim Behav Sci 53 (1997) 89-105
107. Holland P.C. Occasion setting in Pavlovian conditioning. In: Medin D.L. (Ed). The psychology of learning and motivation vol. 28 (1992), Academic Press, San Diego, CA 69-125
108. Horvitz J.C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96 (2000) 651-656
109. Horvitz J.C. Stimulus-response and response-outcome learning mechanisms in the striatum. Behav Brain Res 199 (2009) 129-140
110. Houston A.I. Transitions and time-sharing. Anim Behav 30 (1982) 615-625
111. Howland J.G., Taepavarapruk P., and Phillips A.G. Glutamate receptor-dependent modulation of dopamine efflux in the nucleus accumbens by basolateral, but not central, nucleus of the amygdala in rats. J Neurosci 22 (2002) 1137-1145
112. Hughes B.O., and Duncan I.J.H. The notion of ethological 'need,' models of motivation and animal welfare. Anim Behav 36 (1988) 1696-1707
113. Hull C.L. Principles of behavior (1943), Appleton-Century-Crofts, New York
114. Hutcheson D.M., Parkinson J.A., Robbins T.W., and Everitt B.J. The effects of nucleus accumbens core and shell lesions on intravenous heroin self-administration and the acquisition of drug-seeking behaviour under a second order schedule of heroin reinforcement. Psychopharmacology 153 (2001) 464-472
115. Ikemoto S., and Panksepp J. Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. Behav Neurosci 110 (1996) 331-345
116. Ikemoto S., and Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev 31 (1999) 6-41
117. Inglis F.M., Day J.C., and Fibiger H.C. Enhanced acetylcholine release in hippocampus and cortex during the anticipation and consumption of a palatable meal. Neuroscience 62 (1994) 1049-1056
118. Ingram D.L., and Legge K.F. The thermoregulatory behavior of young pigs in a natural environment. Physiol Behav 5 (1970) 981-987
119. Ito R., Dalley J.W., Robbins T.W., and Everitt B.J. Dopamine release in the dorsal striatum during cocaine-seeking behaviour under the control of a drug-associated cue. J Neurosci 22 (2002) 6247-6253
120. Jacobs S.E., Code R.A., and Juliano S.L. Basal forebrain lesions alter stimulus-evoked metabolic activity in rat somatosensory cortex. Brain Res 560 (1991) 342-345
121. Jackson M.E., and Moghaddam B. Amygdala regulation of nucleus accumbens dopamine output is governed by the prefrontal cortex. J Neurosci 21 (2001) 676-681
122. Johnson L.R., Aylward R.L.M., Hussain Z., and Totterdell S. Input from the amygdala to the rat nucleus accumbens: its relationship with tyrosine hydroxylase immunoreactivity and identified neurons. Neuroscience 61 (1994) 851-865
123. Jones M.R. Introduction. In: Jones M.R. (Ed). Nebraska symposium on motivation (1955), University of Nebraska Press, Lincoln vii
124. Kalivas P.W., and Duffy P. Selective activation of dopamine transmission in the shell of the nucleus accumbens by stress. Brain Res 675 (1995) 325-328
125. Kametani H., and Kawamura H. Circadian rhythms of cortical acetylcholine release as measured by in vivo microdialysis in freely moving rats. Neurosci Lett 132 (1991) 263-266
126. Kamin L.J. Predictability, surprise, attention, and conditioning. In: Campbell B.A., and Church R.M. (Eds). Punishment and aversive behavior (1969), Appleton-Century-Crofts, New York 276-296
127. Keys A.S., and Mark G.P. D1 and D2 dopamine receptor mediation of amphetamine-induced acetylcholine release in nucleus accumbens. Neuroscience 86 (1998) 521-531
128. Kiyatkin E.A., Wise R.A., and Gratton A. Drug- and behavior-associated changes in dopamine-related electrochemical signals during intravenous heroin self-administration in rats. Synapse 14 (1993) 60-72
129. Koepp M.J., Gunn R.N., Lawrence A.D., Cunningham V.J., Dagher A., Jones T., et al. Evidence for striatal dopamine release during a video game. Nature 393 (1998) 266-268
130. Koob G.F., and Le Moal M. Drug addiction, dysregulation of reward, and allostasis. Neuropsychopharmacology 24 (2001) 97-129
131. Kosobud A.E., Harris G.C., and Chapin J.K. Behavioral associations of neuronal activity in the ventral tegmental area of the rat. J Neurosci 14 (1994) 7117-7129
132. Lauwereyns J., Takikawa Y., Kawagoe R., Kobayashi S., Koizumi M., Coe B., et al. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33 (2002) 463-473
133. Lejeune H., Macar F., and Zakay D. Attention and timing: dual-task performance in pigeons. Behav Process 45 (1999) 141-157
134. Lester N.P. The "feed:drink" decision. Behaviour 89 (1984) 200-219
135. Leyton M., Boileau I., Benkelfat C., Diksic M., Baker G., and Dagher A. Amphetamine-induced increases in extracellular dopamine, drug wanting, and novelty seeking: a PET/[11C]raclopride study in healthy men. Neuropsychopharmacology 27 (2002) 1027-1035
136. Ljungberg T. Differential attenuation of water intake and water-rewarded operant responding by repeated administration of haloperidol and SCH 23390 in the rat. Pharmacol Biochem Behav 35 (1990) 111-115
137. Ljungberg T., Apicella P., and Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67 (1992) 145-163
138. Lopez M., and Paredes-Olay C. Sensitivity of instrumental responses to an upshift in water deprivation. Anim Learn Behav 27 (1999) 280-287
139. Lorenz K. The comparative method in studying innate behaviour patterns. Sym Soc Exp Biol 4 (1950) 221-268
140. Ludlow A.R. The behaviour of an animal model. Behaviour 58 (1976) 131-172
141. MacLeod R., and Gosler A.G. Capture and mass change: perceived predation risk or interrupted foraging?. Anim Behav 71 (2006) 1081-1087
142. Maki W.S., and Leith C.R. Shared attention in pigeons. J Exp Anal Behav 19 (1973) 345-349
143. Malechek J.C., and Smith B.S. Behavior of range cows in response to winter weather. J Range Manage 29 (1976) 9-12
144. Marrosu F., Portas C., Mascia M.S., Casu M.A., Fa M., Giagheddu M., et al. Microdialysis measurement of cortical and hippocampal acetylcholine release during sleep-wake cycle in freely moving cats. Brain Res 671 (1995) 329-332
145. Martel P., and Fantino M. Mesolimbic dopaminergic system activity as a function of food reward: a microdialysis study. Pharmacol Biochem Behav 53 (1996) 221-226
146. McCarthy J., and Hayes P.J. Some philosophical problems from the standpoint of artificial intelligence. In: Meltzer B., and Michie D. (Eds). Machine intelligence vol. 4 (1969), Edinburgh University Press, Edinburgh, Scotland 463-502
147. McCullough L.D., and Salamone J.D. Anxiogenic drugs beta-CCE and FG 7142 increase extracellular dopamine levels in nucleus accumbens. Psychopharmacology 109 (1992) 379-382
148. McGaughy J., Kaiser T., and Sarter M. Behavioral vigilance following infusions of 192 IgG-saporin into the basal forebrain: selectivity of the behavioral impairment and relation to cortical to AChE-positive fiber density. Behav Neurosci 110 (1996) 247-265
149. McFarland D.J. Experimental investigation of motivational state. In: McFarland D. (Ed). Motivational control system analysis (1974), Academic Press, London 251-282
150. McFarland D.J., and Sibly R.M. The behavioural final common path. Philos Trans R Soc B 270 (1975) 265-293
151. McFarland K., and Ettenberg A. Haloperidol differentially affects reinforcement and motivational processes in rats running an alley for intravenous heroin. Psychopharmacology 122 (1995) 346-350
152. McFarland K., and Ettenberg A. Haloperidol does not affect motivational processes in an operant runaway model of food-seeking behavior. Behav Neurosci 112 (1998) 630-635
153. McHaffie J.G., Stanford T.R., Stein B.E., Croizet V., and Redgrave P. Subcortical loops through the basal ganglia. Trends Neurosci 28 (2005) 401-407
154. McLennan A.J., and Maier S.F. Coping and stress-induced potentiation of stimulant stereotypy in the rat. Science 219 (1983) 1091-1093
155. Metherate R., and Ashe J.H. Nucleus basalis stimulation facilitates thalamocortical synaptic transmission in the rat auditory cortex. Synapse 14 (1993) 132-143
156. Metherate R., Ashe J.H., and Weinberger N.M. Acetylcholine modifies neuronal acoustic rate-level functions in guinea pig auditory cortex by an action at muscarinic receptors. Synapse 6 (1990) 364-368
157. Miller N.E., and Kessen M.L. Reward effects of food via stomach fistula compared with those of food via mouth. J Comp Physiol Psych 45 (1952) 555-564
158. Mink J.W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50 (1996) 381-425
159. Mirza N.R., and Stolerman I.P. The role of nicotinic and muscarinic acetylcholine receptors in attention. Psychopharmacology 148 (2000) 243-250
160. Mogenson G.J., Swanson L.W., and Wu M. Neural projections from nucleus accumbens to globus pallidus, substantia innominata, and lateral preoptic-lateral hypothalamic area: an anatomical and electrophysiological investigation in the rat. J Neurosci 3 (1983) 189-202
161. Moore H., Sarter M., and Bruno J.P. Bidirectional modulation of stimulated cortical acetylcholine release by benzodiazepine receptor ligands. Brain Res 627 (1993) 267-274
162. Moore H., Fadel J., Sarter M., and Bruno J.P. Role of accumbens and cortical dopamine receptors in the regulation of cortical acetylcholine release. Neuroscience 88 (1999) 811-822
163. Müller C.M., and Singer W. Acetylcholine-induced inhibition in the cat visual cortex is mediated by a GABAergic mechanism. Brain Res 487 (1989) 335-342
164. Myers K.P., and Hall W.G. Evidence that oral and nutrient reinforcers differentially condition appetitive and consummatory responses to flavours. Physiol Behav 64 (1998) 493-500
165. Nader K., Bechara A., and van der Kooy D. Neurobiological constraints on behavioral models of motivation. Annu Rev Psychol 48 (1997) 85-114
166. Neigh G.N., Arnold H.M., Sarter M., and Bruno J.P. Dissociations between the effects of intra-accumbens administration of amphetamine and exposure to a novel environment on accumbens dopamine and cortical acetylcholine release. Brain Res 894 (2001) 354-358
167. Neigh G.N., Arnold H.M., Rabenstein R.L., Sarter M., and Bruno J.P. Neuronal activity in the nucleus accumbens is necessary for performance-related increases in cortical acetylcholine release. Neuroscience 123 (2004) 635-645
168. Nesse R.M., and Berridge K.C. Psychoactive drug use in evolutionary perspective. Science 278 (1997) 63-66
169. Nicola S.M., and Deadwyler S.A. Firing rate of nucleus accumbens neurons is dopamine-dependent and reflects the timing of cocaine-seeking behavior in rats on a progressive ratio schedule of reinforcement. J Neurosci 20 (2000) 5526-5537
170. Nisell M., Nomikos G.G., and Svensson T.H. Infusion of nicotine in the ventral tegmental area or the nucleus accumbens of the rat differentially affects accumbal dopamine release. Pharmacol Toxicol 75 (1994) 348-352
171. Noble M.E., Sanders A.F., and Trumbo D.A. Concurrent costs in double stimulation tasks. Acta Psychol 49 (1981) 141-158
172. Oatley K., and Toates F.M. Osmotic inhibition of eating as a substractive process. J Comp Physiol Psychol 82 (1973) 268-277
173. Olds J., and Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47 (1954) 419-427
174. Oswald L.M., Wong D.F., McCaul M., Zhou Y., Kuwabara H., Choi L., et al. Relationships among ventral striatal dopamine release, cortisol secretion, and subjective responses to amphetamine. Neuropsychopharmacology 30 (2005) 821-832
175. Packard M.G. Glutamate infused posttraining into the hippocampus or caudate-putamen differentially strengthens place and response learning. Proc Natl Acad Sci Biol USA 96 (1999) 12881-12886
176. Packard M.G., and McGaugh J.L. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem 65 (1996) 65-72
177. Panksepp J. Affective neuroscience: the foundations of human and animal emotions (1998), Oxford University Press, Oxford
178. Paré D., and Smith Y. GABAergic projections from the intercalated cell masses of the amygdala to the basal forebrain in cats. J Comp Neurol 344 (1994) 33-49
179. Parker L., and Leeb K. Amphetamine-induced modification of quinine palatability: analysis by the taste reactivity test. Pharmacol Biochem Behav 47 (1994) 413-420
180. Parkinson J.A., Robbins T.W., and Everitt B.J. Dissociable roles of the central and basolateral amygdala in appetitive emotional learning. Eur J Neurosci 12 (2000) 405-413
181. Pecina S., Berridge K.C., and Parker L.A. Pimozide does not shift palatability: separation of anhedonia from sensorimotor suppression by taste reactivity. Pharmacol Biochem Behav 58 (1997) 801-811
182. Pelchat M.L., Grill H.J., Rozin P., and Jacobs J. Quality of acquired responses to taste by Rattus norvegicus depends upon type of associated discomfort. J Comp Physiol Psychol 97 (1983) 140-153
183. Pérez-Uribe A. Using a time-delay actor-critic neural architecture with dopamine-like reinforcement signal for learning in autonomous robots. In: Wermter S., Austin J., and Willshaw D. (Eds). Emerging neural architectures based on neuroscience (2001), Springer-Verlag, Heidelberg, Germany 522-533
184. Petherick J.C., Seawright E., and Waddington D. Influence of motivational state on choice of food or dustbathing/foraging substrate by domestic hens. Behav Process 28 (1993) 209-220
185. Petri H.L. Motivation theory and research (1981), Wadsworth, Belmont, CA
186. Phillips J.M., McAlonan K., Robb W.G.K., and Brown V. Cholinergic neurotransmission influences covert orientation of visuospatial attention in the rat. Psychopharmacology 150 (2000) 112-116
187. Pitcher T.J. Functions of shoaling behaviour in teleost. In: Pitcher T.J. (Ed). The behaviour of teleost fish (1993), Croom Helm, London and Sydney 363-439
188. Pylyshyn Z.W. The robot's dilemma. The frame problem in artificial intelligence (1987), Ablex Publishing, New Jersey
189. Rada P., Mark G.P., and Hoebel B.G. Dopamine in the nucleus accumbens released by hypothalamic stimulation-escape behavior. Brain Res 782 (1998) 228-234
190. Ranaldi R., Pocock D., Zereik R., and Wise R.A. Dopamine fluctuations in the nucleus accumbens during maintenance, extinction, and reinstatement of intravenous d-amphetamine self-administration. J Neurosci 19 (1999) 4102-4109
191. Redgrave P., Prescott T.J., and Gurney K. The basal ganglia: a vertebrate solution to the selection problem?. Neuroscience 89 (1999) 1009-1023
192. Redgrave P., Gurney K., and Reynolds J. What is reinforced by phasic dopamine signals?. Brain Res Rev 58 (2008) 322-339
193. Rescorla R.A. Probability of shock in the presence and absence of CS in fear conditioning. J Comp Physiol Psychol 66 (1968) 1-5
194. Rescorla R.A. Pavlovian conditioning: it is not what you think it is. Am Psychol 43 (1988) 151-160
195. Richardson R.T., and DeLong M.R. Functional implications of tonic and phasic activity changes in nucleus basalis neurons. In: Richardson R.T. (Ed). Activation to acquisition: functional aspects of the basal forebrain cholinergic system (1991), Birkhäuser, Boston 135-166
196. Robbins T.W., and Everitt B.J. Arousal systems and attention. In: Ganzzaniga M.S. (Ed). The cognitive neurosciences (1995), The MIT Press, Cambridge, MA 703-720
197. Robbins T.W., and Everitt B.J. Neurobehavioural mechanisms of reward and motivation. Curr Opin Neurobiol 6 (1996) 228-236
198. Robinson T.E., and Berridge K.C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Rev 18 (1993) 247-291
199. Robinson T.E., and Berridge K.C. The psychology and neurobiology of addiction: an incentive-sensitization view. Addiction 95 Suppl. (2000) S91-S117
200. Romo R., and Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63 (1990) 592-606
201. Roper T.J. How plausible is post-inhibitory rebound as an account of displacement activity? A reply to Kennedy. Anim Behav 33 (1985) 1377-1378
202. Roper T.J., and Crossland G. Mechanisms underlying eating-drinking transitions in rats. Anim Behav 30 (1982) 602-614
203. Rosenblum K., Berman D.E., Hazvi S., Lamprecht R., and Dudai Y. NMDA receptor and the tyrosine phosphorylation of its 2B subunit in taste learning in the rat insular cortex. J Neurosci 17 (1997) 5129-5135
204. Rozin P. Adaptive food-sampling patterns in vitamin deficient rats. J Comp Physiol Psychol 69 (1969) 126-132
205. Sakagami M., and Niki H. Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp Brain Res 97 (1994) 423-436
206. Salamone J.D. Different effects of haloperidol and extinction on instrumental behaviors. Psychopharmacology 88 (1986) 18-23
207. Salamone J.D., and Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137 (2002) 3-25
208. Salamone J.D., Cousins M.S., and Bucher S. Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav Brain Res 65 (1994) 221-229
209. Sano M., Marder K., and Dooneief G. Basal ganglia diseases. In: Fogel B.S., Schiffer R.B., and Rao S.M. (Eds). Neuropsychiatry (1996), Williams and Wilkins, Baltimore 805-834
210. Sarter M., and Bruno J.P. Cognitive functions of cortical acetylcholine: toward a unifying hypothesis. Brain Res Rev 23 (1997) 28-46
211. Sarter M., and Bruno J.P. Abnormal regulation of corticopetal cholinergic neurons and impaired information processing in neuropsychiatric disorders. Trends Neurosci 22 (1999) 67-74
212. Sarter M., Bruno J.P., and Turchi J. Basal forebrain afferent projections modulating cortical acetylcholine, attention, and implications for neuropsychiatric disorders. Ann NY Acad Sci 877 (1999) 368-382
213. Schultz W. Responses of midbrain dopamine neurons to behavioural trigger stimuli in the monkey. J Neurophysiol 56 (1986) 1439-1461
214. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 80 (1998) 1-27
215. Schultz W. Multiple reward signals in the brain. Nat Rev Neurosci 1 (2000) 199-207
216. Schultz W., Apicella P., and Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13 (1993) 900-913
217. Sealfon S.C., and Olanow C.W. Dopamine receptors: from structure to behavior. Trends Neurosci 23 (2000) S34-S40
218. Selden N.R.W., Everitt B.J., Jarrad L.E., and Robbins T.W. Complementary roles for the amygdala and hippocampus in aversive conditioning to explicit and contextual cues. Neuroscience 42 (1991) 335-350
219. Seligman M.E.P., and Maier S.F. Failure to escape traumatic shock. J Exp Psychol 74 (1967) 1-9
220. Shannon C.E. A mathematical theory of communication. AT&T Tech J 27 (1948) 623-656
221. Sharf R., McKelvey J., and Ranaldi R. Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats. Psychopharmacology 186 (2006) 113-121
222. Sibly R.M., and McFarland D.J. On the fitness of behaviour sequence. Am Nat 110 (1976) 601-617
223. Smith Y., Bevan M.D., Shink E., and Bolam J.P. Microcircuitry of the direct and indirect pathways of the basal ganglia. Neuroscience 86 (1998) 353-387
224. Sokolowski J.D., and Salamone J.D. The role of nucleus accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell. Pharmacol Biochem Behav 59 (1998) 557-566
225. Solomon R.L., and Corbit J.D. An opponent-process theory of motivation: I. Temporal dynamics of affect. Psychol Rev 81 (1974) 119-145
226. Spence K.W. Behavior theory and conditioning (1956), Yale University Press, New Haven, CT
227. Squires V.R., and Wilson A.D. Distance between food and water supply and its effect on drinking frequency and food and water intake of Merino and Border Leicester sheep. Aust J Agric 22 (1971) 283-290
228. Styles E.A. The psychology of attention (1997), Psychology Press, East Sussex
229. Taber M.T., and Fibiger H.C. Electrical stimulation of the prefrontal cortex increases dopamine release in the nucleus accumbens of the rat: modulation by metabotropic glutamate receptors. J Neurosci 15 (1995) 3896-3904
230. Thompson T.I. Visual reinforcement in fighting cocks. J Exp Anal Behav 7 (1964) 45-49
231. Tinbergen N. The study of instinct (1951), Oxford University Press, Oxford
232. Toates F. Motivational systems (1986), Cambridge University Press, Cambridge
233. Tolman E.C. There is more than one kind of learning. Psychol Rev 56 (1949) 144-155
234. Tolman E.C. The nature and functioning of wants. Psychol Rev 56 (1949) 357-369
235. Turchi J., and Sarter M. Cortical acetylcholine and processing capacity: effects of cortical cholinergic deafferentation on crossmodal divided attention in rats. Cogn Brain Res 6 (1997) 147-158
236. Turner L.H., Solomon R.L., Stellar E., and Wampler S.N. Humoral factors controlling food intake in dogs. Acta Neurobiol Exp 35 (1975) 491-498
237. Ungerstedt U. Is interruption of the nigro-striatal dopamine system producing the 'lateral hypothalamus syndrome'?. Acta Physiol Scand 80 (1970) 35A-36A
238. Valenstein E.S., Cox V.C., and Kakolewski J.M. Reexamination of the role of the hypothalamus in motivation. Psychol Rev 77 (1970) 16-31
239. Vestergaard K.S., Damm B.I., Abbott U.K., and Bildsoe M. Regulation of dustbathing in feathered and featherless domestic chicks: the Lorenzian model revisited. Anim Behav 58 (1999) 1017-1025
240. Vezina P. Sensitization, drug addiction and psychopathology in animals and humans. Prog Neuro-Psychoph 31 (2007) 1553-1555
241. von Holst E., and von St. Paul U. On the functional organization of drives. Anim Behav 11 (1963) 1-20
242. Wagner A.R., Logan F.A., Haberlandt K., and Price T. Stimulus selection in animal discrimination learning. J Exp Psychol 76 (1968) 171-180
243. Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 382 (1996) 629-632
244. Weingarten H.P. Conditioned cues elicit feeding in sated rats: a role of learning in meal initiation. Science 220 (1983) 431-433
245. Weinstein A., Feldtkeller B., Malizia A., Wilson S., Bailey J., and Nutt D.J. Integrating the cognitive and physiological aspects of craving. J Psychopharmacol 12 (1998) 31-38
246. Whitelaw R.B., Markou A., Robbins T.W., and Everitt B.J. Excitotoxic lesions of the basolateral amygdala impair the acquisition of cocaine-seeking behaviour under a second-order schedule of reinforcement. Psychopharmacology 127 (1996) 213-224
247. Wickens J.R., Budd C.S., Hyland B.I., and Arbuthnott G.W. Striatal contributions to reward and decision making: making sense of regional variations in the reiterated processing matrix. Ann NY Acad Sci 1104 (2007) 192-212
248. Williams G.V., and Millar J. Concentration-dependent actions of stimulated dopamine release on neuronal activity in rat striatum. Neuroscience 39 (1990) 1-16
249. Wilson C., Nomikos G.G., Collu M., and Fibiger H.C. Dopaminergic correlates of motivated behavior: importance of drive. J Neurosci 15 (1995) 5169-5178
250. Winstanley C.A., Theobald D.E.H., Cardinal R.N., and Robbins T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J Neurosci 24 (2004) 4718-4722
251. Winter S., Dieckmann M., and Schwabe K. Dopamine in the prefrontal cortex regulates rat's behavioral flexibility to changing reward value. Behav Brain Res 198 (2009) 206-213
252. Wise R.A. Neuroleptics and operant behavior: the anhedonia hypothesis. Behav Brain Sci 5 (1982) 39-87
253. Wise R.A. Dopamine, learning and motivation. Nat Rev Neurosci 5 (2004) 1-12
254. Wise R.A. Role of brain dopamine in food reward and reinforcement. Philos Trans R Soc B 361 (2006) 1149-1158
255. Wise R.A., and Raptis L. Effects of pre-feeding on food-approach latency and food consumption speed in food deprived rats. Physiol Behav 35 (1985) 961-963
256. Witte E.A., Davidson M.C., and Marrocco R.T. Effects of altering brain cholinergic activity on covert orienting of attention: comparison of monkey and human performance. Psychopharmacology 132 (1997) 324-334
257. Wolf S., and Wolff H.G. Human gastric function: an experimental study of a man and his stomach (1943), Oxford University Press, London
258. Wolffgramm J., and Heyne A. From controlled drug intake to loss of control: the irreversible development of drug addiction in the rat. Behav Brain Res 70 (1995) 77-94
259. Woods S.C., Vasselli J.R., and Milan K.M. Iron appetite and latent learning in rats. Physiol Behav 19 (1977) 623-626
260. Wyvell C.L., and Berridge K.C. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J Neurosci 20 (2000) 8122-8130
261. Yin H.H., and Knowlton B.J. Contributions of striatal subregions to place and response learning. Learn Mem 11 (2004) 459-463
262. Yin H.H., and Knowlton B.J. The role of the basal ganglia in habit formation. Nat Rev Neurosci 7 (2006) 464-476
263. Yin H.H., Knowlton B.J., and Balleine B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 19 (2004) 181-189
264. Young P.T. Motivation and emotion: a survey of the determinants of human and animal activity (1961), John Wiley & Sons, Inc., New York
265. Young A.M., Joseph M.H., and Gray J.A. Latent inhibition of conditioned dopamine release in rat nucleus accumbens. Neuroscience 54 (1993) 5-9
266. Yu A.J., and Dayan P. Uncertainty, neuromodulation, and attention. Neuron 46 (2005) 681-692
267. Zaborszky L., Cullinan W.E., and Braun A. Afferents to basal forebrain cholinergic projection neurons: an update. In: Napier T.C., Kalivas P.W., and Hanin I. (Eds). The basal forebrain: anatomy to function (1991), Plenum Press, New York 1-42
268. Zentall T.R. Selective and divided attention in animals. Behav Process 69 (2005) 1-15