The nucleus accumbens as a nexus between values and goals in goal-directed behavior: A review and a new hypothesis

Frontiers in Behavioral Neuroscience

Volumn , Issue OCT, 2013, Pages

  Mannella, Francesco ^a   Gurney, Kevin ^b   Baldassarre, Gianluca ^a  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Amygdala; Goal selection; Goal directed behavior; Hippocampus; Novelty; Nucleus accumbens; Prefrontal cortex; Value

Indexed keywords

AMYGDALOID NUCLEUS; ANIMAL BEHAVIOR; ANTERIOR CINGULATE; ARTICLE; BASAL GANGLION; BRAIN CORTEX; CORPUS STRIATUM; HIPPOCAMPUS; HUMAN; INFERIOR TEMPORAL CORTEX; INSULA; MOTIVATION; MOTOR CORTEX; NUCLEUS ACCUMBENS; PARIETAL CORTEX; PREFRONTAL CORTEX; PRIMARY MOTOR CORTEX; REINFORCEMENT; SUBTHALAMIC NUCLEUS; THALAMUS;

EID: 84887557650     PISSN: 16625153     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnbeh.2013.00135     Document Type: Article

References (205)

1. Alexander, G. E., DeLong, M. R., and Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357-381. doi: 10.1146/annurev.ne. 09.030186.002041
2. Alexander, W. H., and Brown, J. W., Medial prefrontal cortex as an action-outcome predictor (supplementary material). Nat. Neurosci. 14, e1-e6. doi: 10.1038/ nn.2921
3. Baldassarre, G., Mannella, F., Fiore, V. G., Redgrave, P., Gurney, K., and Mirolli, M. (2012). Intrinsically motivated action- outcome learning and goal-based action recall: a system-level bioconstrained computational model. Neural Netw. 41, 168-187. doi: 10.1016/j.neunet.2012.09.015
4. Baldassarre, G. (2002). Planning with Neural Networks and Reinforcement Learning. Ph.D. thesis, Department of Computer Science, University of Essex.
5. Balleine, B. W., Daw, N. D., and O'Doherty, J. P. (2008). "Multiple forms of value learning and the function of dopamine," in Neuroeconomics: Decision Making and the Brain, eds P. W. Glimcher, F. Camerer, E. Fehr, and R. Poldrack (London: Academic Press), 367-385.
6. Balleine, B. W., Delgado, M. R., and Hikosaka, O. (2007). The role of the dorsal striatum in reward and decision-making. J. Neurosci. 27, 8161-8165. doi: 10.1523/JNEUROSCI.1554-07.2007
7. Balleine, B. W., and Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407-419. doi: 10.1016/S0028- 3908(98)00033-1
8. Balleine, B. W., and Killcross, S. Parallel incentive processing: an integrated view of amygdala function. Trends Neurosci. 29, 272-279. doi: 10.1016/j.tins.2006.03.002
9. Balleine, B. W., Killcross, S. A., and Dickinson, A. (2003). The effect of lesions of the basolateral amygdala on instrumental conditioning. J. Neurosci. 23, 666-675.
10. Balleine, B. W., Liljeholm, M., and Ostlund, S. B. (2009). The integrative function of the basal ganglia in instrumental conditioning. Behav. Brain Res. 199, 43-52. doi: 10.1016/j.bbr.2008.10.034
11. Balleine, B. W., and Ostlund, S. B. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. N.Y. Acad. Sci. 1104, 147-171. doi: 10.1196/annals.1390.006
12. Bandler, R., Keay, K. A., Floyd, N., and Price, J. (2000). Central circuits mediating patterned autonomic activity during active vs. passive emotional coping. Brain Res. Bull. 53, 95-104. doi: 10.1016/S0361-9230(00)00313-0
13. Bast, T. (2007). Toward an integrative perspective on hippocampal function: from the rapid encoding of experience to adaptive behavior. Rev. Neurosci. 18, 253-281.
14. Behbehani, M. M. (1995). Functional characteristics of the midbrain periaqueductal gray. Prog. Neurobiol. 46, 575-605. doi: 10.1016/0301- 0082(95)00009-K
15. Berridge, K. C. (2004). Motivation concepts in behavioral neuroscience. Physiol. Behav. 81, 179-209. doi: 10.1016/j.physbeh.2004.02.004
16. Berridge, K. C. (2007). The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology 191, 391-431. doi: 10.1007/s00213-006-0578-x
17. Bertsekas, D. P. (1987). Dynamic Programming: Deterministic and Stochastic Models. Englewood Cliffs, NJ: Prentice Hall.
18. Bird, C. M., and Burgess, N. (2008). The hippocampus and memory: insights from spatial processing. Nat. Rev. Neurosci. 9, 182-194. doi: 10.1038/nrn2335
19. Bogacz, R., and Gurney, K. (2007). The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 19, 442-477. doi: 10.1162/neco.2007.19.2.442
20. Bogacz, R., and Larsen, T. (2011). Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Comput. 23, 817-851. doi: 10.1162/NEC0_a_00103
21. Bornstein, A., and Daw, N. (2011). Multiplicity of control in the basal ganglia: computational roles of striatal subregions. Curr. Opin. Neurobiol. 21, 374-380. doi: 10.1016/j.conb.2011.02.009
22. Botvinick, M. M., Niv, Y., and Barto, C. (2008). Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition. 113, 262-280. doi: 10.1016/j.cognition.2008.08.011
23. Bunzeck, N., Doeller, C. F., Dolan, R. J., and Duzel, E. (2012). Contextual interaction between novelty and reward processing within the mesolimbic system. Hum. Brain Mapp. 33, 1309-1324. doi: 10.1002/hbm.21288
24. Burnod, Y., Baraduc, P., Battaglia- Mayer, A., Guigon, E., Koechlin, E., Ferraina, S., et al. (1999). Parieto-frontal coding of reaching: an integrated framework. Exp. Brain Res. 129, 325-346. doi: 10.1007/s002210050902
25. Cabib, S., and Puglisi-Allegra, S. (2012). The mesoaccumbens dopamine in coping with stress. Neurosci. Biobehav. Rev. 36, 79-89. doi: 10.1016/j.neubiorev.2011. 04.012
26. Caligiore, D., Borghi, A., Parisi, D., and Baldassarre, G. (2010). Tropicals: a computational embodied neuroscience model of compatibility effects. Psycol. Rev. 117, 1188-1228. doi: 10.1037/a0020887
27. Caligiore, D., Borghi, A., Parisi, Ellis, R., Cangelosi, A., and Baldassarre, G. (2013). How affor- dances associated with a distractor object affect compatibility effects: a study with the computational model tropicals. Psychol. Res. 77, 7-19. doi: 10.1007/s00426- 012-0424-1
28. Cardinal, R. N., Parkinson, J. A., Hall, J., and Everitt, B. J. (2002a). Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci. Biobehav. Rev. 26, 321-352.
29. Cardinal, R. N., Parkinson, J. A., Lachenal, G., Halkerston, K. M., Rudarakanchana, N., Hall, J., et al. (2002b). Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats. Behav. Neurosci. 116, 553-567.
30. Cavigelli, S. A., and McClintock, M. K. (2003). Fear of novelty in infant rats predicts adult corticosterone dynamics and an early death. Proc. Natl. Acad. Sci. U.S.A. 100, 16131-16136. doi: 10.1073/pnas.2535721100
31. Cheatwood, J. L., Reep, R. L., and Corwin, J. V. (2003). The associative striatum: cortical and thalamic projections to the dorsocentral striatum in rats. Brain Res. 968, 1-14. doi: 10.1016/S0006-8993(02)04212-9
32. Chevalier, G., and Deniau, J. M. (1990). Disinhibition as a basic process in the expression of striatal functions. Trends Neurosci. 13, 277-280. doi: 10.1016/0166-2236(90)90109-N
33. Chiba, T., Kayahara, T., and Nakano, K. (2001). Efferent projections of infralimbic and prelimbic areas of the medial prefrontal cortex in the japanese monkey, macaca fus- cata. Brain Res. 888, 83-101. doi: 10.1016/S0006-8993(00)03013-4
34. Chikama, M., McFarland, N. R., Amaral, D. G., and Haber, S. N. (1997). Insular cortical projections to functional regions of the striatum correlate with cortical cytoarchitectonic organization in the primate. J. Neurosci. 17, 9686-9705.
35. Cisek, P. (2007). Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 362, 1585-1599. doi: 10.1098/rstb.2007.2054
36. Conde, F., Maire-Lepoivre, E., Audinat, E., and Crepel, F. (1995). Afferent connections of the medial frontal cortex of the rat. ii. Cortical and subcortical afferents. J. Compt. Neurol. 352, 567-593. doi: 10.1002/cne.903520407
37. Corbit, L. H., and Balleine, B. W., The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146, 145-157. doi: 10.1016/j.bbr.2003.09.023
38. Corbit, L. H., and Balleine, B. W., Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. J. Neurosci. 25, 962-970. doi: 10.1523/JNEUR0SCI.4507-04.2005
39. Corbit, L. H., and Balleine, B. W., The general and outcome-specific forms of pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J. Neurosci. 31, 11786-11794. doi: 10.1523/JNEUR0SCI.2711-11.2011
40. Corbit, L. H., Janak, P. H., and Balleine, B. W. (2007). General and outcome-specific forms of pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur. J. Neurosci. 26, 3141-3149. doi: 10.1111/j.1460-9568.2007.05934.x
41. Corbit, L. H., Muir, J. L., and Balleine, B. W. (2001). The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accum- bens core and shell. J. Neurosci. 21, 3251-3260.
42. Corbit, L. H., Muir, J. L., and Balleine, W. (2003). Lesions of mediodor- sal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats. Eur. J. Neurosci. 18, 1286-1294. doi: 10.1046/j.1460-9568.2003. 02833.x
43. Coutureau, E., and Killcross, S. (2003). Inactivation of the infralimbic prefrontal cortex reinstates goal- directed responding in overtrained rats. Behav. Brain Res. 146, 167-174. doi: 10.1016/j.bbr.2003.09.025
44. Dalley, J. W., Cardinal, R. N., and Robbins, T. W. (2004). Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771-784. doi: 10.1016/j.neubiorev.2004.09.006
45. Davidson, R. E., and Richardson, A. M. (1970). Classical conditioning of skeletal and autonomic responses in the lizard (crotaphytus collaris). Physiol. Behav. 5, 589-594. doi: 10.1016/0031-9384(70)90085-5
46. Davis, M., and Whalen, P. J. (2001). The amygdala: vigilance and emotion. Mol. Psychiatry 6, 13-34. doi: 10.1038/sj.mp.4000812
47. Daw, N. D., Niv, Y., and Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711. doi: 10.1038/nn1560
48. Day, J. J., and Carelli, R. M. (2007). The nucleus accumbens and Pavlovian reward learning. Neuroscientist 13, 148-159. doi: 10.1177/1073858406295854
49. Day, J. J., Wheeler, R. A., Roitman, M. F., and Carelli, R. M. (2006). Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm. Eur. J. Neurosci. 23, 1341-1351. doi: 10.1111/j.1460-9568.2006.04654.x
50. Dayan, P., and Sejnowski, T. J. (1994). Td(X) converges with probability 1. Mach. Learn. 14, 295-301.
51. Dickinson, A., and Balleine, B. (1994). Motivational control of goal- directed action. Anim. Learn. Behav. 22, 1-18. doi: 10.3758/ BF03199951
52. Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M., and Tanila, H. (1999). The hippocampus, memory, and place cells: is it spatial memory or a memory space? Neuron 23, 209-226.
53. Euston, D. R., Gruber, A. J., and McNaughton, B. L. (2012). The role of medial prefrontal cortex in memory and decision making. Neuron 76, 1057-1070. doi: 10.1016/j.neuron.2012.12.002
54. Faure, A., Haberland, U., Conde, F., and Massioui, N. E. (2005). Lesion to the nigrostriatal dopamine system disrupts stimulus- response habit formation. J. Neurosci. 25, 2771-2780. doi: 10.1523/JNEUR0SCI.3894-04.2005
55. Featherstone, R. E., and McDonald, R. J. (2004). Dorsal striatum and stimulus-response learning: lesions of the dorsolateral, but not dorsomedial, striatum impair acquisition of a stimulus- response-based instrumental discrimination task, while sparing conditioned place preference learning. Neuroscience 124, 23-31. doi: 10.1016/j.neuroscience.2003.10.038
56. Finch, D. M. (1996). Neurophysiology of converging synaptic inputs from the rat prefrontal cortex, amygdala, midline thalamus, and hippocampal formation onto single neurons of the caudate/putamen and nucleus accumbens. Hippocampus 6, 495-512. doi: 10.1002/(SICI)1098-1 063(1996)6:5<495::AID-HIP03>3. 3.C0;2-I
57. Floresco, S. B. (2007). Dopaminergic regulation of limbic-striatal interplay. J. Psychiatry Neurosci. 32, 400-411.
58. Floresco, S. B., West, A. R., Ash, B., Moore, H., and Grace, A. A. (2003). Afferent modulation of dopamine neuron firing differentiallyregulates tonic and phasic dopamine transmission. Nat. Neurosci. 6, 968-973. doi: 10.1038/nn1103
59. Fogassi, L., Ferrari, P. F., Gesierich, B., Rozzi, S., Chersi, F., and Rizzolatti, G. (2005). Parietal lobe: from action organization to intention understanding. Science 308, 662-667. doi: 10.1126/science.1106138
60. Frank, M. J., and Claus, E. D. Anatomy of a decision: striato-orbitofrontal interac tions in reinforcement learning, decision making, and reversal. Psychol. Rev. 113, 300-326. doi: 10.1037/0033-295X.113.2.300
61. Frankland, P.W., and Bontempi (2005). The organization of recent and remote memories. Nat. Rev. Neurosci. 6, 119-130. doi: 10.1038/nrn1607
62. Frey, U., and Morris, R. G. (1998). Weak before strong: dissociating synaptic tagging and plasticity-factor accounts of late-ltp. Neuropharmacology 37, 545-552. doi: 10.1016/S0028-3908 (98)00040-9
63. Fudge, J. L., and Emiliano, A. B., The extended amygdala and the dopamine system: another piece of the dopamine puzzle. J. Neuropsychiatry Clin. Neurosci. 15, 306-316. doi: 10.1176/appi.neuropsych.15.3.306
64. Fudge, J. L., and Haber, S. N. (2000). The central nucleus of the amygdala projection to dopamine subpopulations in primates. Neuroscience 97, 479-494. doi: 10.1016/S0306- 4522(00)00092-0
65. Funahashi, S. (2001). Neuronal mechanisms of executive control by the prefrontal cortex. Neurosci. Res. 39, 147-165. doi: 10.1016/S0168- 0102(00)00224-8
66. Fuster, J. M. (1997). The Prefrontal Cortex: Anatomy, Physiology, and Neuropsychology of the Frontal Lobe. 3rd Edn. Philadelphia, PA: Lippincott-Raven.
67. Gasbarri, A., Sulli, A., and Packard, M. G. (1997). The dopaminergic mesencephalic projections to the hippocampal formation in the rat. Prog. Neuropsycho- pharmacol. Biol. Psychiatry 21, 1-22. doi: 10.1016/S0278-5846 (96)00157-1
68. Gerfen, C. R. (2000). Molecular effects of dopamine on striatal-projection pathways. Trends Neurosci. 23(10 Suppl.), S64-S70. doi: 10.1016/S1471-1931(00)00019-7
69. Gerfen, C. R., Engber, T. M., Mahan, L. C., Susel, Z., Chase, T. N., Monsma, F. J., et al. (1990). D1 and d2 dopamine receptor-regulated gene expression of striatoni- gral and striatopallidal neurons. Science 250, 1429-1432. doi: 10.1126/science.2147780
70. Glimcher, P. W., Camerer, C. F., Fehr, E., and Poldrack, R. A., Neuroeconomics: Decision Making and the Brain. Amsterdam: Academic Press.
71. Gold, J. I., and Shadlen, M. N. (2007). The neural basis of decision making. Annu. Rev. Neurosci. 30, 535-574. doi: 10.1146/annurev.neuro.29. 051605.113038
72. Goodale, M. A., and Milner, A. D., Separate visual pathways for perception and action. Trends Neurosci. 15, 20-25. doi: 10.1016/0166-2236(92)90344-8
73. Goto, Y., and Grace, A. A. (2005). Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat. Neurosci. 8, 805-812. doi: 10.1038/nn1471
74. Goto, Y., and O'Donnell, P. (2002). Timing-dependent limbic-motor synaptic integration in the nucleus accumbens. Proc. Natl. Acad. Sci. U.S.A. 99, 13189-13193. doi: 10.1073/pnas.202303199
75. Gruber, A. J., and McDonald, R. J., Context, emotion, and the strategic pursuit of goals: interactions among multiple brain systems controlling motivated behavior. Front. Behav. Neurosci. 6:50. doi: 10.3389/fnbeh.2012.00050
76. Gurney, K., Lepora, N., Shah, A., Koene, A., and Redgrave, P. (2012). "Action discovery and intrinsic motivation: a biologically constrained formalisation," in Intrinsically Motivated Learning in Natural and Artificial Systems, eds G. Baldassarre, M. Mirolli, G. Baldassarre, and M. Mirolli (Berlin: Springer-Verlag).
77. Gurney, K., Prescott, T., and Redgrave, P. (2001). A computational model of action selection in the basal ganglia. i. A new functional anatomy. Biol. Cybern. 84, 401-410. doi: 10.1007/PL00007984
78. Haber, S. N., Fudge, J. L., and McFarland, N. R. (2000). Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369-2382.
79. Haber, S. N., Kunishio, K., Mizobuchi, M., and Lynd-Balta, E. (1995). The orbital and medial prefrontal circuit through the primate basal ganglia. J. Neurosci. 15(7 Pt 1), 4851-4867.
80. Hall, J., Parkinson, J. A., Connor, T. M., Dickinson, A., and Everitt, B. J. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating pavlovian influences on instrumental behavior. Eur. J. Neurosci. 1984-1992. doi: 10.1046/j.0953- 816x.2001.01577.x
81. Hasselmo, M. E., Schnell, E., and Barkai, E. (1995). Dynamics of learning and recall at excitatory recurrent synapses and cholinergic modulation in rat hippocampal region ca3. J. Neurosci. 15(7 Pt 2), 5249-5262.
82. Hatfield, T., Han, J. S., Conley, M., Gallagher, M., and Holland, P. (1996). Neurotoxic lesions of baso- lateral, but not central, amygdala interfere with pavlovian second- order conditioning and reinforcer devaluation effects. J. Neurosci. 16, 5256-5265.
83. Hikosaka, O., Takikawa, Y., and Kawagoe, R. (2000). Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80, 953-978.
84. Hok, V., Save, E., Lenck-Santini, P. P., and Poucet, B. (2005). Coding for spatial goals in the prelim- bic/infralimbic area of the rat frontal cortex. Proc. Natl. Acad. Sci. U.S.A. 102, 4602-4607. doi: 10.1073/pnas.0407332102
85. Horvitz, J. C. (2000). Mesolimbocortical and nigros- triatal dopamine responses to salient non-reward events. Neuroscience 96, 651-656. doi: 10.1016/S0306-4522(00)00019-1
86. Houk, J. C., Adams, J. L., and Barto, A. G. (1995a). "A model of how the basal ganglia generate and use neural signals that predict reinforcement," in Models of Information Processing in the Basal Ganglia, eds J. C. Houk, J. L. Davis, and D. Beiser (Cambridge, MA: MIT Press), 249-270.
87. Houk, J. C., Davids, J. L., and Beiser, G. (1995b). Models of Information Processing in the Basal Ganglia. Cambridge, MA: The MIT Press.
88. Humphries, M. D., and Gurney, K. N. (2002). The role of intra- thalamic and thalamocortical circuits in action selection. Network 13, 131-156. doi: 10.1080/net.13.1.131.15
89. Humphries, M. D., and Prescott, T. J. (2010). The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog. Neurobiol. 90, 385-417. doi: 10.1016/j.pneurobio.2009.11.003
90. Izquierdo, A., Suda, R. K., and Murray, E. A. (2004). Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540-7548. doi: 10.1523/JNEUROSCI.1921-04.2004
91. Jeannerod, M., Arbib, M. A., Rizzolatti, G., and Sakata, H. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci. 18, 314-320. doi: 10.1016/0166-2236 (95)93921-J
92. Joel, D., Niv, Y., and Ruppin, E. (2002). Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netwo. 15, 535-547. doi: 10.1016/S0893-6080(02)00047-3
93. Johnson, A., and Redish, A. D. (2007). Neural ensembles in ca3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176-12189. doi: 10.1523/JNEUROSCI.3761-07.2007
94. Johnson, A., van der Meer, M. A., and Redish, A. D. (2007). Integrating hippocampus and striatum in decision-making. Curr. Opin. Neurobiol. 17, 692-697. doi: 10.1016/j.conb.2008.01.003
95. Kakade, S., and Dayan, P. (2002). Dopamine: generalization and bonuses. Neural Netw. 15, 549-559. doi: 10.1016/S0893-6080(02)00048-5
96. Karlsson, M. P., and Frank, L. M. (2008). Network dynamics underlying the formation of sparse, informative representations in the hippocampus. J. Neurosci. 28, 14271-14281. doi: 10.1523/JNEUROSCI.4261-08.2008
97. Khamassi, M., and Humphries, M. (2012). Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Front. Behav. Neurosci. 6:79. doi: 10.3389/fnbeh.2012.00079
98. Knutson, B., Delgado, M. R., and Phillips, P. E. (2009). "Representation of subjective value in the striatum," in Neuroeconomics - Decision Making and the Brain, eds P. W. Glimcher, F. Camerer, E. Fehr, and R. A. Poldrack (Amsterdam: Elsevier), 389-406.
99. Krebs, R. M., Schott, B. H., Schutze, and Duzel, E. (2009). The novelty exploration bonus and its attentional modulation. Neuropsychologia 47, 2272-2281. doi: 10.1016/j.neuropsychologia. 2009.01.015
100. Kringelbach, M. L., and Rolls, T. (2004). The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 72, 341-372. doi: 10.1016/j.pneurobio.2004. 03.006
101. Kumaran, D., and Maguire, E. A. The human hippocampus: cognitive maps or relational memory? J. Neurosci. 25, 7254-7259. doi: 10.1523/JNEUROSCI.1103- 05.2005
102. Kumaran, D., and Maguire, E. A. Match mismatch processes underlie human hippocampal responses to associative novelty. J. Neurosci. 27, 8517-8524. doi: 10.1523/JNEUROSCI.1677-07.2007
103. Lepora, N. F., and Gurney, K. N., The basal ganglia optimize decision making over general perceptual hypotheses. Neural. Comput. 24, 2924-2945. doi: 10.1162/NECO_a_00360
104. Lewis, J., Chambers, J. M., Redgrave, P., and Gurney, K. N. (2011). A computational model of interconnected basal ganglia- thalamocortical loops for goal directed action sequences. BMC Neurosci. 12(Suppl. 1):136. doi: 10.1186/1471-2202-12-S1-P136
105. Lex, B., and Hauber, W. (2010). The role of nucleus accum- bens dopamine in outcome encoding in instrumental and pavlovian conditioning. Neurobiol. Learn Mem. 93, 283-290. doi: 10.1016/j.nlm.2009.11.002
106. Lisman, J., Grace, A. A., and Duzel, (2011). A neohebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends Neurosci. 34, 536-547. doi: 10.1016/j.tins.2011.07.006
107. Lisman, J. E., and Grace, A. A. The hippocampal-vta loop: controlling the entry of information into long-term memory. Neuron 46, 703-713. doi: 10.1016/j.neuron.2005.05.002
108. Lisman, J. E., and Otmakhova, N. A. Storage, recall, and novelty detection of sequences by the hippocampus: elaborating on the socratic model to account for normal and aberrant effects of dopamine. Hippocampus 11, 551-568. doi: 10.1002/hipo.1071
109. Lubow, R. E., and Moore, A. U. (1959). Latent inhibition: the effect of nonreinforced pre-exposure to the conditional stimulus. J. Comp. Physiol. Psychol. 52, 415-419. doi: 10.1037/h0046700
110. Lucas, G. A., Deich, J. D., and Wasserman, E. A. (1981). Trace autoshaping: acquisition, maintenance, and path dependence at long trace intervals. J. Exp. Anal. Behav. 36, 61-74. doi: 10.1901/jeab.1981.36-61
111. Mannella, F., Koene, A., and Baldassarre, G. (2009). "Navigation via pavlovian conditioning: a robotic bio-constrained model of autoshaping in rats," in Proceedings of the Ninth International Conference on Epigenetic Robotics (EpiRob2009). Vol. 146. 12-14 November 2009 (Lund University, Lund), 97-104.
112. Mannella, F., Mirolli, M., and Baldassarre, G. (2010). "The interplay of pavlovian and instrumental processes in devaluation experiments: a computational embodied neuroscience model tested with a simulated rat," in Modelling Perception With Artificial Neural Networks, eds C. Tosh and G. Ruxton (Cambridge: Cambridge University Press), 93-113. doi: 10.1017/CBO9780511779145.006
113. Matsumoto, K., and Tanaka, K. (2004). The role of the medial prefrontal cortex in achieving goals. Curr. Opin. Neurobiol. 14, 178-185. doi: 10.1016/j.conb. 2004.03.005
114. McClelland, J. L., McNaughton, B. L., and O'Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connection- ist models of learning and memory. Psychol. Rev. 102, 419-457. doi: 10.1037/0033-295X.102.3.419
115. McClure, S. M., Daw, N. D., and Montague, P. R. (2003). A computational substrate for incentive salience. Trends Neurosci. 26, 423-428. doi: 10.1016/S0166-2236(03)00177-2
116. McDonald, R. J., and White, N. M. A triple dissociation of memory systems: hippocampus, amygdala, and dorsal striatum. Behav. Neurosci. 107, 3-22. doi: 10.1037/0735-7044.107.1.3
117. Medina, J. F., Repa, C. J., Mauk, M. D., and LeDoux, J. E. (2002). Parallels between cerebellum- and amygdala-dependent conditioning. Nat. Rev. Neurosci. 3, 122-131. doi: 10.1038/nrn728
118. Meeter, M., Murre, J. M. J., and Talamini, L. M. (2004). Mode shifting between storage and recall based on novelty detection in oscillating hippocampal circuits. Hippocampus 14, 722-741. doi: 10.1002/hipo.10214
119. Meyer, F. F., and Louilot, A. (2011). Latent inhibition-related dopaminergic responses in the nucleus accumbens are disrupted following neonatal transient inactivation of the ventral subicu- lum. Neuropsychopharmacology 36, 1421-1432. doi: 10.1038/ npp.2011.26
120. Middleton, F. A., and Strick, P. L. The temporal lobe is a target of output from the basal ganglia. Proc. Natl. Acad. Sci. U.S.A. 93, 8683-8687. doi: 10.1073/pnas.93.16.8683
121. Miller, E. K., and Cohen, J. D. (2001). An integrative theory of pre- frontal cortex function. Annu. Rev. Neurosci. 24, 167-202. doi: 10.1146/annurev.neuro.24.1.167
122. Mink, J. W. (1996). The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381-425. doi: 10.1016/S0301-0082(96)00042-1
123. Mirolli, M., Santucci, V. G., and Baldassarre, G. (2013). Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcement driving both action acquisition and reward maximization: a simulated robotic study. Neural Netw. 39, 40-51. doi: 10.1016/j.neunet.2012.12.012
124. Mirolli, M., Mannella, F., and Baldassarre, G. (2010). The roles of the amygdala in the affective regulation of body, brain, and behavior. Connect. Sci. 22, 215-245. doi: 10.1080/0954009100 3682553
125. Mogenson, G. J., Jones, D. L., and Yim, C. Y. (1980). From motivation to action: functional interface between the limbic system and the motor system. Prog. Neurobiol. 69-97. doi: 10.1016/0301-0082 (80)90018-0
126. Montague, P. R., Hyman, S. E., and Cohen, J. D. (2004). Computational roles for dopamine in behavioral control. Nature 431, 760-767. doi: 10.1038/nature03015
127. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., and Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063. doi: 10.1038/nn1743
128. Mulder, A. B., Tabuchi, E., and Wiener, S. I. (2004). Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation. Eur. J. Neurosci. 19, 1923-1932. doi: 10.1111/j.1460-9568.2004. 03301.x
129. Mushiake, H., Saito, N., Sakamoto, K., Itoyama, Y., and Tanji, J. (2006). Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans. Neuron 50, 631-641. doi: 10.1016/j.neuron.2006.03.045
130. Nachev, P., Kennard, C., and Husain, M. (2008). Functional role of the supplementary and pre- supplementary motor areas. Nat. Rev. Neurosci. 9, 856-869. doi: 10.1038/nrn2478
131. Niv, Y., Daw, N. D., Joel, D., and Dayan, P. (2007). Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191, 507-520. doi: 10.1007/s00213- 006-0502-4
132. O'Donnell, P., and Grace, A. A. (1995). Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J. Neurosci. 15(5 Pt 1), 3622-3639.
133. O'Keefe, J., and Burgess, N. (1996). Geometric determinants of the place fields of hippocampal neurons. Nature 381, 425-428. doi: 10.1038/381425a0
134. O'Keefe, J., and Nadel, L. (1978). The Hippocampus as a Cognitive Map. Oxford: Clarendon Press; New York, NY: Oxford University Press.
135. O'Reilly, R. C., and Frank, M. J. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283-328. doi: 10.1162/08997660677 5093909
136. Ostlund, S. B., and Balleine, W. (2007a). The contribution of orbitofrontal cortex to action selection. Ann. N.Y. Acad. Sci. 1121, 174-192. doi: 10.1196/annals.1401.033
137. Ostlund, S. B., and Balleine, B. W. (2007b). Orbitofrontal cortex mediates outcome encoding in pavlovian but not instrumental conditioning. J. Neurosci. 27, 4819-4825. doi: 10.1523/JNEUROSCI.5443-06.2007
138. Ostlund, S. B., and Balleine, B. W. Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection. J. Neurosci. 28, 4398-4405. doi: 10.1523/JNEUROSCI.5472-07.2008
139. Otmakova, N., Duzel, E., Deutch, A. Y., and Lisman, J. E. (2013). "The hippocampal-VTA loop: the role of novelty and motivation in controlling the entry of information into long-term memory," in Intrinsically Motivated Learning in Natural and Artificial Systems, eds G. Baldassarre and M. Mirolli (Berlin: Springer- Verlag), 235-254.
140. Packard, M. G., and Knowlton, B. J. Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563-593. doi: 10.1146/annurev. neuro.25.112701.142937
141. Pape, H.-C., and Pare, D. (2010). Plastic synaptic networks of the amygdala for the acquisition, expression, and extinction of conditioned fear. Physiol. Rev. 90, 419-463. doi: 10.1152/physrev.00037.200
142. Parkinson, J. A., Robbins, T. W., and Everitt, B. J. (2000). Dissociable roles of the central and basolat- eral amygdala in appetitive emotional learning. Eur. J. Neurosci. 12, 405-413. doi: 10.1046/j.1460- 9568.2000.00960.x
143. Parkinson, J. A., Willoughby, P. J., Robbins, T. W., and Everitt, B. J. (2000). Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical- ventral striatopallidal systems. Behav. Neurosci. 114, 42-63. doi: 10.1037/0735-7044.114.1.42
144. Pavlides, C., Miyashita, E., and Asanuma, H. (1993). Projection from the sensory to the motor cortex is important in learning motor skills in the monkey. J. Neurophysiol. 70, 733-741.
145. Pecina, S., Schulkin, J., and Berridge, K. C. (2006). Nucleus accum- bens corticotropin-releasing factor increases cue-triggered motivation for sucrose reward: paradoxical positive incentive effects in stress? BMC Biol. 4:8. doi: 10.1186/17417007-4-8
146. Pecina, S., Smith, K. S., and Berridge, K. C. (2006). Hedonic hot spots in the brain. Neuroscientist 12, 500-511. doi: 10.1177/1073858406293154
147. Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P., and Robbins, T. W. (2011). The hippocampal-striatal axis in learning, prediction and goal- directed behavior. Trends Neurosci. 34, 548-559. doi: 10.1016/j.tins. 2011.08.001
148. Penner, M., and Mizumori, S. Neural systems analysis of decision making during goal-directed navigation. Prog. Neurobiol. 96, 96-135. doi: 10.1016/j.pneurobio.2011.08.010
149. Peters, J., LaLumiere, R. T., and Kalivas, P. W. (2008). Infralimbic prefrontal cortex is responsible for inhibiting cocaine seeking in extinguished rats. J. Neurosci. 28, 6046-6053. doi: 10.1523/ JNEUR0SCI.1045-08.2008
150. Peterschmitt, Y., Hoeltzel, A., and Louilot, A. (2005). Striatal dopaminergic responses observed in latent inhibition are dependent on the hippocampal ventral subicular region. Eur. J. Neurosci. 22, 2059-2068. doi: 10.1111/j.1460-9568.2005.04366.x
151. Pezzulo, G., Rigoli, F., and Chersi, (2013). The mixed instrumental controller: using value of information to combine habitual choice and mental simulation. Front. Psychol. 4:92. doi: 10.3389/fpsyg.2013.00092
152. Preuss, T. M. (1995). Do rats have prefrontal cortex? the rose-woolsey- akert program reconsidered. J. Cogn. Neurosci. 7, 1-24. doi: 10.1162/jocn.1995.7.1.1
153. Quintero, E., Diaz, E., Vargas, J. P., Casa de la, G., and Lopez, J. C. (2011). Ventral subicu- lum involvement in latent inhibition context specificity. Physiol. Behav. 102, 414-420. doi: 10.1016/j.physbeh.2010.12.002
154. Quirk, G. J., Likhtik, E., Pelletier, J. G., and Pare, D. (2003). Stimulation of medial prefrontal cortex decreases the responsiveness of central amygdala output neurons. J. Neurosci. 23, 8800-8807.
155. Quirk, G. J., and Mueller, D. (2008). Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology 33, 56-72. doi: 10.1038/sj.npp.1301555
156. Quirk, G. J., Russo, G. K., Barron, J. L., and Lebron, K. (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. J. Neurosci. 20, 6225-6231.
157. Ragozzino, M. E. (2007). The contribution of the medial prefrontal cortex, orbitofrontal cortex, and dorsomedial striatum to behavioral flexibility. Ann. N.Y. Acad. Sci. 1121, 355-375. doi: 10.1196/annals.1401.013
158. Ranganath, C., and Rainer, G. (2003). Neural mechanisms for detecting and remembering novel events. Nat. Rev. Neurosci. 4, 193-202. doi: 10.1038/nrn1052
159. Rebec, G. V., Christensen, J. R., Guerra, C., and Bardo, M. T. Regional and temporal differences in real-time dopamine efflux in the nucleus accumbens during free-choice novelty. Brain Res. 776, 61-67. doi: 10.1016/S0006-8993(97)01004-4
160. Redgrave, P., and Gurney, K. (2006). The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967-975. doi: 10.1038/nrn2022
161. Redgrave, P., Prescott, T. J., and Gurney, K. (1999). The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009-1024. doi: 10.1016/S0306-4522(98)00319-4
162. Redgrave, P., Rodriguez, M., Smith, Y., Rodriguez-0roz, M. C., Lehericy, S., Bergman, H., et al. (2010). Goal-directed and habitual control in the basal ganglia: implications for parkinson's disease. Nat. Rev. Neurosci. 11, 760-772. doi: 10.1038/nrn2915
163. Rhodes, S. E. V., and Killcross, S. Lesions of rat infral- imbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn. Mem. 11, 611-616. doi: 10.1101/ lm.79704
164. Richmond, J., and Colombo, M. Hippocampal lesions, contextual retrieval, and autoshaping in pigeons. Brain Res. 928, 60-68. doi: 10.1016/S0006-8993(01)03355-8
165. Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Annu. Rev. Neurosci. 27, 169-192. doi: 10.1146/annurev.neuro.27.070203.1 44230
166. Roberts, A. C. (2006). Primate orbitofrontal cortex and adaptive behavior. Trends Cogn. Sci. 10, 83-90. doi: 10.1016/j.tics.2005.12.002
167. Robinson, D. L., Venton, J. B., Heien, M. L. A. V., and Wightman, M. R. (2003). Detecting sub- second dopamine release with fast-scan cyclic voltammetry in vivo. Clin. Chem. 49, 1763-1773. doi: 10.1373/49.10.1763
168. Robinson, D. L., and Wightman, M. R. Nomifensine amplifies sub- second dopamine signals in the ventral striatum of freely-moving rats. J. Neurochem. 90, 894-903. doi: 10.1111/j.1471-4159.2004.02559.x
169. Robinson, D. L., Zitzman, D. L., and Williams, S. K. (2011). Mesolimbic dopamine transients in motivated behaviors: focus on maternal behavior. Front. Psychiatry. 2:23. doi: 10.3389/fpsyt.2011.00023
170. Roitman, M. F., Stuber, G. D., Phillips, P. E. M., Wightman, M. R., and Carelli, R. M. (2004). Dopamine operates as a subsec- ond modulator of food seeking. J. Neurosci. 24, 1265-1271. doi: 10.1523/JNEUR0SCI.3823-03.2004
171. Rolls, E. T. (2000a). Hippocampo- cortical and cortico-cortical backprojections. Hippocampus 10, 380-388. doi: 10.1002/1098- 1063(2000)10:4<380::AID-HIP04>3.0.C0;2-0
172. Rolls, E. T. (2000b). The orbitofrontal cortex and reward. Cereb. Cortex 10, 284-294. doi: 10.1093/cercor/10.3.284
173. Rolls, E. T. (2004a). Convergence of sensory systems in the orbitofrontal cortex in primates and brain design for emotion. Anat. Rec. A Discov. Mol. Cell. Evol. Biol. 281,1212-1225. doi: 10.1002/ar.a.20126
174. Rolls, E. T. (2004b). The functions of the orbitofrontal cortex. Brain Cogn. 55, 11-29. doi: 10.1016/S0278-2626(03)00277-X
175. Rolls, E. T., and Treves, A. (1998). Neural Networks and Brain Function. 0xford: 0xford Unversity Press.
176. Romanelli, P., Esposito, V., Schaal, W., and Heit, G. (2005). Somatotopy in the basal ganglia: experimental and clinical evidence for segregated sensorimotor channels. Brain Res. Brain Res. Rev. 48, 112-128. doi: 10.1016/j.brainresrev.2004.09.008
177. Room, P., Russchen, F. T., Groenewegen, H. J., and Lohman, H. (1985). Efferent connections of the prelimbic (area 32) and the infralimbic (area 25) cortices: an anterograde tracing study in the cat. J. Comp. Neurol. 242, 40-55. doi: 10.1002/cne. 902420104
178. Runyan, J. D., Moore, A. N., and Dash, P. K. (2004). A role for prefrontal cortex in memory storage for trace fear conditioning. J. Neurosci. 24, 1288-1295. doi: 10.1523/JNEUR0SCI.4880-03.2004
179. Saito, N., Mushiake, H., Sakamoto, K., Itoyama, Y., and Tanji, J. (2005). Representation of immediate and final behavioral goals in the monkey prefrontal cortex during an instructed delay period. Cereb. Cortex 15, 1535-1546. doi: 10.1093/cercor/bhi032
180. Salamone, J. D., Correa, M., Mingote, S., and Weber, S. M. (2003). Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. J. Pharmacol. Exp. Ther. 305, 1-8. doi: 10.1124/jpet.102.035063
181. Schoenbaum, G., Chiba, A. A., and Gallagher, M. (1998). 0rbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1, 155-159. doi: 10.1038/407
182. Schoenbaum, G., Setlow, B., Nugent, S. L., Saddoris, M. P., and Gallagher, M. (2003). Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 10, 129-140. doi: 10.1101/lm. 55203
183. Schultz, W. (2002). Getting formal with dopamine and reward. Neuron 36, 241-263. doi: 10.1016/S0896- 6273(02)00967-4
184. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599. doi: 10.1126/sci- ence.275.5306.1593
185. Sesack, S. R., Deutch, A. Y., Roth, R. H., and Bunney, B. S. (1989). Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with phaseolus vulgaris leucoagglutinin. J. Comp. Neurol. 290, 213-242. doi: 10.1002/cne.902900205
186. Shadlen, M. N., and Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 86, 1916-1936.
187. Smith, D. M., and Mizumori, S. J. Y. Hippocampal place cells, context, and episodic memory. Hippocampus 16, 716-729. doi: 10.1002/hipo.20208
188. Solway, A., and Botvinick, M. M. Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol. Rev. 119, 120-154. doi: 10.1037/a0026435
189. Sotres-Bayon, F., and Quirk, G. J. (2010). Prefrontal control of fear: more than just extinction. Curr. Opin. Neurobiol. 20, 231-235. doi: 10.1016/j.conb.2010.02.005
190. Sutton, R. S., and Barto, A. G. (1987). "A temporal-difference model of classical conditioning," in In Proceedings of the Ninth Annual Conference of the Cognitive Science Society (Seattle, WA).
191. Sutton, R. S., and Barto, A. G., Reinforcement Learning: An Introduction. Cambridge MA: The MIT Press.
192. Tokimura, H., Di Lazzaro, V., Tokimura, Y., Oliviero, A., Profice, P., Insola, A., et al. (2000). Short latency inhibition of human hand motor cortex by somatosensory input from the hand. J. Physiol. 523(Pt 2), 503-513. doi: 10.1111/j.1469-7793.2000.t01- 1-00503.x
193. Toni, I., Ramnani, N., Josephs, O., Ashburner, J., and Passingham, R. E. (2001). Learning arbitrary visuomotor associations: temporal dynamic of brain activity. Neuroimage 14, 1048-1057. doi: 10.1006/nimg.2001.0894
194. VanElzakker, M., Fevurly, R. D., Breindel, T., and Spencer, R. L. Environmental novelty is associated with a selective increase in fos expression in the output elements of the hippocampal formation and the perirhinal cortex. Learn. Mem. 15, 899-908. doi: 10.1101/lm.1196508
195. Vertes, R. P. (2004). Differential projections of the infralimbic and prelimbic cortex in the rat. Synapse 51, 32-58. doi: 10.1002/ syn.10279
196. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., and Pennartz, C. M. A. (2004). Putting a spin on the dorsal- ventral divide of the striatum. Trends Neurosci. 27, 468-474. doi: 10.1016/j.tins.2004.06.006
197. Watkins, C. J., and Dayan, P. (1992). Q-learning. Mach. learn. 8, 279-292. doi: 10.1023/A:10226 76722315
198. Wise, S. P. (2008). Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 31, 599-608. doi: 10.1007/BF00992698
199. Wittmann, B. C., Bunzeck, N., Dolan, R. J., and Duzel, E., Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuroimage 38, 194-202. doi: 10.1016/j.neuroimage. 2007.06.038
200. Wyvell, C. L., and Berridge, K. C. (2000). Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J. Neurosci. 20, 8122-8130.
201. Yin, H. H., and Knowlton, B. J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464-476. doi: 10.1038/ nrn1919
202. Yin, H. H., Knowlton, B. J., and Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181-189. doi: 10.1111/j.1460-9568.2004. 03095.x
203. Yin, H. H., Ostlund, S. B., and Balleine, B. W. (2008). Reward- guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico- basal ganglia networks. Eur. J. Neurosci. 28, 1437-1448. doi: 10.1111/j.1460-9568.2008.06422.x
204. Yin, H. H., Ostlund, S. B., Knowlton, J., and Balleine, B. W. (2005). The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513-523. doi: 10.1111/j.1460-9568.2005. 04218.x
205. Zahm, D. S. (2000). An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci. Biobehav. Rev. 24, 85-105. doi: 10.1016/S0149-7634(99)00065-2