Information processing in decision-making systems

Neuroscientist

Volumn 18, Issue 4, 2012, Pages 342-359

  Van Der Meer, Matthijs ^a   Kurth Nelson, Zeb ^b   Redish, A David ^c  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF MINNESOTA   (United States)

Author keywords

decision making; dorsal striatum; hippocampus; rat; ventral striatum

Indexed keywords

ANATOMICAL CONCEPTS; BRAIN FUNCTION; BRAIN REGION; CORPUS STRIATUM; DECISION MAKING; DOPAMINERGIC TRANSMISSION; HIPPOCAMPUS; HUMAN; INFORMATION PROCESSING; NEUROANATOMY; NONHUMAN; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; STATE DEPENDENT LEARNING; STRIATE CORTEX; TASK PERFORMANCE;

ANIMALS; BRAIN; CONDITIONING, CLASSICAL; DECISION MAKING; HABITS; HUMANS; MENTAL PROCESSES; REINFORCEMENT (PSYCHOLOGY);

EID: 84859344211     PISSN: 10738584     EISSN: 10894098     Source Type: Journal    
DOI: 10.1177/1073858411435128     Document Type: Review

References (87)

1. Addis DR, Wong AT, Schacter DL. Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuro-psychologia. 2007 ; 45 (7). 1363-77 (Pubitemid 46096997)
2. Aragona BJ, Day JJ, Roitman MF, Cleveland NA, Wightman RM, Carelli RM. 2009. Regional specificity in the real-time development of phasic dopamine transmission patterns during acquisition of a cue-cocaine association in rats. Eur J Neurosci. 2009 ; 30 (10). 1889-99
3. Atallah EH, Lopeq-Paniagua D, Rudy JW, O'Reilly RC. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience. 2007 ; 10 (1). 126-31 (Pubitemid 46026536)
4. Atance CM, O'Neill DK. Episodic future thinking. Trends Cogn Sci. 2001 ; 5 (12). 533-39 (Pubitemid 33109079)
5. Badiani A, Belin D, Epstein D, Calu D, Shaham Y. Opiate versus psychostimulant addiction: the differences do matter. Nat Rev Neurosci. 2011 ; 12 (11). 685-700
6. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005 ; 437: 1158-61 (Pubitemid 41509353)
7. Bellman R. On a routing problem. Quart J Appl Math. 1958 ; 16 (1). 87-90
8. Berke JD, Eichenbaum H. Striatal versus hippocampal representations during win-stay maze performance. J Neurophysiol. 2009 ; 101 (3). 1575-87
9. Bouton ME Learning and behavior: a contemporary synthesis. Sunderland (MA): Sinauer Associates ; 2007 :
10. Boysen ST, Berntson GG. Responses to quantity: Perceptual versus cognitive mechanisms in chimpanzees (Pan troglodytes). J Exp Psychol Anim Behav Process. 1995 ; 21 (1). 82-86
11. Breland K, Breland M. The misbehavior of organisms. Am Psychol. 1961 ; 16 (11). 682-84
12. Buckner RL, Carroll DC. Self-projection and the brain. Trends Cogn Sci. 2007 ; 11 (2). 49-57 (Pubitemid 46176697)
13. Bushong B, King LM, Camerer CF, Rangel A. Pavlovian processes in consumer choice: the physical presence of a good increases willingness-to-pay. Am Econ Rev. 2010 ; 100 (4). 1556-71
14. Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002 ; 26 (3). 321-52 (Pubitemid 34555410)
15. Cisek P, Kalaska JF. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci. 2010 ; 33: 269-98
16. Cohen NJ, Eichenbaum H Memory, amnesia, and the hippocampal system. Cambridge (MA): MIT Press ; 1993 :
17. Craig AD. Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol. 2003 ; 13 (4). 500-5 (Pubitemid 37088242)
18. D'Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 2008 ; 319 (5867). 1264-67 (Pubitemid 351323026)
19. Dayan P, Niv Y. Reinforcement learning: the good, the bad and the ugly. Curr Opin Neurobiol. 2008 ; 18 (2). 185-96
20. Dayan P, Niv Y, Seymour B, Daw ND. The misbehavior of value and the discipline of the will. Neural Netw. 2006 ; 19: 1153-60 (Pubitemid 44466331)
21. Fenton AA, Lytton WW, Barry JM, Lenck-Santini PP, Zinyuk LE, Kubik S. Attention-like modulation of hippocampus place cell discharge. J Neurosci. 2010 ; 30 (13). 4613-25
22. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I. A selective role for dopamine in stimulus-reward learning. Nature. 2011 ; 469 (7328). 53-57
23. Frank MJ. Computational models of motivated action selection in corticostriatal circuits. Curr Opin Neurobiol. 2011 ; 21 (3). 381-86
24. Gupta AS Pittsburgh (PA): Carnegie Mellon University ; 2011 :
25. Haber SN Handbook of reward and decision making. Dreher JC Tremblay L, ed. New York: Academic Press ; 2009: 327.
26. Hassabis D, Maguire EA Predictions in the brain: using our past to generate a future. Bar M, ed. Oxford: Oxford University Press ; 2011: 70-82.
27. Hershberger WA. An approach through the looking glass. Learn Behav. 1986 ; 14 (4). 443-51
28. Heyman G Addiction: a disorder of choice. Cambridgex (MA): Harvard University Press ; 2009 :
29. Hikosaka O, Sakamoto M, Usui S. Functional properties of monkey caudate neurons. J Neurophysiol. 1989 ; 61 (4). 780-832
30. Hill AJ. First occurrence of hippocampal spatial firing in a new environment. Exp Neurol. 1978 ; 62: 282-97 (Pubitemid 9055034)
31. Hill C. The rationality of preference construction (and the irrationality of rational choice). Min J Law Sci Technol. 2008 ; 9 (2). 689-742
32. Hollup SA, Molden S, Donnett JG, Moser MB, Moser EI. Accumulation of hippocampal place fields at the goal location in an annular watermaze task. J Neurosci. 2001 ; 21 (5). 1635-44 (Pubitemid 32182204)
33. Hu D, Amsel A. A simple test of the vicarious trial-and-error hypothesis of hippocampal function. Proc Natl Acad Sci U S A. 1995 ; 92: 5506-9
34. Huys QJM University College London: Gatsby Computational Neuroscience Unit ; 2007 :
35. Huys QJM, Moutoussis M, Williams J. Are computational models of any use to psychiatry. Neural Netw. 2011 ; 24 (6). 544-51
36. Johnson A, Crowe DA. Revisiting Tolman, his theories and cognitive maps. Cogn Crit. 2009 ; 1: 43-72
37. Johnson A, Jackson J, Redish AD Mechanisms of information processing in the brain: encoding of information in neural populations and networks. Hölscher C Munk MHJ, ed. Cambridge: Cambridge University Press ; 2008: 95-119.
38. Johnson A, Redish AD. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci. 2007 ; 27 (45). 12176-12189 (Pubitemid 350085471)
39. Kentros CG, Agnihotri NT, Streater S, Hawkins RD, Kandel ER. Increased attention to spatial context increases both place field stability and spatial memory. Neuron. 2004 ; 42: 283-95 (Pubitemid 38501650)
40. Killcross S, Coutureau E. Coordination of actions and habits in the medial prefrontral cortex of rats. Cereb Cortex. 2003 ; 13 (8). 400-8 (Pubitemid 36373685)
41. Kimchi EY, Laubach M. Dynamic encoding of action selection by the medial striatum. J Neurosci. 2009 ; 29 (10). 3148-59
42. Kishida KT, King-Casas B, Montague PR. Neuroeconomic approaches to mental disorders. Neuron. 2010 ; 67 (4). 543-54
43. Kurth-Nelson Z, Redish AD Computational neuroscience of drug addiction. Gutkin B Ahmed SH, ed. New York: Springer ; 2012 :
44. Langer EJ, Roth J. Heads I win, tails it's chance: the illusion of control as a function of the sequence of outcomes in a purely chance task. J Pers Soc Psychol. 1975 ; 32 (6). 951-5
45. Lansink CS, Goltstein PM, Lankelma JV, McNaughton BL, Pennartz CMA. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol. 2009 ; 7 (8). e1000173
46. Lau B, Glimcher PW. Action and outcome encoding in the primate caudate nucleus. J Neurosci. 2007 ; 27 (52). 14502-14
47. Lavoie AM, Mizumori SJY. Spatial-, movement- and reward-sensitive discharge by medial ventral striatum neurons in rats. Brain Res. 1994 ; 638: 157-68 (Pubitemid 24063300)
48. Ledoux J The synaptic self. New York: Penguin ; 2002 :
49. Maia TV, Frank MJ. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci. 2011 ; 14 (2). 154-65
50. McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci. 2011 ; 31 (7). 2700-5
51. McHugh PR, Slavney PR The perspectives of psychiatry. Baltimore, MD: Johns Hopkins ; 1998 :
52. Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980 ; 14: 69-97 (Pubitemid 10053405)
53. Morris RGM, Garrud P, Rawlins JNP, O'Keefe J. Place navigation impaired in rats with hippocampal lesions. Nature. 1982 ; 297: 681-83 (Pubitemid 12096189)
54. Muenzinger KF, Gentry E. Tone discrimination in white rats. J Comp Psychol. 1931 ; 12 (2). 195-206
55. Nicola SM. The flexible approach hypothesis: unification of effort and cue-responding hypotheses for the role of nucleus accumbens dopamine in the activation of reward-seeking behavior. J Neurosci. 2010 ; 30 (49). 16585-600
56. Niv Y, Joel D, Dayan P. A normative perspective on motivation. Trends Cogn Sci. 2006 ; 10 (8). 375-81 (Pubitemid 44093326)
57. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004 ; 304 (5669). 452-54 (Pubitemid 38495943)
58. O'Keefe J, Nadel L The hippocampus as a cognitive map. Oxford: Clarendon Press ; 1978 :
59. Redish AD Beyond the cognitive map: from place cells to episodic memory. Cambridge (MA): MIT Press ; 1999 :
60. Redish AD. Addiction as a computational process gone awry. Science. 2004 ; 306 (5703). 1944-47 (Pubitemid 39627860)
61. Redish AD, Jensen S, Johnson A. A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci. 2008 ; 31: 415-87
62. Redish AD, Jensen S, Johnson A, Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychol Rev. 2007 ; 114 (3). 784-805
63. Redish AD, Touretzky DS. The role of the hippocampus in solving the Morris water maze. Neural Comput. 1998 ; 10 (1). 73-111 (Pubitemid 128463654)
64. Rescorla RA, Wagner AR Classical Conditioning II: Current Research and Theory. Black AH Prokesy WF, ed. New York: Appleton Century Crofts ; 1972: 64-99.
65. Rich EL, Shapiro M. Rat prefrontal cortical neurons selectively code strategy switches. J Neurosci. 2009 ; 29 (22). 7208-19
66. Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science. 2005 ; 310 (5752). 1337-40 (Pubitemid 41698867)
67. Samsonovich AV, Ascoli GA. A simple neural network model of the hippocampus suggesting its pathfinding role in episodic memory retrieval. Learn Mem. 2005 ; 12 (2). 193-208 (Pubitemid 40517087)
68. Sanfey AG, Rilling JK, Aronson JA, Nystrom LE, Cohen JD. The neural basis of economic decision-making in the ultimatum game. Science. 2003 ; 300 (5626). 1755-58 (Pubitemid 36712573)
69. Schacter DL, Addis DR Predictions in the brain: using our past to generate a future. Bar M, ed. Oxford: Oxford University Press ; 2011: 58-69.
70. Schmitzer-Torbert NC, Redish AD. Neuronal activity in the rodent dorsal striatum in sequential navigation: Separation of spatial and reward responses on the multiple-T task. J Neurophysiol. 2004 ; 91 (5). 2259-72 (Pubitemid 38500941)
71. Schmitzer-Torbert NC, Redish AD. Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype. Neuroscience. 2008 ; 153 (2). 349-60
72. Schultz W, Dayan P, Montague R. A neural substrate of prediction and reward. Science. 1997 ; 275: 1593-99 (Pubitemid 27120526)
73. Singh T, McDannald MA, Haney RZ, Cerri DH, Schoenbaum G. Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on Pavlovian conditioned responding. Front Integr Neurosci. 2010 ; 4: 126
74. Stalnaker T, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front Integr Neurosci. 2010 ; 19: 12
75. Sutherland GR, McNaughton BL. Memory trace reactivation in hippocampal and neocortical neuronal ensembles. Curr Opin Neurobiol. 2000 ; 10 (2). 180-6 (Pubitemid 30201109)
76. Sutton RS, Barto AG Reinforcement learning: an introduction. Cambridge (MA): MIT Press ; 1998 :
77. Swanson LW. Cerebral hemisphere regulation of motivated behavior. Brain Res. 2000 ; 886 (1-2). 113-64
78. Taha SA, Fields HL. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J Neurosci. 2005 ; 25 (5). 1193-1202 (Pubitemid 40268961)
79. Talmi D, Seymour B, Dayan P, Dolan RJ. Human pavlovian-instrumental transfer. J Neurosci. 2008 ; 28 (2). 360-8
80. Thorn CA, Atallah H, Howe M, Graybiel AM. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron. 2010 ; 66 (5). 781-95
81. Tolman EC Purposive behavior in animals and men. New York: Appleton-Century-Crofts ; 1932 :
82. Tse D, Langston RF, Kakeyama M, Bethus I, Spooner PA, Wood ER. Schemas and memory consolidation. Science. 2007 ; 316 (5821). 76-82 (Pubitemid 46559520)
83. van der Meer MAA, Johnson A, Schmitzer-Torbert NC, Redish AD. Triple dissociation of information processing in dorsal striatum, ventral striatum, and hippocampus on a learned spatial decision task. Neuron. 2010 ; 67 (1). 25-32
84. van der Meer MAA, Redish AD. Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci. 2009 ; 3 (1). 1-15
85. van der Meer MAA, Redish AD. Theta phase precession in rat ventral striatum links place and reward information. J Neurosci. 2011 ; 31 (8). 2843-54
86. Wikenheiser AM, Redish AD. Changes in reward contingency modulate the trial-to-trial variability of hippocampal place cells. J Neurophysiol. 2011 ; 106 (2). 589-98
87. Yin HH, Knowlton BJ. Contributions of striatal subregions to place and response learning. Learn Mem. 2004 ; 11 (4). 459-63 (Pubitemid 39079705)