Human dorsal striatal activity during choice discriminates reinforcement learning behavior from the gambler's fallacy

Journal of Neuroscience

Volumn 31, Issue 17, 2011, Pages 6296-6304

  Jessup, Ryan K ^a,b   O'Doherty, John P ^a,b  

^a TRINITY COLLEGE DUBLIN   (Ireland)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BEHAVIOR CONTROL; BRAIN FUNCTION; CONTROLLED STUDY; CORPUS STRIATUM; DECISION MAKING; DORSAL STRIATUM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GAMBLER FALLACY; HUMAN; HUMAN EXPERIMENT; MALE; NEUROIMAGING; PREDICTION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; STIMULUS RESPONSE;

CHOICE BEHAVIOR; CORPUS STRIATUM; DISCRIMINATION (PSYCHOLOGY); FEMALE; GAMBLING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LINEAR MODELS; MAGNETIC RESONANCE IMAGING; MALE; MODELS, STATISTICAL; OXYGEN; REINFORCEMENT (PSYCHOLOGY); REINFORCEMENT SCHEDULE;

EID: 79955770628     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.6421-10.2011     Document Type: Article

References (50)

1. Akitsuki Y, Sugiura M, Watanabe J, Yamashita K, Sassa Y, Awata S, Matsuoka H, Maeda Y, Matsue Y, Fukuda H, Kawashima R (2003) Context-dependent cortical activation in response to financial reward and penalty: an event-related fMRI study. Neuroimage 19:1674-1685.
2. Atallah HE, Lopez-Paniagua D, Rudy JW, O'Reilly RC (2007) Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10:126-131.
3. Ayton P, Fischer I (2004) The hot hand fallacy and the gambler's fallacy: two faces of subjective randomness? Mem Cognit 32:1369-1378.
4. Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neurophar-macology 37:407-419.
5. Balleine BW, O'Doherty JP (2010) Human and rodent homologiesin action control: corticostriatal determinantsof goal-directed and habitual action. Neuropsychopharmacology 35:48-69.
6. Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal stria-tum in reward and decision-making. J Neurosci 27:8161-8165.
7. Barron G, Leider S (2010) The role of experience in the gambler's fallacy. J Behav Decis Mak 23:117-129.
8. Croson R, Sundali J (2005) The gambler's fallacy and the hot hand: empirical data from casinos. J Risk Uncertain 30:195-209.
9. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704-1711.
10. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature 441:876-879.
11. Deichmann R, Gottfried JA, Hutton C, Turner R (2003) Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage 19:430-441.
12. Diederich A, Busemeyer JR (1999) Conflict and the stochastic-dominance principle of decision making. Psychol Sci 10:353-359.
13. Doya K (1999) What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Netw 12:961-974.
14. Doya K (2002) Metalearning and neuromodulation. Neural Netw 15:495-506.
15. Estes WK (1964) Probability learning. In: Categories of human learning (Melton AW, ed), pp 89-128. New York: Academic.
16. Frank MJ, Seeberger LC, O'reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943.
17. Gläscher J, Daw N, Dayan P, O'Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66:585-595.
18. Grether DM, Plott CR (1979) Economic theoryof choice and the preference reversal phenomenon. Am Econ Rev 69:623-638.
19. Hahn U, Warren PA (2009) Perceptions of randomness: why three heads are better than four. Psychol Rev 116:454-461.
20. Haruno M, Kuroda T, Doya K, Toyama K, Kimura M, Samejima K, Imamizu H, Kawato M (2004) A neural correlate of reward-based behavioral learning in caudate nucleus: a functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 24:1660-1665.
21. Hikosaka O, Nakamura K, Nakahara H (2006) Basal ganglia orient eyes to reward. J Neurophysiol 95:567-584.
22. Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304-309.
23. Huettel SA, Mack PB, McCarthy G (2002) Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex. Nat Neurosci 5:485-490.
24. Jarvik ME (1951) Probability learning and a negative recency effect in the serial anticipation of alternative symbols. J Exp Psychol 41:291-297.
25. Jessup RK, Bishara AJ, Busemeyer JR (2008) Feedback produces divergence from prospect theory in descriptive choice. Psychol Sci 19:1015-1022.
26. Kahneman D, Frederick S (2002) Representativeness revisited: attribute substitution in intuitive judgment. In: Heuristics and biases: the psychology of intuitive judgment (Gilovich T, Griffin D, Kahneman D, eds), pp 49-81. New York: Cambridge UP.
27. Knutson B, Cooper JC (2005) Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol 18:411-417.
28. Lau B, Glimcher PW (2008) Value representations in the primate striatum during matching behavior. Neuron 58:451-463.
29. Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural correlate of response bias in monkey caudate nucleus. Nature 418:413-417.
30. Luce RD (1959) Individual choice behavior: a theoretical analysis. New York: Wiley.
31. McClure SM, Laibson DI, Loewenstein G, Cohen JD (2004) Separate neural systems value immediate and delayed monetary rewards. Science 306:503-507.
32. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesen-cephalic dopamine systems based on predictive Hebbian learning. J Neu-rosci 16:1936-1947.
33. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454.
34. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337.
35. O'Doherty JP, Hampton A, Kim H (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104:35-53.
36. Prevost C, McCabe JA, Jessup RK, Bossaerts P, O'Doherty JP (2011) Differ-entiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning. Eur J Neurosci, In press.
37. Rabin M (2002) Inference bybelieversinthe lawofsmall numbers. Q JEcon 117:775-816.
38. Roitman MF, Stuber GD, Phillips PE, Wightman RM, Carelli RM (2004) Dopamine operates as a subsecond modulatoroffood seeking. J Neurosci 24:1265-1271.
39. Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337-1340.
40. Schonberg T, Daw ND, Joel D, O'Doherty JP (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci 27:12860-12867.
41. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.
42. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461-464.
43. Shiffrin RM, Schneider W (1977) Controlled and automatic human information processing: II. Perceptual learning, automatic attending, and a general theory. Psychol Rev 84:127-190.
44. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
45. Tricomi EM, Delgado MR, Fiez JA (2004) Modulation of caudate activity by action contingency. Neuron 41:281-292.
46. Tversky A, Kahneman D (1971) Belief in the law of small numbers. Psychol Bull 76:105-110.
47. Valentin VV, O'Doherty JP (2009) Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain. J Neurophysiol 102:3384-3391.
48. Valentin VV, Dickinson A, O'Doherty JP (2007) Determining the neural substrates of goal-directed learning in the human brain. J Neurosci 27:4019-4126.
49. Wickens JR, Reynolds JN, Hyland BI (2003) Neural mechanisms of reward-related motor learning. Curr Opin Neurobiol 13:685-690.
50. Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formationin instrumental learning. Eur J Neurosci 19:181-189.