Dual learning processes underlying human decision-making in reversal learning tasks: Functional significance and evidence from the model fit to human behavior

Frontiers in Psychology

Volumn 5, Issue AUG, 2014, Pages

  Bai, Yu ^a   Katahira, Kentaro ^a   Ohira, Hideki ^a  

^a NAGOYA UNIVERSITY   (Japan)

Author keywords

Decision making; Learning rate; Reinforcement learning model; Reversal learning; Value

Indexed keywords

EID: 84906346340     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.00871     Document Type: Article

References (23)

1. Behrens, T. E. J., Woolrich, M. W., Walton, M. E., and Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221. doi: 10.1038/nn1954
2. Butter, C. M. (1968). Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in Macaca mulatta. Physiol. Behav. 4, 163-171.
3. Cools, R., Clark, L., Owen, A. M., and Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci. 22, 4563-4567.
4. Corrado, G., and Doya, K. (2007). Understanding neural coding through the model-based analysis of decision making. J. Neurosci. 27, 8178-8180. doi: 10.1523/JNEUROSCI.1590-07.2007
5. Courville, A. C., Daw, N. D., and Touretzky, D. S. (2006). Bayesian theories of conditioning in a changing world. Trends Cogn. Sci. 10, 294-300. doi: 10.1016/j.tics.2006.05.004
6. Daw, N. D. (2011). "Trial-by-trial data analysis using computational models," in Decision Making, Affect, and Learning: Attention and Performance XXIII , eds M. R. Delgado, E. A. Phelps, and T. W. Robbins (Oxford: Oxford University Press), 1-26. doi: 10.1093/acprof:oso/9780199600434.001.0001
7. Daw, N. D., and Doya, K. (2006). The computational neurobiology of learning and reward. Curr. Opin. Neurobiol. 16, 199-204 doi: 10.1016/j.conb.2006.03.006
8. Erdeniz, B., and Atalay, N. B. (2010). "Simulating probability learning and probabilistic reversal learning using the attention-gated reinforcement learning (AGREL) model," in The 2010 International Joint Conference on Neural Networks (IJCNN) , 1-6. doi: 10.1109/IJCNN.2010.5596783
9. Kakade, S., and Dayan, P. (2002). Acquisition and extinction in autoshaping. Psychol. Rev. 109, 533-544. doi: 10.1037//0033-295X.109.3.533
10. Li, J., Schiller, D., Schoenbaum, G., Phelps, E. A., and Daw, N. D. (2011). Differential roles of human striatum and amygdala in associative learning. Nat. Neurosci. 14, 1250-1252. doi: 10.1038/nn.2904
11. McAlonan, K., and Brown, V. J. (2003). Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav. Brain Res. 146, 97-103. doi: 10.1016/j.bbr.2003.09.019
12. O'Doherty, J., Critchley, H., Deichmann, R., and Dolan, R. J. (2003). Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 23, 7931-7939.
13. Ohira, H., Matsunaga, M., Kimura, K., Murakami, H., Osumi, T., Isowa, T., et al. (2011). Chronic stress modulates neural and cardiovascular responses during reversal learning. Neuroscience 193, 193-204. doi: 10.1016/j.neuroscience.2011.07.014
14. Pearce, J. M., and Hall, G. (1980). A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532-552.
15. Preuschoff, K., and Bossaerts, P. (2007). Adding prediction risk to the theory of reward learning. Ann. N.Y. Acad. Sci. 1104, 135-146. doi: 10.1196/annals.1390.005
16. Rescorla, R. A., and Wagner, A. R. (1972). A Theory of Pavlovian Conditioning: Variations in the Effectiveness of Reinforcement and Nonreinforcement. New York, NY: Classical Conditioning II, Appleton-Century-Crofts.
17. Roesch, M. R., Calu, D. J., Esber, G. R., and Schoenbaum, G. (2010). Neural correlates of variations in event processing during learning in basolateral amygdala. J. Neurosci. 30, 2464-2471. doi: 10.1523/JNEUROSCI.5781-09.2010
18. Rolls, E. T., Hornak, J., Wade, D., and McGrath, J. (1994). Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. J. Neurol. Neurosurg. Psychiatr. 57, 1518-1524.
19. Schultz, W. (1997). A neural substrate of prediction and reward. Science 275, 1593-1599.
20. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6, 461-464.
21. Sutton, R. S., and Barto, A. G. (1998). Reinforcement learning: an introduction. IEEE Trans. Neural Netw. 9, 1054-1054.
22. Tsuchida, A., Doll, B. B., and Fellows, L. K. (2010). Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J. Neurosci. 30, 16868-16875. doi: 10.1523/JNEUROSCI.1958-10.2010
23. Watkins, C. J. C. H., and Dayan, P. (1992). Q-learning. Mach. Learn. 8, 279-292.