Experiential reward learning outweighs instruction prior to adulthood

Cognitive, Affective and Behavioral Neuroscience

Volumn 15, Issue 2, 2015, Pages 310-320

  Decker, Johannes H ^a   Lourenco, Frederico S ^a   Doll, Bradley B ^b   Hartley, Catherine A ^a  

^a WEILL CORNELL MEDICAL COLLEGE   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

Cognitive control; Decision making; Development; Reinforcement learning

Indexed keywords

ADAPTIVE BEHAVIOR; ADOLESCENT; ADULT; AGE; CHILD; DECISION MAKING; DEVELOPMENTAL PSYCHOLOGY; FEMALE; HUMAN; LEARNING; MALE; NEUROPSYCHOLOGICAL TEST; PHYSIOLOGY; REWARD; YOUNG ADULT;

ADOLESCENT; ADULT; AGE FACTORS; CHILD; CHOICE BEHAVIOR; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; LEARNING; MALE; NEUROPSYCHOLOGICAL TESTS; PROBABILITY LEARNING; PSYCHOLOGY, DEVELOPMENTAL; REWARD; YOUNG ADULT;

EID: 84939969840     PISSN: 15307026     EISSN: None     Source Type: Journal    
DOI: 10.3758/s13415-014-0332-5     Document Type: Article

References (60)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, AC-19, 716–723. doi:10.1109/TAC.1974.1100705
2. Amso, D., & Davidow, J. (2012). The development of implicit learning from infancy to adulthood: Item frequencies, relations, and cognitive flexibility. Developmental Psychobiology, 54, 664–673.
3. Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68, 255–278. doi:10.1016/j.jml.2012.11.001
4. Bates, D., Maechler, M., Bolker, B., & Walker, S. (2014). lme4: Linear mixed-effects models using Eigen and S4 [Software]. Retrieved from http://CRAN.R-project.org/package=lme4
5. Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129–141.
6. Biele, G., Rieskamp, J., Krugel, L. K., & Heekeren, H. R. (2011). The neural basis of following advice. PLoS Biology, 9, e1001089. doi:10.1371/journal.pbio.1001089
7. Biele, G., Rieskamp, J., & Gonzalez, R. (2009). Computational models for the combination of advice and individual learning. Cognitive Science, 33, 206–242. doi:10.1111/j.1551-6709.2009.01010.x
8. Bischoff-Grethe, A., Goedert, K. M., Willingham, D. T., & Grafton, S. T. (2004). Neural substrates of response-based sequence learning using fMRI. Journal of Cognitive Neuroscience, 16, 127–138.
9. Bunge, S. A., & Zelazo, P. D. (2006). A brain-based account of the development of rule use in childhood. Current Directions in Psychological Science, 15, 118–121.
10. Chein, J., Albert, D., O’Brien, L., Uckert, K., & Steinberg, L. (2011). Peers increase adolescent risk taking by enhancing activity in the brain’s reward circuitry. Developmental Science, 14, F1–F10.
11. Cohen, J. R., Asarnow, R. F., Sabb, F. W., Bilder, R. M., Bookheimer, S. Y., Knowlton, B. J., & Poldrack, R. A. (2010). A unique adolescent response to reward prediction errors. Nature Neuroscience, 13, 669–671.
12. Crone, E. A., Wendelken, C., Donohue, S., van Leijenhorst, L., & Bunge, S. A. (2006). Neurocognitive development of the ability to manipulate information in working memory. Proceedings of the National Academy of Sciences, 103, 9315–9320.
13. Daniel, R., & Pollmann, S. (2010). Comparing the neural basis of monetary reward and cognitive feedback during information-integration category learning. Journal of Neuroscience, 30, 47–55.
14. Daw, N. D. (2011). Trial-by-trial data analysis using computational models. In M. R. Delgado, E. A. Phelps, & T. W. Robbins (Eds.), Decision making, affect, and learning: Attention and performance XXIII (pp. 3–38). New York, NY: Oxford University Press.
15. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215.
16. den Ouden, H. E., Daw, N. D., Fernandez, G., Elshout, J. A., Rijpkema, M., Hoogman, M., & Cools, R. (2013). Dissociable effects of dopamine and serotonin on reversal learning. Neuron, 80, 1090–1100.
17. Diamond, A. (2006). The early development of executive functions. In E. Bialystok & F. Craik (Eds.), Lifespan cognition: Mechanisms of change (pp. 70–95). New York, NY: Oxford University Press.
18. Doll, B. B., Hutchison, K. E., & Frank, M. J. (2011). Dopaminergic genes predict individual differences in susceptibility to confirmation bias. Journal of Neuroscience, 31, 6188–6198.
19. Doll, B. B., Jacobs, W. J., Sanfey, A. G., & Frank, M. J. (2009). Instructional control of reinforcement learning: A behavioral and neurocomputational investigation. Brain Research, 1299, 74–94.
20. Doll, B. B., Waltz, J. A., Cockburn, J., Brown, J. K., Frank, M. J., & Gold, J. M. (2014). Reduced susceptibility to confirmation bias in schizophrenia. Cognitive, Affective, & Behavioral Neuroscience, 14, 715–728. doi:10.3758/s13415-014-0250-6
21. Engelmann, J. B., Moore, S., Capra, C. M., & Berns, G. S. (2012). Differential neurobiological effects of expert advice on risky choice in adolescents and adults. Social Cognitive and Affective Neuroscience, 7, 557–567.
22. Ennett, S. T., Tobler, N. S., Ringwalt, C. L., & Flewelling, R. L. (1994). How effective is drug abuse resistance education? A meta-analysis of Project DARE outcome evaluations. American Journal of Public Health, 84, 1394–1401.
23. Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T., & Hutchison, K. E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences, 104, 16311–16316.
24. Frank, M. J., & O’Reilly, R. C. (2006). A mechanistic account of striatal dopamine function in human cognition: Psychopharmacological studies with cabergoline and haloperidol. Behavioral Neuroscience, 120, 497–517. doi:10.1037/0735-7044.120.3.497
25. Frank, M. J., Seeberger, L. C., & O’Reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science, 306, 1940–1943. doi:10.1126/science.1102941
26. Galvan, A. (2014). Insights about adolescent behavior, plasticity, and policy from neuroscience research. Neuron, 83, 262–265.
27. Galvan, A., Hare, T. A., Parra, C. E., Penn, J., Voss, H., Glover, G., & Casey, B. J. (2006). Earlier development of the accumbens relative to orbitofrontal cortex might underlie risk-taking behavior in adolescents. Journal of Neuroscience, 26, 6885–6892.
28. Gardner, M., & Steinberg, L. (2005). Peer influence on risk taking, risk preference, and risky decision making in adolescence and adulthood: An experimental study. Developmental Psychology, 41, 625–635. doi:10.1037/0012-1649.41.4.625
29. Hayes, R. L. (1993). A facilitative role for counselors in restructuring schools. Journal of Humanistic Education and Development, 8, 156–162.
30. Imperati, D., Colcombe, S., Kelly, C., Di Martino, A., Zhou, J., Castellanos, F. X., & Milham, M. P. (2011). Differential development of human brain white matter tracts. PLoS ONE, 6, e23437. doi:10.1371/journal.pone.0023437
31. Jones, R. M., Somerville, L. H., Li, J., Ruberry, E. J., Powers, A., Mehta, N., & Casey, B. J. (2014). Adolescent-specific patterns of behavior and neural activity during social reinforcement learning. Cognitive, Affective, & Behavioral Neuroscience, 14, 683–697. doi:10.3758/s13415-014-0257-z
32. Kirkham, N. Z., Slemmer, J. A., & Johnson, S. P. (2002). Visual statistical learning in infancy: Evidence for a domain general learning mechanism. Cognition, 83, B35–B42.
33. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.
34. Li, J. A., Delgado, M. R., & Phelps, E. A. (2011). How instructed knowledge modulates the neural systems of reward learning. Proceedings of the National Academy of Sciences, 108, 55–60.
35. Liston, C., Watts, R., Tottenham, N., Davidson, M. C., Niogi, S., Ulug, A. M., & Casey, B. (2006). Frontostriatal microstructure modulates efficient recruitment of cognitive control. Cerebral Cortex, 16, 553–560.
36. McClure, S. M., Berns, G. S., & Montague, P. R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38, 339–346.
37. Meshi, D., Biele, G., Korn, C. W., & Heekeren, H. R. (2012). How expert advice influences decision making. PLoS ONE, 7, e49748. doi:10.1371/journal.pone.0049748
38. Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202. doi:10.1146/annurev.neuro.24.1.167
39. Munakata, Y., Snyder, H. R., & Chatham, C. H. (2012). Developing cognitive control: Three key transitions. Current Directions in Psychological Science, 21, 71–77.
40. O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38, 329–337.
41. Pagnoni, G., Zink, C. F., Montague, P. R., & Berns, G. S. (2002). Activity in human ventral striatum locked to errors of reward prediction. Nature Neuroscience, 5, 97–98.
42. Pessiglione, M., Seymour, B., Flandin, G., Dolan, R. J., & Frith, C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature, 442, 1042–1045.
43. Peters, S., Braams, B. R., Raijmakers, M. E., Koolschijn, P. C., & Crone, E. A. (2014). The neural coding of feedback learning across child and adolescent development. Journal of Cognitive Neuroscience, 26, 1705–1720. doi:10.1162/jocn_a_00594
44. Rauch, S. L., Whalen, P. J., Savage, C. R., Curran, T., Kendrick, A., Brown, H. D., & Rosen, B. R. (1997). Striatal recruitment during an implicit sequence learning task as measured by functional magnetic resonance imaging. Human Brain Mapping, 5, 124–132.
45. Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psychology: General, 118, 219–235. doi:10.1037/0096-3445.118.3.219
46. Reyna, V. F., & Farley, F. (2006). Risk and rationality in adolescent decision making implications for theory, practice, and public policy. Psychological Science in the Public Interest, 7, 1–44.
47. Rodriguez, P., Aron, A., & Poldrack, R. (2006). Ventral–striatal/nucleus-accumbens sensitivity to prediction errors during classification learning. Human Brain Mapping, 27, 306–313.
48. Rubia, K., Smith, A. B., Woolley, J., Nosarti, C., Heyman, I., Taylor, E., & Brammer, M. (2006). Progressive increase of frontostriatal brain activation from childhood to adulthood during event-related tasks of cognitive control. Human Brain Mapping, 27, 973–993. doi:10.1002/hbm.20237
49. Rueda, R. M., Posner, M. I., & Rothbart, M. K. (2005). The development of executive attention: Contributions to the emergence of self-regulation. Developmental Neuropsychology, 28, 573–594.
50. Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274, 1926–1928. doi:10.1126/science.274.5294.1926
51. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599.
52. Somerville, L. H., & Casey, B. J. (2010). Developmental neurobiology of cognitive control and motivational systems. Current Opinion in Neurobiology, 20, 236–241. doi:10.1016/j.conb.2010.01.006
53. Staudinger, M. R., & Buchel, C. (2013). How initial confirmatory experience potentiates the detrimental influence of bad advice. NeuroImage, 76, 125–133.
54. Steinberg, L., & Monahan, K. C. (2007). Age differences in resistance to peer influence. Developmental Psychology, 43, 1531–1543. doi:10.1037/0012-1649.43.6.1531
55. Thomas, K. M., Hunt, R. H., Vizueta, N., Sommer, T., Durston, S., Yang, Y., & Worden, M. S. (2004). Evidence of developmental differences in implicit sequence learning: An fMRI study of children and adults. Journal of Cognitive Neuroscience, 16, 1339–1351.
56. Thompson-Schill, S. L., Ramscar, M., & Chrysikou, E. G. (2009). Cognition without control when a little frontal lobe goes a long way. Current Directions in Psychological Science, 18, 259–263.
57. Trenholm, C., Devaney, B., Fortson, K., Quay, K., Wheeler, J., & Clark, M. (2007). Impacts of four title V, section 510 abstinence education programs: Final report. Princeton, NJ: Mathematica Policy Research, Inc.
58. van den Bos, W., Cohen, M. X., Kahnt, T., & Crone, E. A. (2012). Striatum–medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning. Cerebral Cortex, 22, 1247–1255. doi:10.1093/cercor/bhr198
59. van den Bos, W., Güroğlu, B., van den Bulk, B. G., Rombouts, S. A., & Crone, E. A. (2009). Better than expected or as bad as you thought? The neurocognitive development of probabilistic feedback processing. Frontiers in Human Neuroscience, 3(52), 1–11. doi:10.3389/neuro.09.052.2009
60. Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279–292.