Of goals and habits: Age-related and individual differences in goal-directed decision-making

Frontiers in Neuroscience

Volumn , Issue 7 DEC, 2013, Pages

  Eppinger, Ben ^a,b   Walter, Maik ^c   Heekeren, Hauke R ^b,c   Li, Shu Chen ^a,b  

^a DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

^b MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

^c FREIE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Aging; Decision making and learning; Dopamine; Goal directed; Habitual

Indexed keywords

ADULT; AGE; AGE DISTRIBUTION; AGED; ARTICLE; COMPUTER PROGRAM; COMPUTER SIMULATION; CONTROLLED STUDY; DECISION MAKING; FEMALE; GOAL DIRECTED DECISION MAKING; GROUPS BY AGE; HABIT; HABITUAL DECISION MAKING; HUMAN; HUMAN EXPERIMENT; MALE; MARKOV DECISION TASK; MENTAL PERFORMANCE; MENTAL TASK; MOTIVATION; NEUROLOGIC EXAMINATION; NORMAL HUMAN; PERSEVERATION; PROCESS MODEL; RESPONSE TIME; REWARD; STIMULUS RESPONSE; WORKING MEMORY;

EID: 84896993947     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2013.00253     Document Type: Article

References (51)

1. Baddeley, A. D., Emslie, H., and Nimmo-Smith, I. (eds.). (1992). The Speed and Capacity of Language Processing (SCOLP) Test. Bury St. Edmunds, Suffolk, England: Thames Valley Test Company.
2. Balleine, B. W., and O'Doherty, J. P. (2010). Human and rodent homologies in action control: cortico-striatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35, 48-69. doi: 10.1038/npp.2009.131
3. Bates, D., Maechler, M., and Bolker, B. (2013). lme4: linear mized-effects models using S4 classes. R version 3.0.2.
4. Braver, T. S., and Cohen, J. D. (2000). "On the control of control: the role of dopamine in regulating prefrontal function and working memory," in Attention and Performance XVIII, eds. S. Monsell and J. Driver (Cambridge, MA: MIT Press), 713-737.
5. Carver, C. S., and White, T. L. (1994). Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J. Pers. Soc. Psychol. 67, 319-333. doi: 10.1037/0022-3514.67.2.319
6. Chowdury, R., Guitart-Masip, M., Lambert, C., Dayan, P., Huys, Q., Duezel, E., et al. (2013). Dopamine restores reward prediction errors in old age. Nat. Neurosci. 16, 648-653. doi: 10.1038/nn.3364
7. D'Ardenne, K., Eshel, N., Luka, J., Lenartowicz, A., Nystrom, L. E., and Cohen, J. D. (2012). Role of prefrontal cortex and the midbrain dopamine system in working memory updating. Proc. Natl. Acad. Sci. U.S.A.109, 19900-19909. doi: 10.1073/pnas.1116727109
8. D'Ardenne, K., Mcclure, S. M., Nystrom, L., and Cohen, J. D. (2008). BOLD Responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319, 1264-1267. doi: 10.1126/science.1150605
9. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., and Dolan, R. J. (2011). Model-based influences on humans' choices and striatal prediction errors. Neuron 69, 1204-1215. doi: 10.1016/j.neuron.2011.02.027
10. Daw, N. D., Niv, Y., and Dayan, P. (2005). Human and rodent homologies in action control: cortico-striatal determinants of goal-directed and habitual action. Nat. Neurosci. 8, 1704-1711. doi: 10.1038/nn1560
11. Doll, B. B., Simon, D. A., and Daw, N. D. (2012). The ubiquity of model-based reinforcement learning. Curr. Opin. Neurobiol. 22, 1075-1081. doi: 10.1016/j.conb.2012.08.003
12. Duncan, J., Schramm, M., Thompson, R., and Dumontheil, I. (2012). Task rules, working memory and fluid intelligence. Psychon. Bull. Rev. 19, 864-870. doi: 10.3758/s13423-012-0225-y
13. Ekstrom, R. B., French, J. W., and Harman, H. H. (1976). Kit of Factor-Referenced Cognitive Tests. Princeton, NJ: E.T. Service.
14. Eppinger, B., Haemmerer, D., and Li, S.-C. (2011). Neuromodulation of reward-based learning and decision making in human aging. Ann. N.Y. Acad. Sci. 1235, 1-17. doi: 10.1111/j.1749-6632.2011.06230.x
15. Eppinger, B., Heekeren, H. R., and Li, S.-C. (2012). "When two birds in the bush are better than one in the hand: age-related impairments in learning to predict future rewards," in Annual Meeting of the Society for Neuroscience, (New Orleans, LA).
16. Eppinger, B., and Kray, J. (2011). To choose or to avoid: age differences in learning form positive and negative feedback. J. Cogn. Neurosci. 23, 41-52. doi: 10.1162/jocn.2009.21364
17. Eppinger, B., Kray, J., Mock, B., and Mecklinger, A. (2008). Better or worse than expected? Aging, Learning, and the ERN. Neuropsychologia 46, 521-539. doi: 10.1016/j.neuropsychologia.2007.09.001
18. Eppinger, B., Schuck, N. W., Nystrom, L. E., and Cohen, J. D. (2013). Reduced striatal responses to reward prediction errors in older compared to younger adults. J. Neurosci. 33, 9905-9912. doi: 10.1523/JNEUROSCI.2942-12.2013
19. Frank, M. J., and Kong, L. (2008). Learning to avoid in older age. Psychol. Aging 23, 392-398. doi: 10.1037/0882-7974.23.2.392
20. Frank, M. J., Loughry, B., and O'Reilly, R. C. (2001). Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 1, 137-160. doi: 10.3758/CABN.1.2.137
21. Gershman, S. J., Markman, A. B., and Otto, A. R. (2013). Retrospective revaluation in sequential decision making: a tale of two systems. J. Exp. Psychol. Gen. doi: 10.1037/a0030844. [Epub ahead of print].
22. Gläscher, J., Daw, N. D., Dayan, P., and O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585-595. doi: 10.1016/j.neuron.2010.04.016
23. Hämmerer, D., and Eppinger, B. (2012). Dopaminergic and prefrontal contributions to reward-based learning and outcome monitoring during child development and aging. Dev. Psychol. 48, 862-874. doi: 10.1037/a0027342
24. Hämmerer, D., Li, S.-C., Mueller, V., and Lindenberger, U. (2011). Lifespan differences in electrophysiological correlates of monitoring gains and losses during probabilistic reinforcement learning. J. Cogn. Neurosci. 23, 579-592. doi: 10.1162/jocn.2010.21475
25. Kahneman, D. (2011). Thinking, Fast and Slow. New York. NY: Farrar, Strauss, Giroux.
26. Lau, B., and Glimcher, P. W. (2005). Dynamic response-by-response models of matching behavior in Rhesus monkeys. J. Exp. Anal. Behav. 84, 555-579. doi: 10.1901/jeab.2005.110-04
27. Li, S.-C., Lindenberger, U., Hommel, B., Aschersleben, G., Prinz, W., and Baltes, P. B. (2004). Transformations in the couplings among intellectual abilities and constituent cognitive processes across the lifespan. Psychol. Sci. 15, 155-163. doi: 10.1111/j.0956-7976.2004.01503003.x
28. Mata, R., Von Helversen, B., and Rieskamp, J. (2010). Learning to choose: cognitive aging and strategy selection learning in decision making. Psychol. Aging 25, 299-309. doi: 10.1037/a0018923
29. Miller, E. K., and Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167-202. doi: 10.1146/annurev.neuro.24.1.167
30. Montague, P. R., Hyman, S. E., and Cohen, J. D. (2004). Computational roles for dopamine in behavioral control. Nature 431, 760-767. doi: 10.1038/nature03015
31. Nieuwenhuis, S., Ridderinkhof, K. R., Talsma, D., Coles, M. G. H., Holroyd, C. B., Kok, A., et al. (2002). A computational account of altered error processing in older age: dopamine and the error-related negativity. Cogn. Affect. Behav. Neurosci. 2, 19-36. doi: 10.3758/CABN.2.1.19
32. Niv, Y., Edlund, J. A., Dayan, P., and O'Doherty, J. P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. J. Neurosci. 32, 551-562. doi: 10.1523/JNEUROSCI.5498-10.2012
33. Niv, Y., and Schoenbaum, G. (2008). Dialogues on prediction errors. Trends Cogn. Sci. 12, 265-272. doi: 10.1016/j.tics.2008.03.006
34. Otto, A. R., Gershman, S. J., Markman, A. B., and Daw, N. D. (2013). The curse of planning: dissecting multiple reinforcement learning systems by taxing the central executive. Psychol. Sci. 24, 751-761. doi: 10.1177/0956797612463080
35. Pietschmann, M., Endrass, T., Czerwon, B., and Kathmann, N. (2011). Aging, probabilistic learning and performance monitoring. Biol. Psychol. 86, 74-82. doi: 10.1016/j.biopsycho.2010.10.009
36. Raven, J. C., Raven, J. E., and Court, J. H. (eds.). (1998). Progressive Matrices. Oxford: Oxford Psychologists Press.
37. R Development Core and Team (2010). R: A Language and Environment for Statistical Computing. Vienna: R.F.F.S. Computing.
38. Rummery, G., and Niranjan, M. (1994). On-line Q-learning Using Connectionist Systems. Cambridge: Cambridge University.
39. Salthouse, T. A. (1994). The aging of working memory. Neuropsychology 8, 535-543. doi: 10.1037/0894-4105.8.4.535
40. Salthouse, T. A., Mitchell, D. R. D., Skovronek, E., and Babcock, R. L. (1989). Effects of adult age and working memory on reasoning and spatial abilities. J. Exp. Psychol. Learn. Mem. Cogn. 15, 507-516. doi: 10.1037/0278-7393.15.3.507
41. Samanez-Larkin, G. R., Kuhnen, C. M., Yoo, D. J., and Knutson, B. (2010). Variability in nucleus accumbens activity mediates age-related suboptimal financial risk taking. J. Neurosci. 27, 1426-1434.
42. Samanez-Larkin, G. R., Levens, S. M., Perry, L. M., Dougherty, R. F., and Knutson, B. (2012). Frontostriatal white matter integrity mediates adult age differences in probabilistic reward learning. J. Neurosci. 32, 5333-5337. doi: 10.1523/JNEUROSCI.5756-11.2012
43. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of predicition and reward. Science 275, 1593-1599. doi: 10.1126/science.275.5306.1593
44. Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). Cambridge, MA: MIT Press.
45. Thorndike, E. L. (1911). Animal Intelligence. New York, NY: Macmillan.
46. Tolman, E. C. (1948). Cognitive maps in rats and men. Psychol. Rev. 55, 189-208. doi: 10.1037/h0061626
47. Turner, M. L., and Engle, R. W. (1989). Is working memory capacity task dependent. J. Mem. Lang. 28, 127-154. doi: 10.1016/0749-596X(89)90040-5
48. Unsworth, N., Heitz, R. P., Dschrock, J. C., and Engle, R. W. (2005). An automated version of the operation span task. Behav. Res. Methods 37, 498-505. doi: 10.3758/BF03192720
49. Worthy, D. A., Gorlick, M. A., Pacheco, J. L., Schnyer, D. M., and Maddox, W. T. (2011). With age come wisdom: decision making in younger and older adults. Psychol. Sci. 22, 1375-1380. doi: 10.1177/0956797611420301
50. Worthy, D. A., and Maddox, W. T. (2012). Age-based differences in strategy use in choice tasks. Front. Neurosci. 5:145. doi: 10.3389/fnins.2011.00145
51. Wunderlich, K., Smittenaar, P., and Dolan, R. J. (2012). Dopamine enhances model-based over model-free behavior. Neuron 75, 418-424. doi: 10.1016/j.neuron.2012.03.042