A Healthy Fear of the Unknown: Perspectives on the Interpretation of Parameter Fits from Computational Models in Neuroscience

PLoS Computational Biology

Volumn 9, Issue 4, 2013, Pages

  Nassar, Matthew R ^a   Gold, Joshua I ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATION THEORY; DIGITAL STORAGE; REINFORCEMENT LEARNING;

BEHAVIORAL DATA; COMPUTATIONAL MODELLING; DATA SET; FITTING MODEL; GOODNESS OF FIT; INCOMPLETE MODEL; LEARNING TASKS; MODEL FIT; MODEL SELECTION; REINFORCEMENT LEARNING MODELS;

COMPUTATIONAL METHODS;

ACCURACY; ANALYTICAL ERROR; ARTICLE; BEHAVIOR; CLASSIFICATION ALGORITHM; HUMAN; INTERMETHOD COMPARISON; LEARNING THEORY; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NEUROSCIENCE; PROCESS DESIGN; TASK PERFORMANCE;

EID: 84876943382     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003015     Document Type: Article

References (35)

1. Beeler JA, Daw N, Frazier CRM, Zhuang X, (2010) Tonic dopamine modulates exploitation of reward learning. Front Behav Neurosci 4: 170.
2. Doll BB, Hutchison KE, Frank MJ, (2011) Dopaminergic genes predict individual differences in susceptibility to confirmation bias. J Neurosci 31: 6188-6198.
3. Frank MJ, Doll BB, Oas-Terpstra J, Moreno F, (2009) Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat Neurosci 12: 1062-1068.
4. Jepma M, Nieuwenhuis S, (2011) Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory. J Cogn Neurosci 23: 1587-1596.
5. Seo H, Lee D, (2008) Cortical mechanisms for reinforcement learning in competitive games. Philos Trans R Soc Lond B Biol Sci 363: 3845-3857.
6. Strauss GP, Frank MJ, Waltz JA, Kasanova Z, Herbener ES, et al. (2011) Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biol Psychiatry 69: 424-431.
7. Sul JH, Jo S, Lee D, Jung MW, (2011) Role of rodent secondary motor cortex in value-based action selection. Nat Neurosci 14: 1202-1208.
8. Walton ME, Behrens TEJ, Buckley MJ, Rudebeck PH, Rushworth MFS, (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65: 927-939.
9. Nassar MR, Wilson RC, Heasly B, Gold JI, (2010) An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J Neurosci 30: 12366-12378.
10. Behrens TE, Woolrich MW, Walton ME, Rushworth MF, (2007) Learning the value of information in an uncertain world. Nat Neurosci 10: 1214-1221.
11. Krugel LK, Biele G, Mohr PNC, Li S-C, Heekeren HR, (2009) Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proc Natl Acad Sci U S A 106: 17951-17956.
12. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ, (2006) Cortical substrates for exploratory decisions in humans. Nature 441: 876-879.
13. Luksys G, Gerstner W, Sandi C, (2009) Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning. Nat Neurosci 12: 1180-1186.
14. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts. pp. 64-99.
15. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge (Massachusetts): MIT Press. 342 p.
16. Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming. Belmont (Massachusetts): Athena Scientific. 512 p.
17. Dayan P, Abbott LF (2001) Theoretical neuroscience. Cambridge (Massachusetts): MIT Press.
18. Shanks DR (1995) The psychology of associative learning (problems in the behavioral sciences). Gray J, editor. Cambridge: Cambridge University Press. 194 p.
19. Miller RR, Barnet RC, Grahame NJ, (1995) Assessment of the Rescorla-Wagner model. Psychol Bull 117: 363.
20. Dayan P, Daw N, (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8: 429-453.
21. Schultz W, Dayan P, Montague PR, (1997) A neural substrate of prediction and reward. Science 275: 1593-1599.
22. Huys QJM, Moutoussis M, Williams J, (2011) Are computational models of any use to psychiatry? Neural Netw 24: 544-551.
23. Huys Q, Vogelstein J, Dayan P, (2009) Psychiatry: insights into depression through normative decision-making models. Adv Neural Inf Process Syst 21: 729-736.
24. Maia TV, Frank MJ, (2011) From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 14: 154-162.
25. Ishii S, Yoshida W, Yoshimoto J, (2002) Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw 15: 665-687.
26. Yu AJ, Dayan P, (2005) Uncertainty, neuromodulation, and attention. Neuron 46: 681-692.
27. Mathys C, Daunizeau J, Friston KJ, Stephan KE, (2011) A bayesian foundation for individual learning under uncertainty. Front Hum Neurosci 5: 39.
28. Preuschoff K, Bossaerts P, (2007) Adding prediction risk to the theory of reward learning. Ann N Y Acad Sci 1104: 135-146.
29. Lau B, Glimcher PW, (2005) Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84: 555-579.
30. Schönberg T, Daw ND, Joel D, O'Doherty JP, (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci 27: 12860-12867.
31. Wilson RC, Nassar MR, Gold JI, (2010) Bayesian online learning of the hazard rate in change-point problems. Neural Comput 22: 2452-2476.
32. Daw ND (2009) Trial-by-trial data analysis using computational models. In: Phelps E, Robbins T, Delgrado M, editors. Decision making, affect, and learning: attention and performance XXIII. Oxford: Oxford University Press. pp. 3-38.
33. Ding L, Gold JI, (2012) Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron 75: 865-874.
34. Frank MJ, Seeberger LC, O'Reilly RC, (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306: 1940-1943.
35. Pitt MA, Myung IJ, (2002) When a good fit can be bad. Trends Cogn Sci 6: 421-425.