Learning and generalization under ambiguity: An fmri study

PLoS Computational Biology

Volumn 8, Issue 1, 2012, Pages

  Chumbley, J R ^a,b   Flandin, G ^a   Bach, D R ^a   Daunizeau, J ^a,b   Fehr, E ^b   Dolan, R J ^a   Friston, K J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

'CURRENT; ADAPTIVE BEHAVIOUR; BAYESIAN LEARNING; CONTEXT-SENSITIVE; GENERALISATION; LEARNING THEORY; NEURONAL RESPONSE; NEURONAL SYSTEMS; PRIOR EXPERIENCE; UNCERTAINTY;

STATISTICAL TESTS;

ADULT; AMBIGUITY; ARTICLE; BAYESIAN LEARNING; DECISION MAKING; EXPERIENCE; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HIPPOCAMPUS; HUMAN; HUMAN EXPERIMENT; LEARNING; MALE; NEUROIMAGING; NORMAL HUMAN; REINFORCEMENT; REWARD; STATISTICAL MODEL; TASK PERFORMANCE; BAYES THEOREM; BIOLOGY; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY;

ADULT; BAYES THEOREM; CHOICE BEHAVIOR; COMPUTATIONAL BIOLOGY; FEMALE; GENERALIZATION (PSYCHOLOGY); HIPPOCAMPUS; HUMANS; LEARNING; LOGISTIC MODELS; MAGNETIC RESONANCE IMAGING; MALE; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 84857465048     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002346     Document Type: Article

References (67)

1. Bishop C, (2006) Pattern Recognition and Machine Learning. Springer.
2. Beck A, (1993) Cognitive Therapy and the Emotional Disorders NY Penguin.
3. Dayan P, Huys QJM, (2008) Serotonin, inhibition, and negative mood. PLoS Comp Biol 4: e4.
4. Huys Q, Vogelstein J, Dayan P, (2009) Psychiatry: insights into depression through normative decision-making models. In: Koller D, Schuurmans D, Bengio Y, Bottou L, editors. Advances in Neural Information Processing Systems 21 Cambrige (Massachusetts) MIT Press pp. 729-736.
5. Carrell PL, Eisterhold JC, (1983) Schema theory and ESL reading pedagogy. TESOL Quarterly 17: 553-573.
6. Hestenes D, (1987) Toward a modeling theory of physics instruction. Am J Phys 55: 440-454.
7. Tenebaum JB, Griffiths TL, (2001) Generalization, similarity, and Bayesian inference. Behav Brain Sci 24: 629-+.
8. Griffiths TL, Kemp C, Tenenbaum JB, (2008) Bayesian models of cognition. In: Sun Ron, editors. The Cambridge handbook of computational cognitive modeling Cambridge University Press.
9. O'Reilly RC, Norman KA, (2002) Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. Trends Cogn Sci 6: 505-510.
10. Wittmann BC, Bunzeck N, Dolan RJ, Duzel E, (2007) Anticipation of novelty recruits reward system and hippocampus while promoting recollection. Neuro Image 38: 194-202.
11. Yu AJ, Dayan P, (2005) Uncertainty, neuromodulation, and attention. Neuron 46: 681-692.
12. Chater N, Tenenbaum J, Yuille A, (2006) Probabilistic models of cognition: Conceptual foundations. Trends Cogn Sci 10: 287-291.
13. Schultz W, Preuschoff K, Camerer C, Hsu M, Fiorillo C, et al. (2008) Explicit neural signals reflecting reward uncertainty. Philos Trans R Soc Lond, Ser B: Biol Sci 363: 3801.
14. Payzan-LeNestour E, Bossaerts P, (2011) Risk, Unexpected Uncertainty, and Estimation Uncertainty: Bayesian Learning in Unstable Settings. PLoS Comp Biol 7: e1001048.
15. Rescorla R, Wagner A, (1972) Variations in the Effectiveness of Reinforcement and Nonreinforcement. Classical Conditioning II: Current Research and Theory New York Appleton-Century-Crofts.
16. Schultz W, Dayan P, Montague PR, (1997) A neural substrate of prediction and reward. Science 275: 1593-1599.
17. Zilli EA, Hasselmo ME, (2008) Modeling the role of working memory and episodic memory in behavioral tasks. Hippocampus 18: 193-209.
18. Sutton R, Barto A, (1998) Reinforcement learning: An introduction Cambridge The MIT Press.
19. Gustafson NJ, Daw ND, (2011) Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning. PLoS Comp Biol 7: e1002235.
20. Ludvig EA, Sutton RS, Kehoe EJ, (2008) Stimulus representation and the timing of reward-prediction errors in models of the dopamine system. Neural Comput 20: 3034-3054.
21. Preuschoff K, Bossaerts P, (2007) Adding prediction risk to the theory of reward learning. Ann N Y Acad Sci 1104: 135-146.
22. Cox RT, (1946) Probability, Frequency and reasonable expectation. Am J Phys 14: 1-13.
23. Strange BA, Duggins A, Penny W, Dolan RJ, Friston KJ, (2005) Information theory, novelty and hippocampal responses: unpredicted or unpredictable? Neural Netw 18: 225-230.
24. Hsu M, Lin H, McNamara P, (2008) Neuroeconomics of decision-making in the aging brain: the example of long-term care. Adv Health Econ Health Serv Res 20: 203-225.
25. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF, (2005) Neural systems responding to degrees of uncertainty in human decision-making. Science 310: 1680.
26. Bach DR, Hulme O, Penny WD, Dolan RJ, (2011) The Known Unknowns: Neural Representation of Second-Order Uncertainty, and Ambiguity. J Neurosci 31: 4811.
27. Kemp C, Perfors A, Tenenbaum JB, (2007) Learning overhypotheses with hierarchical Bayesian models. Dev Sci 10: 307-321.
28. Congdon P, (2003) Applied bayesian modelling John Wiley & Sons Inc.
29. Greene A, Gross W, Elsinger C, Rao S, (2006) An FMRI analysis of the human hippocampus: inference, context, and task awareness. J Cogn Neurosci 18: 1156-1173.
30. Shohamy D, Wagner A, (2008) Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events. Neuron 60: 378-389.
31. Zeithamova D, Preston A, (2010) Flexible Memories: Differential Roles for Medial Temporal Lobe and Prefrontal Cortex in Cross-Episode Binding. J Neurosci 30: 14676.
32. Heckers S, Zalesak M, Weiss A, Ditman T, Titone D, (2004) Hippocampal activation during transitive inference in humans. Hippocampus 14: 153-162.
33. Preston A, Shrager Y, Dudukovic N, Gabrieli J, (2004) Hippocampal contribution to the novel use of relational information in declarative memory. Hippocampus 14: 148-152.
34. Eichenbaum H, Cohen N, (2004) From conditioning to conscious recollection: Memory systems of the brain. Oxford University Press, USA.
35. Harrison LM, Duggins A, Friston KJ, (2006) Encoding uncertainty in the hippocampus. Neural Netw 19: 535-546.
36. Weiskopf N, Hutton C, Josephs O, Deichmann R, (2006) Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: A whole-brain analysis at 3 T and 1.5 T. NeuroImage 33: 493-504.
37. Weiskopf N, Hutton C, Josephs O, Turner R, Deichmann R, (2007) Optimized EPI for fMRI studies of the orbitofrontal cortex: compensation of susceptibility-induced gradients in the readout direction. MAGMA 20: 39-49.
38. Deichmann R, Schwarzbauer C, Turner R, (2004) Optimisation of the 3D MDEFT sequence for anatomical brain imaging: Technical implications at 1.5 and 3 T. NeuroImage 21: 757-767.
39. Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS, (2007) Learning the value of information in an uncertain world. Nat Neurosci 10: 1214-1221.
40. Teh YW, Jordan MI, Beal MJ, Blei DM, (2006) Hierarchical dirichlet processes. J Am Stat Assoc 101: 1566-1581.
41. Griffiths TL, Canini KR, Sanborn AN, Navarro DJ, (2007) Unifying rational models of categorization via the hierarchical Dirichlet process. Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society.
42. Gershman SJ, Blei DM, Niv Y, (2010) Context, learning, and extinction. Psychol Rev 117: 197.
43. Redish AD, Jensen S, Johnson A, Kurth-Nelson Z, (2007) Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychol Rev 114: 784-805.
44. Sethuraman J, (1994) A constructive definition of Dirichlet priors. Statistica Sinica 4: 639-650.
45. Teh Y, (2010) Dirichlet Processes. In: Sammut CW, Geoffrey I, editors. Encyclopedia of Machine Learning Springer.
46. Blackwell D, MacQueen JB, (1973) Ferguson distributions via Polya urn schemes. Ann Stat 1: 353-355.
47. Neal RM, (2000) Markov chain sampling methods for Dirichlet process mixture models. J Comput Graph Stat 9: 249-265.
48. Escobar MD, West M, (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90: 577-588.
49. Gelman A, (2004) Bayesian data analysis. Florida. CRC press.
50. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ, (2006) Cortical substrates for exploratory decisions in humans. Nature 441: 876-879.
51. Kakade S, Dayan P, (2002) Dopamine: generalization and bonuses. Neural Netw 15: 549-559.
52. Dayan P, Niv Y, Seymour B, Daw ND, (2006) The misbehavior of value and the discipline of the will. Neural Netw 19: 1153-1160.
53. O'Doherty JP, Hampton A, Kim H, (2007) Model-based fMRI and its application to reward learning and decision making. Ann N Y Acad Sci 1104: 35.
54. Dobson AJ, (2002) An introduction to generalized linear models. CRC Pr I Llc.
55. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, et al. (2002) Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage 15: 273-289.
56. Friston KJ, Holmes A, Poline JB, Price CJ, Frith CD, (1996) Detecting activations in PET and fMRI: Levels of inference and power. NeuroImage 4: 223-235.
57. Vanni-Mercier G, Mauguiere F, Isnard J, Dreher JC, (2009) The Hippocampus Codes the Uncertainty of Cue-Outcome Associations: An Intracranial Electrophysiological Study in Humans. J Neurosci 29: 5287-5294.
58. den Ouden HEM, Daunizeau J, Roiser J, Friston KJ, Stephan KE, (2010) Striatal prediction error modulates cortical coupling. J Neurosci 30: 3210.
59. Hasselmo ME, (1995) Neuromodulation and cortical function- modeling the physiological basis of behaviour. Behav Brain Res 67: 1-27.
60. Dayan P, Niv Y, (2008) Reinforcement learning: The Good, The Bad and The Ugly. Curr Opin Neurobiol 18: 185-196.
61. Cahill L, Weinberger NM, Roozendaal B, McGaugh JL, (1999) Is the amygdala a locus of "conditioned fear"? Some questions and caveats. Neuron 23: 227-228.
62. Prévost C, McCabe JA, Jessup RK, Bossaerts P, O'Doherty JP, (2011) Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning. Eur J Neurosci 34: 134-145.
63. Mackintosh NJ, (1983) Conditioning and associative learning Oxford Clarendon Press.
64. Pearce JM, Hall G, (1980) A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87: 532.
65. Sutton RS, (1992) Gain adaptation beats least squares. pp. 161-166 Citeseer.
66. Foraker S, Regier T, Khetarpal N, Perfors A, Tenenbaum J, (2009) Indirect Evidence and the Poverty of the Stimulus: The Case of Anaphoric One. Cogn Sci 33: 287-300.
67. Huys QJM, Dayan P, (2009) A Bayesian formulation of behavioral control. Cognition 113: 314-328.