Surprised at all the entropy: Hippocampal, caudate and midbrain contributions to learning from prediction errors

PLoS ONE

Volumn 7, Issue 5, 2012, Pages

  Schiffer, Anne Marike ^a,b   Ahlheim, Christiane ^b,c   Wurm, Moritz F ^b,d   Schubotz, Ricarda I ^b,c  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b MAX PLANCK INSTITUTE FOR NEUROLOGICAL RESEARCH   (Germany)

^c UNIVERSITY OF MÜNSTER   (Germany)

^d UNIVERSITY OF TRENTO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ANALYTICAL ERROR; ARTICLE; BOLD SIGNAL; BOOTSTRAPPING; CAUDATE NUCLEUS; CONTROLLED STUDY; DOPAMINERGIC SYSTEM; ENTROPY; FALSE POSITIVE RESULT; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HABENULA; HIPPOCAMPUS; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; MALE; MESENCEPHALON; NORMAL HUMAN; PERCEPTION; PREDICTION; SENSORY STIMULATION; STATE DEPENDENT LEARNING; STIMULUS RESPONSE; SUBSTANTIA NIGRA;

ADULT; ALGORITHMS; BEHAVIOR; BRAIN MAPPING; CAUDATE NUCLEUS; FEMALE; HIPPOCAMPUS; HUMANS; LEARNING; MAGNETIC RESONANCE IMAGING; MALE; MESENCEPHALON; MODELS, NEUROLOGICAL; YOUNG ADULT;

EID: 84860516327     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0036445     Document Type: Article

References (68)

1. Bubic A, von Cramon DY, Schubotz RI, (2010) Prediction, cognition and the brain. Frontiers in Human Neuroscience Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2904053/. Accessed: 12/22/11.
2. Huang Y, Rao RPN, (2011) Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science 2: 580-593.
3. Montague PR, Dayan P, Sejnowski TJ, (1996) A framework for mescencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16 (5): 1936-1947.
4. Schubotz RI, (2007) Prediction of external events with our motor system: towards a new framework. TiCS 11 (5): 211-218.
5. Schütz-Bosbach S, Prinz W, (2007) Prospective coding in event representation. Cognitive processing 8 (2): 93-102.
6. Wolpert DM, Flanagan JR, (2001) Motor prediction. Current Biology 11 (18): R729-R732.
7. Fiser J, Berkes P, Orbán G, Lengyel M, (2010) Statistically optimal perception and learning: from behavior to neural representations. TiCS 14: 119-130.
8. Friston K, (2010) The free-energy principle: a unified brain theory? Nat. Reviews. Neurosci 11 (2): 127-138.
9. Rao RPN, Ballard DH, (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2: 79-87.
10. Friston KJ, (2002) Functional integration and inference in the brain. Prog Neurobiol 68 (2): 113/143.
11. Friston KJ, (2005) A theory of cortical responses. Philos Trans R Soc Lond 360: 815-836.
12. Kilner JM Friston KJ, Frith CD, (2007) Predictive coding: an account of the mirror neuron system. Cogn Process 8 (3): 159-66.
13. Kersten D, Mamassian M, Yuille A, (2011) Object Perception as Bayesian Inference UC Los Angeles Department of Statistics, UCLA Available: http://escholarship.org/uc/item/69d797cq. Accessed:12/22/11.
14. Knill DC, Pouget A, (2004) The Bayesian brain: The role of uncertainty in neural coding and computation for perception and action, TiN 27 (12): 712-719.
15. Crapse TB, Sommer MA, (2008) The frontal eye field as a prediction map. Progress in brain research 71 (08): 383-90.
16. Kiebel SJ, Daunizeau J, Friston KJ, (2008) A Hierarchy of Time-Scales and the Brain. PLoS Comput Biol 4 (11): e1000209 Available: http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000209. Accessed: 12/22/11.
17. Doya K, Ishii S, (2007) A probability primer. In: Doya K, Ishii S, Pouget A, Rao R, editors. Bayesian Brain: Probabilistic Approach to Neural Coding and Learning Cambridge MIT Press pp. 3-13.
18. Strange BA, Duggins A, Penny W, Dolan RJ, Friston KJ, (2005) Information theory, novelty and hippocampal responses: unpredicted or unpredictable? Neural Netw 18: 225-230.
19. Shannon, CE, (1948) A mathematical theory of communication. Bell System Technical Journal 27: 379-423 and 623-656.
20. Wolpert DM, Kawato M, (1998) Multiple paired forward and inverse models for motor control. Neural Networks 11 (7-8): 1317-1329.
21. Luce RD, (2003) Whatever happened to information theory in psychology? Review of General Psychology 7 (2): 183-188.
22. Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS, (2007) Learning the value of information in an uncertain world. Nat Neurosci 10 (9): 1214-1221.
23. Doeller CF, King JA, Burgess N, (2008) Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. PNAS 105 (15): 5915-5920.
24. Atallah HE, Frank MJ, O'Reilly RC, (2004) Hippocampus, cortex, and basal ganglia: Insights from computational models of complementary learning systems. Neurobiology of Learning and Memory 82: 253-267.
25. Poldrack RA, Packard MG, (2003) Competition among multiple memory systems: converging evidence from animal and human brain studies. Neuropsychologia 41 (3): 245-251.
26. Packard MG, Knowlton BJ, (2002) Learning and Memory Functions of the Basal Ganglia. Annu Rev Neurosci 25: 563-93.
27. Ashby FG, Alfonso-Reese LA, Turken AU, Waldron EM, (1998) A Neuropsychological Theory of Multiple Systems in Category Learning. Psychological Review 105 (3): 442-481.
28. Shohamy D, Adcock RA, (2010) Dopamine and adaptive memory. TiCS 14 (10): 464-472.
29. Kumaran D, Maguire EA, (2006) An unexpected sequence of events: mismatch detection in the human hippocampus. PLoS biology 4 (12): e424 doi:10.1371/journal.pbio.0040424 Available: http://www.plosbiology.org/article/info:doi/10.1371/journal.pbio.0040424. Accessed: 12/22/11.
30. Schultz W, (2000) Multiple reward signals in the brain. Nature Rev Neurosci 1 (3): 199-207.
31. Schultz W, Dayan P, Montague PR, (1997) A neural substrate of prediction and reward. Science 275 (5306): 1593-9.
32. Graybiel AM, (2005) The basal ganglia: learning new tricks and loving it. Current Opinion in Neurobiology 15 (6): 638-644.
33. Jay TM, (2003) Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Progress in Neurobiology 69 (6): 375-390.
34. Duzel E, Habib R, Rotte M, Guderian S, Tulving E, et al. (2003) Human hippocampal and parahippocampal activity during visual associative recognition memory for spatial and nonspatial stimulus configurations. J Neurosci 23: 9439-9444.
35. Lisman JE, Grace AA, (2005) The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory. Neuron 46 (5): 703-713.
36. Gluck MA, Myers C, Meeter M, (2005) Cortico-hippocampal interaction and adaptive stimulus representation: A neurocomputational theory of associative learning and memory. Neural Networks 18: 1265-1279.
37. Kumaran D, Duzel E, (2008) The Hippocampus and Dopaminergic Midbrain: Old Couple, New Insights. Neuron 60 (2): 197-200.
38. O'Reilly R, Norman KA, (2002) Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. TICS 6 (12): 505-510.
39. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston KJ, et al. (2004) Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning. Science 304 (5669): 452-454.
40. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston KJ, et al. (2003) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Current Opinion in Neurobiology 14 (6): 769-776.
41. Schiffer A-M, Schubotz RI, (2011) Caudate nucleus signals for breaches of expectation in a movement observation paradigm. Frontiers in Human Neuroscience 5 Available: http://www.frontiersin.org/human_neuroscience/10.3389/fnhum.2011.00038/full. Accessed: 12/22/11.
42. den Ouden HEJ, Danizeau J, Roiser J, Friston KJ, Stephan KE, (2010) Striatal Prediction Error Modulates Cortical Coupling. J Neurosci 30 (33): 11177-11187.
43. Spicer J, Galván A, Hare TA, Voss H, Glover G, et al. (2007) Sensitivity of the nucleus accumbens to violations in expectation of reward. Neuroimage 34: 455-461.
44. Oldfield RC, (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9: 97-113.
45. Lohmann G, Mueller K, Bosch V, Mentzel H, Hessler S, et al. (2001) Lipsia- a new software system for the evaluation of functional magnetic resonance images of the human brain. Computerized medical imaging and graphics: the official journal of the Computerized Medical Imaging Society 25 (6): 449-457.
46. Talairach J, Tourneaux P, (1988) Co-planar stereotaxic atlas of the human brain Stuttgart Thieme.
47. Worsley KJ, Friston KJ, (1995) Analysis of FMRI time series revisited- again. Neuroimage 2 (3): 173-81.
48. Nichols TE, Holmes AP, (2001) Nonparameric Permutation Tests For Functional Neuroimaging: A Primer with Examples. Human Brain Mapping 15: 1-25.
49. Colder B, (2011) Emulation as an Integrating Principle for Cognition. Frontiers in Human Neuroscience Available at: http://www.frontiersin.org/human_neuroscience/10.3389/fnhum.2011.00054/full. Accessed: 12/22/11.
50. Summerfield C, Trittschuh EH, Monti JM, Mesulam MM, Egner T, (2008) Neural repetition suppression reflects fulfilled perceptual expectations. Nat Neurosci 1 (9): 1004-1006.
51. Summerfield C, Egner T, Greene M, Koechlin E, Mangels J, et al. (2006) Predictive codes for forthcoming perception in the frontal cortex. Science 314 (5803): 1311-1314.
52. Turk-Browne NB, Scholl BJ, Johnson MK, Chun MM, (2010) Implicit Perceptual Anticipation Triggered by Statistical Learning. J Neurosci 30 (33): 11177-11187.
53. Rugg MD, Johnson JD, Park H, Uncapher MR, (2008) Encoding-retrieval overlap in human episodic memory: A functional neuroimaging perspective. Progress in Brain Research 169: 339-352.
54. Tubridy S, Davachi L, (2011) Medial Temporal Lobe Contributions to Episodic Sequence Encoding. Cereb Cortex 21 (2): 272-280.
55. Davachi L, (2006) Item, context and relational episodic encoding in humans. Current Opinion in Neurobiology 16 (6): 693-700.
56. Eichenbaum H, (2000) A cortical-hippocampal system for declarative memory. Nature Rev Neuroscience 1 (1): 41-50.
57. Devan BD, White NM, (1999) Parallel information processing in the dorsal striatum: relation to hippocampal function J Neurosci 19 (7): 2789-2798.
58. Rosenbaum RS, Ziegler M, Winocur G, Grady CL, Moscovitch M, (2004) I have often walked down this street before: fMRI studies on the hippocampus and other structures during mental navigation of an old environment. Hippocampus 14 (7): 826-35.
59. Dusek JA, Eichenbaum H, (1997) The hippocampus and memory for orderly stimulus relations. Proc Natl Acad Sci USA 94: 7109-7114.
60. Chun MM, Phelps EA, (1999) Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage. Nat Neurosci 2: 844-847.
61. Frank MJ, Rudy JW, O'Reilly RC, (2003) Transitivity, Flexibility, Conjunctive Representations and the Hippocampus: II. A Computational Analysis. Hippocampus 13: 341-354.
62. Norman KA, O'Reilly RC, (2003) Modeling Hippocamapl and Neocortical Contributions to Recognition Memory: A Complementary-Learning-Systems Approach. Psychological Review 110 (4): 611-646.
63. Lisman J, Redish AD, (2009) Prediction, sequences and the hippocampus. Phil Trans R Soc B 364: 1193-1201.
64. Gluck MA, Myers CE, (1993) Hippocampal Mediation of Stimulus Representation: A Computational Theory. Hippocampus 3 (4): 491-516.
65. Harrison LM, Duggins A, Friston KJ, (2006) Encoding uncertainty in the hippocampus. Neural Networks 19: 535-546.
66. Davis T, Love BC, Preston AR, (in press) Striatal and Hippocampal Entropy and Recognition Signals in Category Learning: Simultaneous Processes Revealed by Model-based fMRI. J Exp Psychol: Learn, Mem, and Cogn.
67. Hikosaka O, Bromberg-Martin E, Hong S, Matsumoto M, (2008) New insights on the subcortical representation of reward. Current opinion in neurobiology 18 (2): 203-208.
68. Friston KJ, Danizeau J, Kiebel SJ, (2009) Reinforcement Learning or Active Inference? PLoS ONE 4 (7): e 6421 Available: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0006421. Accessed: 12/22/11.