Surprise beyond prediction error

Human Brain Mapping

Volumn 35, Issue 9, 2014, Pages 4805-4814

  Chumbley, Justin ^a   Burke, Christopher ^a   Stephan, Klaas ^b   Friston, Karl ^c   Tobler, Philippe ^a   Fehr, Ernst ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b ETH ZURICH   (Switzerland)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Learning; Prediction error; Reward

Indexed keywords

ADULT; AMYGDALOID NUCLEUS; ARTICLE; ASSOCIATION; CORPUS STRIATUM; FRONTAL CORTEX; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HUMAN; IMAGE ANALYSIS; LEARNING; MALE; NORMAL HUMAN; PREDICTION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; REWARD PREDICTION ERROR; SENSORY CORTEX; STATE PREDICTION ERROR; ADAPTIVE BEHAVIOR; ANTICIPATION; BIOLOGICAL MODEL; BRAIN; BRAIN MAPPING; NEUROPSYCHOLOGICAL TEST; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; SELF REPORT; YOUNG ADULT;

ADULT; ANTICIPATION, PSYCHOLOGICAL; BRAIN; BRAIN MAPPING; FEEDBACK, PSYCHOLOGICAL; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MODELS, NEUROLOGICAL; NEUROPSYCHOLOGICAL TESTS; REINFORCEMENT (PSYCHOLOGY); SELF REPORT; YOUNG ADULT;

EID: 84904510838     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.22513     Document Type: Article

References (47)

1. Alink A, Schwiedrzik CM, Kohler A, Singer W, Muckli L (2010): Stimulus predictability reduces responses in primary visual cortex. J Neurosci 30:2960-2966.
2. Balleine BW (2005): Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav 86:717-730.
3. Balleine BW, Daw ND, O'Doherty JP (2008): Multiple forms of value learning and the function of dopamine. Neuroeconomics: decision making and the brain 367-385.
4. Balleine B, Daw N, O'Doherty J, Balleine B (2009): Multiple forms of value learning and the function of dopamine. Adv Virus Res 367.
5. Boly M, Garrido MI, Gosseries O, Bruno MA, Boveroux P, Schnakers C, Massimini M, Litvak V, Laureys S, Friston K (2011): Preserved feedforward but impaired top-down processes in the vegetative state. Science 332:858.
6. Bornstein AM, Nylen EL, Steele SA (2011): Unblocking the neural substrates of model-based value. J Neurosci 31:10117-10118.
7. Burke KA, Franz TM, Miller DN, Schoenbaum G (2008): The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 454:340-344.
8. Corlett P, Murray G, Honey G, Aitken M, Shanks D, Robbins T, Bullmore E, Dickinson A, Fletcher P (2007): Disrupted prediction-error signal in psychosis: Evidence for an associative account of delusions. Brain 130:2387-2400.
9. Corlett P, Taylor J, Wang X, Fletcher P, Krystal J (2010): Toward a neurobiology of delusions. Prog Neurobiol 345-369.
10. d'Acremont M, Schultz W, Bossaerts P (2013): The human brain encodes event frequencies while forming subjective beliefs. J Neurosci 33:10887-10897.
11. Dayan P, Balleine BW (2002): Reward, motivation, and reinforcement learning. Neuron 36:285-298.
12. Dayan P, Niv Y (2008): Reinforcement learning: The good, the bad and the ugly. Curr Opin Neurobiol 18:185-196.
13. Dayan P, Kakade S, Montague PR (2000): Learning and selective attention. Nat Neurosci 3:1218-1223.
14. den Ouden HEM, Friston KJ, Daw ND, McIntosh AR, Stephan KE (2009): A dual role for prediction error in associative learning. Cereb Cortex 19:1175-1185.
15. den Ouden HEM, Daunizeau J, Roiser J, Friston KJ, Stephan KE (2010): Striatal prediction error modulates cortical coupling. J Neurosci 30:3210.
16. Fiser J, Aslin RN (2001): Unsupervised statistical learning of higher-order spatial structures from visual scenes. Psychol Sci 12:499-504.
17. Fletcher P, Anderson J, Shanks D, Honey R, Carpenter T, Donovan T, Papadakis N, Bullmore E (2001): Responses of human frontal cortex to surprising events are predicted by formal associative learning theory. Nat Neurosci 4:1043-1048.
18. Friston K (2009): The free-energy principle: A rough guide to the brain? Trends Cogn Sci 13:293-301.
19. Friston K, Kilner J, Harrison L (2006): A free energy principle for the brain. J Physiol Paris 100:70-87.
20. Genovesio A, Wise SP, Passingham RE (2013): Prefrontal-parietal function: From foraging to foresight. Trend Cogn Sci 2014;18:72-81.
21. Gläscher J, Daw N, Dayan P, O'Doherty J (2010): States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66:585-595.
22. Glimcher PW (2011): Understanding dopamine and reinforcement learning: The dopamine RPE hypothesis. Proc Natl Acad Sci 108(Suppl 3):15647-15654.
23. Griffiths TL, Canini KR, Sanborn AN, Navarro DJ (2007): Unifying rational models of categorization via the hierarchical Dirichlet process. Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society 323-328.
24. Kok P, Jehee JF, de Lange FP (2012): Less is more: Expectation sharpens representations in the primary visual cortex. Neuron 75:265-270.
25. Ljungberg T, Apicella P, Schultz W (1992): Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67:145-163.
26. Ludvig EA, Sutton RS, Kehoe EJ (2012): Evaluating the TD model of classical conditioning. Learn Behav 40:305-319.
27. MacKay D (2003): Information Theory, Inference, and Learning Algorithms. Cambridge University Press.
28. McClure SM, Berns GS, Montague PR (2003): Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346.
29. McDannald MA, Takahashi YK, Lopatina N, Pietras BW, Jones JL, Schoenbaum G (2012): Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. Eur J Neurosci 35:991-996.
30. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003): Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337.
31. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004): Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454.
32. Pearce JM, Hall G (1980): A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87:532.
33. Preuschoff K, Bossaerts P (2007): Adding prediction risk to the theory of reward learning. Ann N Y Acad Sci 1104:135-146.
34. Rescorla R, Wagner A (1972): Variations in the Effectiveness of Reinforcement and Nonreinforcement. New York: Classical Conditioning II: Current Research and Theory, Appleton-Century-Crofts:113-123.
35. Rescorla RA (1999): Learning about qualitatively different outcomes during a blocking procedure. Anim Learn Behav 27:140-151.
36. Roesch MR, Esber GR, Li J, Daw ND, Schoenbaum G (2012): Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain. Eur J Neurosci 35:1190-1200.
37. Schultz W, Dickinson A (2000): Neuronal coding of prediction errors. Ann Rev Neurosci 23:473-500.
38. Schultz W, Dayan P, Montague PR (1997): A neural substrate of prediction and reward. Science 275:1593-1599.
39. Schultz W, Preuschoff K, Camerer C, Hsu M, Fiorillo C, Tobler P, Bossaerts P (2008): Explicit neural signals reflecting reward uncertainty. Phil Trans R Soc Lond, Ser B: Biol Sci 363:3801.
40. Summerfield C, Koechlin E (2008): A neural representation of prior information during perceptual inference. Neuron 59:336-347.
41. Summerfield C, Monti JMP, Trittschuh EH, Mesulam MM, Egner T (2008): Neural repetition suppression reflects fulfilled perceptual expectations. Nat Neurosci 11:1004.
42. Sutton R, Barto A (1990): Time-derivative models of Pavlovian reinforcement. Learning and computational neuroscience: Foundations of adaptive networks:497-537.
43. Sutton R, Barto A (1998): Reinforcement learning: An introduction. Cambridge: The MIT press.
44. Tobler PN, Fletcher PC, Bullmore ET, Schultz W (2007): Learning-related human brain activations reflecting individual finances. Neuron 54:167-175.
45. Turk-Browne NB, Jungé JA, Scholl BJ (2005): The automaticity of visual statistical learning. J Exp Psychol Gen 134:552-563.
46. Turner DC, Aitken MR, Shanks DR, Sahakian BJ, Robbins TW, Schwarzbauer C, Fletcher PC (2004): The role of the lateral frontal cortex in causal associative learning: Exploring preventative and super-learning. Cereb Cortex 14:872-880.
47. Tversky A, Kahneman D (1992): Advances in prospect theory: Cumulative representation of uncertainty. J Risk Uncertainty 5:297-323.