Learning from feedback: The neural mechanisms of feedback processing facilitating better performance

Behavioural Brain Research

Volumn 261, Issue , 2014, Pages 356-368

  Luft, Caroline Di Bernardi ^a  

^a UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Connectivity; ERP; Feedback; FRN; Learning; Oscillations

Indexed keywords

ANALYTIC METHOD; BETA RHYTHM; BIOLOGICAL MONITORING; BRAIN ELECTROPHYSIOLOGY; BRAIN FUNCTION; CONCEPTUAL FRAMEWORK; CORTICAL SYNCHRONIZATION; ELECTROENCEPHALOGRAPHY; EVENT RELATED POTENTIAL; FACILITATION; FRONTAL LOBE; FUNCTIONAL ASSESSMENT; GAMMA RHYTHM; HUMAN; LEARNING; MENTAL CAPACITY; MENTAL PERFORMANCE; NEGATIVE FEEDBACK; NEUROFEEDBACK; NEUROPHYSIOLOGY; NONHUMAN; OSCILLATION; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; SKILL; TASK PERFORMANCE; THETA RHYTHM;

CONNECTIVITY; ERP; FEEDBACK; FRN; LEARNING; OSCILLATIONS;

BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; FEEDBACK, PSYCHOLOGICAL; HUMANS; LEARNING; THETA RHYTHM;

EID: 84892527506     PISSN: 01664328     EISSN: 18727549     Source Type: Journal    
DOI: 10.1016/j.bbr.2013.12.043     Document Type: Review

References (89)

1. Masters R.S., Maxwell J.P., Eves F.F. Marginally perceptible outcome feedback, motor learning and implicit processes. Conscious Cogn 2009, 18(3):639-645.
2. Burkhard D.G., Patterson J., Rapue R. Effect of film feedback on learning the motor skills of karate. Percept Mot Skills 1967, 25(1):65-69.
3. Kawashima R., Tajima N., Yoshida H., Okita K., Sasaki T., Schormann T., Ogawa A., Fukuda H., Zilles K. The effect of verbal feedback on motor learning-a PET study. Positron emission tomography. Neuroimage 2000, 12(6):698-706.
4. Wu W.F., Young D.E., Schandler S.L., Meir G., Judy R.L., Perez J., Cohen M.J. Contextual interference and augmented feedback: is there an additive effect for motor learning?. Hum Mov Sci 2011, 30(6):1092-1101.
5. Wulf G., Shea C.H., Matschiner S. Frequent feedback enhances complex motor skill learning. J Mot Behav 1998, 30(2):180-192.
6. Nieuwenhuis S., Holroyd C.B., Mol N., Coles M.G. Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci Biobehav Rev 2004, 28(4):441-448.
7. Cohen M.X., Wilmes K., Vijver I. Cortical electrophysiological network dynamics of feedback learning. Trends Cogn Sci 2011, 15(12):558-566.
8. Cohen M.X. Neurocomputational mechanisms of reinforcement-guided learning in humans: a review. Cogn Affect Behav Neurosci 2008, 8(2):113-125.
9. Walsh M.M., Anderson J.R. Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice. Neurosci Biobehav Rev 2012, 36(8):1870-1884.
10. Gehring W.J., Goss B., Coles M.G.H., Meyer D.E., Donchin E. A neural system for error detection and compensation. Psych Sci 1993, 4(6):385-390.
11. Falkenstein M., Hohnsbein J., Hoormann J., Blanke L. Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr Clin Neurophysiol 1991, 78(6):447-455.
12. Gehring W.J., Coles M.G.H., Meyer D.E., Donchin E. The error-related negativity: an event-related brain potential accompanying errors. Psychophysiology 1990, 27(S34.).
13. Miltner W.H.R., Braun C., Coles M.G.H. Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a generic neural system for error detection. J Cogn Neurosci 1997, 9(6):788-798.
14. Balconi M., Scioli S. Error monitoring functions in response to an external feedback when an explicit judgement is required ERP modulation and cortical source localisation. Int J Psychophysiol 2011, 83(3):295-301.
15. Luu P., Tucker D.M., Derryberry D., Reed M., Poulsen C. Electrophysiological responses to errors and feedback in the process of action regulation. Psychol Sci 2003, 14(1):47-53.
16. Potts G.F., Martin L.E., Kamp S.M., Donchin E. Neural response to action and reward prediction errors: comparing the error-related negativity to behavioral errors and the feedback-related negativity to reward prediction violations. Psychophysiology 2010, 48:218-228.
17. Santesso D.L., Bogdan R., Birk J.L., Goetz E.L., Holmes A.J., Pizzagalli D.A. Neural responses to negative feedback are related to negative emotionality in healthy adults. Soc Cogn Affect Neurosci 2011, Epub ahead of print.
18. Ladouceur C.D., Dahl R.E., Carter C.S. Development of action monitoring through adolescence into adulthood: ERP and source localization. Dev Sci 2007, 10(6):874-891.
19. Van Veen V., Carter C.S. The timing of action-monitoring processes in the anterior cingulate cortex. J Cogn Neurosci 2002, 14(4):593-602.
20. Carter C.S., Braver T.S., Barch D.M., Botvinick M.M., Noll D., Cohen J.D. Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 1998, 280(5364):747-749.
21. Zhou Z., Yu R., Zhou X. To do or not to do? Action enlarges the FRN and P300 effects in outcome evaluation. Neuropsychologia 2010, 48(12):3606-3613.
22. Hauser T.U., Iannaccone R., Stampfli P., Drechsler R., Brandeis D., Walitza S., Brem S. The feedback-related negativity (FRN) revisited: new insights into the localization, meaning, and network organization. Neuroimage 2013, 84C:159-168.
23. Herrmann C.S., Grigutsch M., Busch N.A. EEG oscillations and wavelet analysis. Event-related potentials: a methods handbook 2005, The MIT Press, Cambridge. T.C. Handy (Ed.).
24. Cohen M.X., Elger C.E., Ranganath C. Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage 2007, 35(2):968-978.
25. Christie G.J., Tata M.S. Right frontal cortex generates reward-related theta-band oscillatory activity. Neuroimage 2009, 48(2):415-422.
26. Cavanagh J.F., Frank M.J., Klein T.J., Allen J.J. Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. Neuroimage 2010, 49(4):3198-3209.
27. Papo D., Douiri A., Bouchet F., Bourzeix J.C., Caverni J.P., Baudonniere P.M. Modulation of late alpha band oscillations by feedback in a hypothesis testing paradigm. Int J Psychophysiol 2007, 63(1):110-116.
28. Marco-Pallares J., Cucurell D., Cunillera T., Garcia R., Andres-Pueyo A., Munte T.F., Rodriguez-Fornells A. Human oscillatory activity associated to reward processing in a gambling task. Neuropsychologia 2008, 46(1):241-248.
29. Luft C.D.B., Nolte G., Bhattacharya J. High-learners present larger mid-frontal theta power and connectivity in response to incorrect performance feedback. J Neurosci 2013, 33(5):2029-2038.
30. van de Vijver I., Ridderinkhof K.R., Cohen M.X. Frontal oscillatory dynamics predict feedback learning and action adjustment. J Cogn Neurosci 2011, 23(12):4106-4121.
31. van der Helden J., Boksem M.A., Blom J.H. The importance of failure: feedback-related negativity predicts motor learning efficiency. Cereb Cortex 2010, 20(7):1596-1603.
32. Bellebaum C., Daum I. Learning-related changes in reward expectancy are reflected in the feedback-related negativity. Eur J Neurosci 2008, 27(7):1823-1835.
33. Heldmann M., Russeler J., Munte T.F. Internal and external information in error processing. BMC Neurosci 2008, 9:33.
34. Cavanagh J.F., Zambrano-Vazquez L., Allen J.J. Theta lingua franca: a common mid-frontal substrate for action monitoring processes. Psychophysiology 2012, 49(2):220-238.
35. van de Vijver I., Cohen M.X., Ridderinkhof K.R. Aging affects medial but not anterior frontal learning-related theta oscillations. Neurobiol Aging 2014, 35(3):692-704.
36. Luft C.D.B., Takase E., Bhattacharya J. Processing graded feedback: electrophysiological correlates of learning from small and large errors. J Cogn Neurosci 2014, in press.
37. Wolpert D.M., Diedrichsen J., Flanagan J.R. Principles of sensorimotor learning. Nat Rev Neurosci 2011, 12(12):739-751.
38. Sutton R.S., Barto A.G. Reinforcement learning: an introduction 1998, The MIT Press, Cambridge, Massachusets.
39. Ashby F.G., Alfonso-Reese L.A., Turken A.U., Waldron E.M. A neuropsychological theory of multiple systems in category learning. Psychol Rev 1998, 105(3):442-481.
40. Cohen M.X., Ranganath C. Reinforcement learning signals predict future decisions. J Neurosci 2007, 27(2):371-378.
41. Sailer U., Fischmeister F.P., Bauer H. Effects of learning on feedback-related brain potentials in a decision-making task. Brain Res 2010, 1342:85-93.
42. Chase H.W., Swainson R., Durham L., Benham L., Cools R. Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. J Cogn Neurosci 2011, 23(4):936-946.
43. Oliveira F.T., McDonald J.J., Goodman D. Performance monitoring in the anterior cingulate is not all error related: expectancy deviation and the representation of action-outcome associations. J Cogn Neurosci 2007, 19(12):1994-2004.
44. Van Boxtel G.J.M. The use of the subtraction technique in the psychophysiology of response inhibition and conflict. Errors, conflicts, and the brain current opinions on performance monitoring 2004, 219-225. Max Planck Institute of Cognitive Neuroscience. M. Ullsperger, M. Falkenstein, Leipzig (Eds.).
45. Holroyd C.B., Coles M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 2002, 109(4):679-709.
46. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 1998, 80(1):1-27.
47. Schultz W. Getting formal with dopamine and reward. Neuron 2002, 36(2):241-263.
48. Jocham G., Ullsperger M. Neuropharmacology of performance monitoring. Neurosci Biobehav Rev 2009, 33(1):48-60.
49. Walton M.E., Devlin J.T., Rushworth M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci 2004, 7(11):1259-1265.
50. Joel D., Weiner I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 2000, 96(3):451-474.
51. Eblen F., Graybiel A.M. Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J Neurosci 1995, 15(9):5999-6013.
52. Yasuda A., Sato A., Miyawaki K., Kumano H., Kuboki T. Error-related negativity reflects detection of negative reward prediction error. Neuroreport 2004, 15(16):2561-2565.
53. Bellebaum C., Polezzi D., Daum I. It is less than you expected: the feedback-related negativity reflects violations of reward magnitude expectations. Neuropsychologia 2010, 48(11):3343-3350.
54. Holroyd C.B., Krigolson O.E., Baker R., Lee S., Gibson J. When is an error not a prediction error? An electrophysiological investigation. Cogn Affect Behav Neurosci 2009, 9(1):59-70.
55. Bellebaum C., Kobza S., Thiele S., Daum I. Processing of expected and unexpected monetary performance outcomes in healthy older subjects. Behav Neurosci 2011, 125(2):241-251.
56. Pfabigan D.M., Alexopoulos J., Bauer H., Sailer U. Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. Psychophysiology 2011, 48(5):656-664.
57. Holroyd C.B., Nieuwenhuis S., Yeung N., Cohen J.D. Errors in reward prediction are reflected in the event-related brain potential. Neuroreport 2003, 14(18):2481-2484.
58. Hajcak G., Moser J.S., Holroyd C.B., Simons R.F. It's worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology 2007, 44(6):905-912.
59. Gentsch A., Ullsperger P., Ullsperger M. Dissociable medial frontal negativities from a common monitoring system for self- and externally caused failure of goal achievement. Neuroimage 2009, 47(4):2023-2030.
60. Muller S.V., Moller J., Rodriguez-Fornells A., Munte T.F. Brain potentials related to self-generated and external information used for performance monitoring. Clin Neurophysiol 2005, 116(1):63-74.
61. Krigolson O.E., Pierce L.J., Holroyd C.B., Tanaka J.W. Learning to become an expert: reinforcement learning and the acquisition of perceptual expertise. J Cogn Neurosci 2009, 21(9):1834-1841.
62. Philiastides M.G., Biele G., Vavatzanidis N., Kazzer P., Heekeren H.R. Temporal dynamics of prediction error processing during reward-based decision making. Neuroimage 2010, 53(1):221-232.
63. Frank M.J., Woroch B.S., Curran T. Error-related negativity predicts reinforcement learning and conflict biases. Neuron 2005, 47(4):495-501.
64. Arbel Y., Goforth K., Donchin E. The good, the bad, or the useful? The examination of the relationship between the feedback-related negativity (FRN) and long-term learning outcomes. J Cogn Neurosci 2013, 25(8):1249-1260.
65. Cavanagh J.F., Figueroa C.M., Cohen M.X., Frank M.J. Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cereb Cortex 2011, Epub ahead of print.
66. Cohen M.X., Elger C.E., Fell J. Oscillatory activity and phase-amplitude coupling in the human medial frontal cortex during decision making. J Cogn Neurosci 2009, 21(2):390-402.
67. HajiHosseini A., Rodriguez-Fornells A., Marco-Pallares J. The role of beta-gamma oscillations in unexpected rewards processing. Neuroimage 2012, 60(3):1678-1685.
68. Marco-Pallares J., Cucurell D., Cunillera T., Kramer U.M., Camara E., Nager W., Bauer P., Schule R., Schols L., Munte T.F. et al: Genetic variability in the dopamine system (dopamine receptor D4, catechol-O-methyltransferase) modulates neurophysiological responses to gains and losses. Biol Psychiatry 2009, 66(2):154-161.
69. De Pascalis V., Varriale V., Rotonda M. EEG oscillatory activity associated to monetary gain and loss signals in a learning task: effects of attentional impulsivity and learning ability. Int J Psychophysiol 2012, 85(1):68-78.
70. Cunillera T., Fuentemilla L., Perianez J., Marco-Pallares J., Kramer U.M., Camara E., Munte T.F., Rodriguez-Fornells A. Brain oscillatory activity associated with task switching and feedback processing. Cogn Affect Behav Neurosci 2012, 12(1):16-33.
71. Rushworth M.F., Behrens T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 2008, 11(4):389-397.
72. Cavanagh J.F., Eisenberg I., Guitart-Masip M., Huys Q., Frank M.J. Frontal theta overrides pavlovian learning biases. J Neurosci 2013, 33(19):8541-8548.
73. Hallschmid M., Mölle M., Fischer S., Born J. EEG synchronization upon reward in man. Clin Neurophysiol 2002, 113(7):1059-1065.
74. Courtemanche R., Fujii N., Graybiel A.M. Synchronous, focally modulated beta-band oscillations characterize local field potential activity in the striatum of awake behaving monkeys. J Neurosci 2003, 23(37):11741-11752.
75. Berke J.D. Fast oscillations in cortical-striatal networks switch frequency following rewarding events and stimulant drugs. Eur J Neurosci 2009, 30(5):848-859.
76. Engel A.K., Fries P. Beta-band oscillations-signalling the status quo?. Curr Opin Neurobiol 2010, 20(2):156-165.
77. Galea J.M., Vazquez A., Pasricha N., de Xivry J.J., Celnik P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb Cortex 2011, 21(8):1761-1770.
78. Cavanagh J.F., Cohen M.X., Allen J.J. Prelude to and resolution of an error: EEG phase synchrony reveals cognitive control dynamics during action monitoring. J Neurosci 2009, 29(1):98-105.
79. Rahnev D., Lau H., de Lange F.P. Prior expectation modulates the interaction between sensory and prefrontal regions in the human brain. J Neurosci 2011, 31(29):10741-10748.
80. Cohen M.X. Error-related medial frontal theta activity predicts cingulate-related structural connectivity. Neuroimage 2011, 55(3):1373-1383.
81. Sagi Y., Tavor I., Hofstetter S., Tzur-Moryosef S., Blumenfeld-Katzir T., Assaf Y. Learning in the fast lane: new insights into neuroplasticity. Neuron 2012, 73(6):1195-1203.
82. Paus T. Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat Rev Neurosci 2001, 2(6):417-424.
83. Rushworth M.F.S., Behrens T.E.J., Rudebeck P.H., Walton M.E. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci 2007, 11(4):168-176.
84. Cohen M.X., van Gaal S. Dynamic interactions between large-scale brain networks predict behavioral adaptation after perceptual errors. Cereb Cortex 2013, 23(5):1061-1072.
85. Matsumoto M., Matsumoto K., Abe H., Tanaka K. Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 2007, 10(5):647-656.
86. Kennerley S.W., Walton M.E., Behrens T.E., Buckley M.J., Rushworth M.F. Optimal decision making and the anterior cingulate cortex. Nat Neurosci 2006, 9(7):940-947.
87. Wang C., Ulbert I., Schomer D.L., Marinkovic K., Halgren E. Responses of human anterior cingulate cortex microdomains to error detection, conflict monitoring, stimulus-response mapping, familiarity, and orienting. J Neurosci 2005, 25(3):604-613.
88. Leung L.W., Borst J.G. Electrical activity of the cingulate cortex. I. Generating mechanisms and relations to behavior. Brain Res 1987, 407(1):68-80.
89. Lam J.M., Wachter T., Globas C., Karnath H.O., Luft A.R. Predictive value and reward in implicit classification learning. Hum Brain Mapp 2013, 34(1):176-185.