Frontal theta oscillatory activity is a common mechanism for the computation of unexpected outcomes and learning rate

Journal of Cognitive Neuroscience

Volumn 26, Issue 3, 2014, Pages 447-458

  Mas Herrero, Ernest ^a   Marco Pallarés, Josep ^a,b  

^a HOSPITAL UNIVERSITARI DE BELLVITGE   (Spain)

^b UNIVERSITY OF BARCELONA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BRAIN ELECTROPHYSIOLOGY; FRONTAL CORTEX; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; OSCILLATION; OUTCOME ASSESSMENT; PREFRONTAL CORTEX; PRIORITY JOURNAL; REWARD; THETA RHYTHM;

ASSOCIATION LEARNING; BRAIN; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FEEDBACK, PSYCHOLOGICAL; FEMALE; HUMANS; LINEAR MODELS; MALE; NEUROPSYCHOLOGICAL TESTS; PROBABILITY LEARNING; REWARD; TASK PERFORMANCE AND ANALYSIS; THETA RHYTHM; UNCERTAINTY; YOUNG ADULT;

EID: 84893389472     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn_a_00516     Document Type: Article

References (46)

1. Alexander, W. H., & Brown, J. W. (2011). Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14, 1338-1344.
2. Aston-Jones, G., & Cohen, J. D. (2005). An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance. Annual Review of Neuroscience, 28, 403-450.
3. Behrens, T. E. J., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214-1221.
4. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., & Cohen, J. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108, 624-652.
5. Brázdil, M., Babiloni, C., Roman, R., Daniel, P., Bares, M., Rektor, I., et al. (2009). Directional functional coupling of cerebral rhythms between anterior cingulate and dorsolateral prefrontal areas during rare stimuli: A directed transfer function analysis of human depth EEG signal. Human Brain Mapping, 30, 138-146.
6. Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V., & Roesch, M. R. (2011). Attention for learning signals in anterior cingulate cortex. The Journal of Neuroscience, 31, 18266-18274.
7. Buzsáki, G., & Draguhn, A. (2004). Neuronal oscillations in cortical networks. Science, 304, 1926-1929.
8. Cavanagh, J. F., Figueroa, C. M., Cohen, M. X., & Frank, M. J. (2011). Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation. Cerebral Cortex, 22, 2575-2586.
9. Cavanagh, J. F., Frank, M. J., Klein, T. J., & Allen, J. J. B. (2010). Frontal theta links prediction errors to behavioral adaptation in reinforcement learning. Neuroimage, 49, 3198-3209.
10. Cavanagh, J. F., Zambrano-Vazquez, L., & Allen, J. J. B. (2012). Theta lingua franca: A common mid-frontal substrate for action monitoring processes. Psychophysiology, 49, 220-238.
11. Chase, H. W., Swainson, R., Durham, L., Benham, L., & Cools, R. (2011). Feedback-related negativity codes prediction error but not behavioral adjustment during probabilistic reversal learning. Journal of Cognitive Neuroscience, 23, 936-946.
12. Cohen, M. X., Elger, C. E., & Ranganath, C. (2007). Reward expectation modulates feedback-related negativity and EEG spectra. Neuroimage, 35, 968-978.
13. Cohen, M. X., & Ranganath, C. (2007). Reinforcement learning signals predict future decisions. The Journal of Neuroscience, 27, 371-378.
14. Cohen, M. X., Ridderinkhof, K. R., Haupt, S., Elger, C. E., & Fell, J. (2008). Medial frontal cortex and response conflict: Evidence from human intracranial EEG and medial frontal cortex lesion. Brain Research, 1238, 127-142.
15. Cools, R., Clark, L., Owen, A. M., & Robbins, T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. The Journal of Neuroscience, 22, 4563-4567.
16. Deiber, M.-P., Missonnier, P., Bertrand, O., Gold, G., Fazio-Costa, L., Ibañez, V., et al. (2007). Distinction between perceptual and attentional processing in working memory tasks: A study of phase-locked and induced oscillatory brain dynamics. Journal of Cognitive Neuroscience, 19, 158-172.
17. Ferdinand, N. K., Mecklinger, A., Kray, J., & Gehring, W. J. (2012). The processing of unexpected positive response outcomes in the mediofrontal cortex. The Journal of Neuroscience, 32, 12087-12092.
18. Gehring, W. J., & Willoughby, A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science, 295, 2279-2282.
19. Gevins, A., Smith, M. E., McEvoy, L., & Yu, D. (1997). Highresolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice. Cerebral Cortex, 7, 374-385.
20. Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66, 585-595.
21. Hajihosseini, A., & Holroyd, C. B. (2013). Frontal midline theta and N200 amplitude reflect complementary information about expectancy and outcome evaluation. Psychophysiology, 50, 550-562.
22. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., & Platt, M. L. (2011). Surprise signals in anterior cingulate cortex: Neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. The Journal of Neuroscience, 31, 4178-4187.
23. Holroyd, C. B., & Coles, M. G. H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709.
24. Jensen, O., & Tesche, C. D. (2002). Frontal theta activity in humans increases with memory load in a working memory task. The European Journal of Neuroscience, 15, 1395-1399.
25. Jocham, G., Neumann, J., Klein, T. A., Danielmeier, C., & Ullsperger, M. (2009). Adaptive coding of action values in the human rostral cingulate zone. The Journal of Neuroscience, 29, 7489-7496.
26. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773-779.
27. Kerns, J. G. (2006). Anterior cingulate and prefrontal cortex activity in an fMRI study of trial-to-trial adjustments on the Simon task. Neuroimage, 33, 399-405.
28. Kerns, J. G., Cohen, J. D., MacDonald, A. W., Cho, R. Y., Stenger, V. A., & Carter, C. S. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science, 303, 1023-1026.
29. Krugel, L. K., Biele, G., Mohr, P. N. C., Li, S.-C., & Heekeren, H. R. (2009). Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proceedings of the National Academy of Sciences, U.S.A., 106, 17951-17956.
30. Luu, P., Tucker, D. M., Derryberry, D., Reed, M., & Poulsen, C. (2003). Electrophysiological responses to errors and feedback in the process of action regulation. Psychological Science, 14, 47-53.
31. Luu, P., Tucker, D. M., & Makeig, S. (2004). Frontal midline theta and the error-related negativity: Neurophysiological mechanisms of action regulation. Clinical Neurophysiology, 115, 1821-1835.
32. Marco-Pallares, J., Cucurell, D., Cunillera, T., García, R., Andrés-Pueyo, A., Münte, T. F., et al. (2008). Human oscillatory activity associated to reward processing in a gambling task. Neuropsychologia, 46, 241-248.
33. Oliveira, F. T. P., McDonald, J. J., & Goodman, D. (2007). Performance monitoring in the anterior cingulate is not all error related: Expectancy deviation and the representation of action-outcome associations. Journal of Cognitive Neuroscience, 19, 1994-2004.
34. Paus, T. (2001). Primate anterior cingulate cortex: Where motor control, drive and cognition interface. Nature Reviews Neuroscience, 2, 417-424.
35. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532-552.
36. Philiastides, M. G., Biele, G., Vavatzanidis, N., Kazzer, P., & Heekeren, H. R. (2010). Temporal dynamics of prediction error processing during reward-based decision making. Neuroimage, 53, 221-232.
37. Riba, J., Rodríguez-Fornells, A., Morte, A., Münte, T. F., & Barbanoj, M. J. (2005). Noradrenergic stimulation enhances human action monitoring. The Journal of Neuroscience, 25, 4370-4374.
38. Rushworth, M. F. S., Walton, M. E., Kennerley, S. W., & Bannerman, D. M. (2004). Action sets and decisions in the medial frontal cortex. Trends in Cognitive Sciences, 8, 410-417.
39. Schultz, W. (1997). Dopamine neurons and their role in reward mechanisms. Current Opinion in Neurobiology, 7, 191-197.
40. Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain and cognitive control in the cingulate cortex. Nature Reviews Neuroscience, 12, 154-167.
41. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
42. Walton, M. E., Croxson, P. L., Behrens, T. E. J., Kennerley, S. W., & Rushworth, M. F. S. (2007). Adaptive decision making and value in the anterior cingulate cortex. Neuroimage, 36 (Suppl. 2), T142-T154.
43. Walton, M. E., Devlin, J. T., & Rushworth, M. F. S. (2004). Interactions between decision making and performance monitoring within prefrontal cortex. Nature Neuroscience, 7, 1259-1265.
44. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279-292.
45. Wu, Y., & Zhou, X. (2009). The P300 and reward valence, magnitude, and expectancy in outcome evaluation. Brain Research, 1286, 114-122.
46. Yoshida, W., & Ishii, S. (2006). Resolution of uncertainty in prefrontal cortex. Neuron, 50, 781-789.