A general role for medial prefrontal cortex in event prediction

Frontiers in Computational Neuroscience

Volumn 8, Issue JUL, 2014, Pages

  Alexander, William H ^a,b   Brown, Joshua W ^b  

^a GHENT UNIVERSITY   (Belgium)

^b INDIANA UNIVERSITY   (United States)

Author keywords

Anterior cingulate; Attention; Cognitive control; Medial prefrontal cortex; Reinforcement learning

Indexed keywords

FORECASTING; REINFORCEMENT LEARNING;

ANTERIOR CINGULATE; ATTENTION; COGNITIVE CONTROL; COMPUTATIONAL NEURAL MODELS; MODEL-BASED REINFORCEMENT LEARNING; PREDICTION ERRORS; PREFRONTAL CORTEX; THEORETICAL FRAMEWORK;

COMPUTER SIMULATION;

ARTICLE; BAYESIAN LEARNING; BRAIN FUNCTION; COGNITION; CONCEPTUAL FRAMEWORK; ELECTROENCEPHALOGRAPHY; EVOKED RESPONSE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MEDIAL PREFRONTAL CORTEX; MISMATCH NEGATIVITY; NONHUMAN; PREDICTED RESPONSE OUTCOME MODEL; PREDICTION; PREFRONTAL CORTEX; SIMULATION;

EID: 84904216442     PISSN: None     EISSN: 16625188     Source Type: Journal    
DOI: 10.3389/fncom.2014.00069     Document Type: Article

References (68)

1. Aarts, E., and Roelofs, A. (2010). Attentional control in anterior cingulate cortex based on probabilistic cueing. J. Cogn. Neurosci. 23, 716-727. doi: 10.1162/jocn.2010.21435
2. Aarts, E., Roelofs, A., and van Turennout, M. (2008). Anticipatory activity in anterior cingulate cortex can be independent of conflict and error likelihood. J. Neurosci. 28, 4671-4678. doi: 10.1523/jneurosci.4400-07.2008
3. Alexander, W. H. (2007). Shifting attention using a temporal difference prediction error and high-dimensional input. Adapt. Behav. 15, 121-133. doi: 10.1177/1059712307078663
4. Alexander, W. H., and Brown, J. W. (2010). Computational models of performance monitoring and cognitive control. Top. Cogn. Sci. 2, 658-677. doi: 10.1111/j.1756-8765.2010.01085.x
5. Alexander, W. H., and Brown, J. W. (2011). Medial prefrontal cortex as an action-outcome predictor. Nat. Neurosci. 14, 1338-1344. doi: 10.1038/nn.2921
6. Amador, N., Schlag-Rey, M., and Schlag, J. (2000). Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84, 2166-2170.
7. Amiez, C., Joseph, J. P., and Procyk, E. (2006). Reward encoding in the monkey anterior cingulate cortex. Cereb. Cortex 16, 1040-1055. doi: 10.1093/cercor/bhj046
8. Apps, M. A. J., Balsters, J. H., and Ramnani, N. (2012). The anterior cingulate cortex: monitoring the outcomes of others' decisions. Soc. Neurosci. 7, 424-435. doi: 10.1080/17470919.2011.638799
9. Barbas, H., and Pandya, D. N. (1989). Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 286, 353-375. doi: 10.1002/cne.902860306
10. Behrens, T. E., Woolrich, M. W., Walton, M. E., and Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214-1221. doi: 10.1038/nn1954
11. Blais, C., and Bunge, S. (2010). Behavioral and neural evidence for item-specific performance monitoring. J. Cogn. Neurosci. 22, 2758-2767. doi: 10.1162/jocn.2009.21365
12. Blakemore, S.-J., Frith, C., and Wolpert, D. M. (2001). The cerebellum is involved in predicting the sensory consequences of action. [Miscellaneous Article]. Neuroreport 12, 1879-1884. doi: 10.1097/00001756-200107030-00023
13. Boorman, E. D., Rushworth, M. F., and Behrens, T. E. (2013). Ventromedial prefrontal and anterior cingulate cortex adopt choice and default reference frames during sequential multialternative choice. J. Neurosci. 33, 2242-2253. doi: 10.1523/jneurosci.3022-12.2013
14. Botvinick, M. M., Braver, T. S., Barch, D. M., Carter, C. S., and Cohen, J. C. (2001). Conflict monitoring and cognitive control. Psychol. Rev. 108, 624-652. doi: 10.1037/0033-295x.108.3.624
15. Brown, J. W. (2013). Beyond conflict monitoring cognitive control and the neural basis of thinking before you act. Curr. Dir. Psychol. Sci. 22, 179-185. doi: 10.1177/0963721412470685
16. Brown, J. W., and Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science 307, 1118-1121. doi: 10.1126/science.1105783
17. Bryden, D. W., Johnson, E. E., Tobia, S. C., Kashtelyan, V., and Roesch, M. R. (2011). Attention for learning signals in anterior cingulate cortex. J. Neurosci. 31, 18266-18274. doi: 10.1523/JNEUROSCI.4715-11.2011
18. Büchel, C., Bornhövd, K., Quante, M., Glauche, V., Bromm, B., and Weiller, C. (2002). Dissociable neural responses related to pain intensity, stimulus intensity and stimulus awareness within the anterior cingulate cortex: a parametric single-trial laser functional magnetic resonance imaging study. J. Neurosci. 22, 970-976.
19. Bugg, J. M., Jacoby, L. L., and Toth, J. P. (2008). Multiple levels of control in the Stroop task. Mem. Cognit. 36, 1484-1494. doi: 10.3758/MC.36.8.1484
20. Camille, N., Tsuchida, A., and Fellows, L. K. (2011). Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J. Neurosci. 31, 15048-15052. doi: 10.1523/JNEUROSCI.3164-11.2011
21. Cardinal, R. N., and Cheung, T. H. C. (2005). Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat. BMC Neurosci. 6:9. doi: 10.1186/1471-2202-6-9
22. Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D. C., and Cohen, J. D. (1998). Anterior cingulate cortex, error detection and the online monitoring of performance. Science 280, 747-749. doi: 10.1126/science.280.5364.747
23. Carter, C. S., Macdonald, A. M., Botvinick, M., Ross, L. L., Stenger, V. A., Noll, D., et al. (2000). Parsing executive processes: strategic vs. evaluative functions of the anterior cingulate cortex. Proc. Natl. Acad. Sci. U S A 97, 1944-1948. doi: 10.1073/pnas.97.4.1944
24. Chandrasekhar, P. V. S., Capra, C. M., Moore, S., Noussair, C., and Berns, G. S. (2008). Neurobiological regret and rejoice functions for aversive outcomes. Neuroimage 39, 1472-1484. doi: 10.1016/j.neuroimage.2007.10.027
25. Cohen, M. X., Axmacher, N., Lenartz, D., Elger, C. E., Sturm, V., and Schlaepfer, T. E. (2009). Neuroelectric signatures of reward learning and decision-making in the human nucleus accumbens. Neuropsychopharmacology34, 1649-1658. doi: 10.1038/npp.2008.222
26. Crottaz-Herbette, S., and Menon, V. (2006). Where and when the anterior cingulate cortex modulates attentional response: combined fMRI and ERP evidence. J. Cogn. Neurosci. 18, 766-780. doi: 10.1162/jocn.2006.18.5.766
27. Daw, N. D., and Doya, K. (2006). The computational neurobiology of learning and reward. Curr. Opin. Neurobiol.16, 199-204. doi: 10.1016/j.conb.2006.03.006
28. Dayan, P., and Niv, Y. (2008). Reinforcement learning: the good, the bad and the ugly. Curr. Opin. Neurobiol. 18, 185-196. doi: 10.1016/j.conb.2008.08.003
29. Dien, J., Spencer, K. M., and Donchin, E. (2003). Localization of the event-related potential novelty response as defined by principal components analysis. Brain Res. Cogn. Brain Res. 17, 637-650. doi: 10.1016/s0926-6410(03)00188-5
30. Downar, J., Crawley, A. P., Mikulis, D. J., and Davis, K. D. (2002). A cortical network sensitive to stimulus salience in a neutral behavioral context across multiple sensory modalities. J. Neurophysiol. 87, 615-620.
31. Doya, K. (2007). Reinforcement learning: computational theory and biological mechanisms. HFSP J. 1, 30-40. doi: 10.2976/1.2732246/10.2976/1
32. Eriksen, C. W. (1995). The flankers task and response competition: a useful tool for investigating a variety of cognitive problems. Vis. Cogn. 2, 101-118. doi: 10.1080/13506289508401726
33. Falkenstein, M., Hohnsbein, J., Hoorman, J., and Blanke, L. (1991). Effects of crossmodal divided attention on late ERP components: II. Error processing in choice reaction tasks. Electroencephalogr. Clin. Neurophysiol. 78, 447-455. doi: 10.1016/0013-4694(91)90062-9
34. Forster, S. E., and Brown, J. W. (2011). Medial prefrontal cortex predicts and evaluates the timing of action outcomes. Neuroimage 55, 253-265. doi: 10.1016/j.neuroimage.2010.11.035
35. Friston, K. (2010). The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127-138. doi: 10.1038/nrn2787
36. Gläscher, J., Daw, N., Dayan, P., and O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66, 585-595. doi: 10.1016/j.neuron.2010.04.016
37. Goldstein, R. Z., Tomasi, D., Rajaram, S., Cottone, L. A., Zhang, L., Maloney, T., et al. (2007). Role of the anterior cingulate and medial orbitofrontal cortex in processing drug cues in cocaine addiction. Neuroscience 144, 1153-1159. doi: 10.1016/j.neuroscience.2006.11.024
38. Grinband, J., Savitskaya, J., Wager, T. D., Teichert, T., Ferrera, V. P., and Hirsch, J. (2011). The dorsal medial frontal cortex is sensitive to time on task, not response conflict or error likelihood. Neuroimage 57, 303-311. doi: 10.1016/j.neuroimage.2010.12.027
39. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M., and Platt, M. L. (2011a). Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178-4187. doi: 10.1523/JNEUROSCI.4652-10.2011
40. Hayden, B. Y., Pearson, J. M., and Platt, M. L. (2011b). Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933-939. doi: 10.1038/nn.2856
41. Holroyd, C. B., and Coles, M. G. (2002). The neural basis of human error processing: reinforcement learning, dopamine and the error-related negativity. Psychol. Rev. 109, 679-709. doi: 10.1037//0033-295x.109.4.679
42. Ide, J. S., Shenoy, P., Yu, A. J., and Li, C. R. (2013). Bayesian prediction and evaluation in the anterior cingulate cortex. J. Neurosci. 33, 2039-2047. doi: 10.1523/JNEUROSCI.2201-12.2013
43. Jahn, A., Nee, D. E., Alexander, W. H., and Brown, J. W. (2014). Distinct regions of anterior cingulate cortex signal prediction and outcome evaluation. Neuroimage 95, 80-89. doi: 10.1016/j.neuroimage.2014.03.050
44. Jessup, R. K., Busemeyer, J. R., and Brown, J. W. (2010). Error effects in anterior cingulate cortex reverse when error likelihood is high. J. Neurosci. 30, 3467-3472. doi: 10.1523/JNEUROSCI.4130-09.2010
45. Kennerley, S. W., Behrens, T. E. J., and Wallis, J. D. (2011). Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589. doi: 10.1038/nn.2961
46. Kolling, N., Behrens, T. E. J., Mars, R. B., and Rushworth, M. F. S. (2012). Neural mechanisms of foraging. Science336, 95-98. doi: 10.1126/science.1216930
47. Koyama, T., Kato, K., Tanaka, Y. Z., and Mikami, A. (2001). Anterior cingulate activity during pain-avoidance and reward tasks in monkeys. Neurosci. Res. 39, 421-430. doi: 10.1016/s0168-0102(01)00197-3
48. Kruschke, J. K. (2001). Toward a unified model of attention in associative learning. J. Math. Psychol. 45, 812-863. doi: 10.1006/jmps.2000.1354
49. Litt, A., Plassmann, H., Shiv, B., and Rangel, A. (2011). Dissociating valuation and saliency signals during decision-making. Cereb. Cortex 21, 95-102. doi: 10.1093/cercor/bhq065
50. Mackintosh, N. (1975). A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 82, 276-298. doi: 10.1037/h0076778
51. Matsumoto, M., Matsumoto, K., Abe, H., and Tanaka, K. (2007). Medial prefrontal cell activity signaling prediction errors of action values. Nat. Neurosci. 10, 647-656. doi: 10.1038/nn1890
52. Mian, M. K., Sheth, S. A., Patel, S. R., Spiliopoulos, K., Eskandar, E. N., and Williams, Z. M. (2014). Encoding of rules by neurons in the human dorsolateral prefrontal cortex. Cereb. Cortex 24, 807-816. doi: 10.1093/cercor/bhs361
53. Nee, D. E., and Brown, J. W. (2013). Dissociable frontal-striatal and frontal-parietal networks involved in updating hierarchical contexts in working memory. Cereb. Cortex 23, 2146-2158. doi: 10.1093/cercor/bhs194
54. Pearce, J. M., and Hall, G. (1980). A model for pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532-552. doi: 10.1037/0033-295x.87.6.532
55. Rudebeck, P. H., Behrens, T. E., Kennerley, S. W., Baxter, M. G., Buckley, M. J., Walton, M. E., et al. (2008). Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28, 13775-13785. doi: 10.1523/jneurosci.3541-08.2008
56. Sallet, J., Quilodran, R., Rothé, M., Vezoli, J., Joseph, J.-P., and Procyk, E. (2007). Expectations, gains, and losses in the anterior cingulate cortex. Cogn. Affect. Behav. Neurosci. 7, 327-336. doi: 10.3758/cabn.7.4.327
57. Shima, K., and Tanji, J. (1998). Role of cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335-1338. doi: 10.1126/science.282.5392.1335
58. Simon, D. A., and Daw, N. D. (2011). Neural correlates of forward planning in a spatial decision task in humans. J. Neurosci. 31, 5526-5539. doi: 10.1523/JNEUROSCI.4647-10.2011
59. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. J. Exp. Psychol. 18, 643-662. doi: 10.1037/h0054651
60. Sutton, R. S. (1988). Learning to predict by the methods of temporal difference. Mach. Learn. 3, 9-44. doi: 10.1007/bf00115009
61. Talmi, D., Atkinson, R., and El-Deredy, W. (2013). The feedback-related negativity signals salience prediction errors, not reward prediction errors. J. Neurosci. 33, 8264-8269. doi: 10.1523/jneurosci.5695-12.2013
62. Vachon, F., Hughes, R. W., and Jones, D. M. (2012). Broken expectations: violation of expectancies, not novelty, captures auditory attention. J. Exp. Psychol. Learn. Mem. Cogn. 38, 164-177. doi: 10.1037/a0025054
63. van der Meer, M. A., and Redish, A. D. (2010). Expectancies in decision making, reinforcement learning and ventral striatum. Front. Neurosci. 4:6. doi: 10.3389/neuro.01.006.2010
64. Wallis, J. D., Anderson, K. C., and Miller, E. K. (2001). Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953-956. doi: 10.1038/35082081
65. Wessel, J. R., Danielmeier, C., Morton, J. B., and Ullsperger, M. (2012). Surprise and error: common neuronal architecture for the processing of errors and novelty. J. Neurosci. 32, 7528-7537. doi: 10.1523/jneurosci.6352-11.2012
66. Wilson, R. C., Takahashi, Y. K., Schoenbaum, G., and Niv, Y. (2014). Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279. doi: 10.1016/j.neuron.2013.11.005
67. Yeung, N., and Nieuwenhuis, S. (2009). Dissociating response conflict and error likelihood in anterior cingulate cortex. J. Neurosci. 29, 14506-14510. doi: 10.1523/jneurosci.3615-09.2009
68. Yu, A. J., Dayan, P., and Cohen, J. D. (2009). Dynamics of attentional selection under conflict: toward a rational Bayesian account. J. Exp. Psychol. Hum. Percept. Perform. 35, 700-717. doi: 10.1037/a0013553