Rule learning enhances structural plasticity of long-range axons in frontal cortex

Nature Communications

Volumn 7, Issue , 2016, Pages

  Johnson, Carolyn M ^a   Peckler, Hannah ^b   Tai, Lung Hao ^b   Wilbrecht, Linda ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; CELLS AND CELL COMPONENTS; CONNECTIVITY; ERROR ANALYSIS; LEARNING; MAGNITUDE; PHYSIOLOGY; PREDICTION;

ERROR; FRONTAL CORTEX; GENE LOSS; LEARNING; NERVE FIBER; ORBITAL CORTEX; PLASTICITY; PREDICTION; REWARD; ANIMAL; ANIMAL MODEL; AXON; C57BL MOUSE; DECISION MAKING; HUMAN; MALE; MOUSE; PHYSIOLOGY; PREFRONTAL CORTEX;

ANIMALS; AXONS; CHOICE BEHAVIOR; HUMANS; LEARNING; MALE; MICE; MICE, INBRED C57BL; MODELS, ANIMAL; PREFRONTAL CORTEX; REWARD;

EID: 84960449501     PISSN: None     EISSN: 20411723     Source Type: Journal    
DOI: 10.1038/ncomms10785     Document Type: Article

References (70)

1. Bunge, S. A. & Wallis, J. D. Neuroscience of Rule-Guided Behavior (Oxford Univ. Press, 2007).
2. White, I. M. & Wise, S. P. Rule-dependent neuronal activity in the prefrontal cortex. Exp. Brain Res. 126, 315-335 (1999).
3. Asaad, W. F., Rainer, G. & Miller, E. K. Task-specific neural activity in the primate prefrontal cortex. J. Neurophysiol. 84, 451-459 (2000).
4. Wallis, J. D., Anderson, K. C. & Miller, E. K. Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953-956 (2001).
5. Badre, D., Kayser, A. S. & D'Esposito, M. Frontal cortex and the discovery of abstract action rules. Neuron 66, 315-326 (2010).
6. Durstewitz, D., Vittoz, N. M., Floresco, S. B. & Seamans, J. K. Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron 66, 438-448 (2010).
7. Mian, M. K. et al. Encoding of rules by neurons in the human dorsolateral prefrontal cortex. Cereb. Cortex 24, 807-816 (2014).
8. Schoenbaum, G., Roesch, M. R., Stalnaker, T. A. & Takahashi, Y. K. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885-892 (2009).
9. Luk, C.-H. & Wallis, J. D. Choice coding in frontal cortex during stimulusguided or action-guided decision-making. J. Neurosci. 33, 1864-1871 (2013).
10. Milner, B. Effects of different brain lesions on card sorting: the role of the frontal lobes. Arch. Neurol. 9, 90-100 (1963).
11. Kim, J. & Ragozzino, M. E. The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiol. Learn. Mem. 83, 125-133 (2005).
12. Johnson, C. & Wilbrecht, L. Juvenile mice show greater flexibility in multiple choice reversal learning than adults. Dev. Cogn. Neurosci. 1, 540-551 (2011).
13. Buckley, M. J. et al. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52-58 (2009).
14. Kovach, C. K. et al. Anterior prefrontal cortex contributes to action selection through tracking of recent reward trends. J. Neurosci. 32, 8434-8442 (2012).
15. Gremel, C. & Costa, R. Premotor cortex is critical for goal-directed actions. Front. Comput. Neurosci. 712, 110 (2013).
16. Holtmaat, A. & Svoboda, K. Experience-dependent structural synaptic plasticity in the mammalian brain. Nat. Rev. Neurosci. 10, 647-658 (2009).
17. Caroni, P., Donato, F. & Muller, D. Structural plasticity upon learning: regulation and functions. Nat. Rev. Neurosci. 13, 478-490 (2012).
18. Yang, G., Pan, F. & Gan, W.-B. Stably maintained dendritic spines are associated with lifelong memories. Nature 462, 920-924 (2009).
19. Xu, T. et al. Rapid formation and selective stabilization of synapses for enduring motor memories. Nature 462, 915-919 (2009).
20. Lai, C. S., Franke, T. F. & Gan, W. B. Opposite effects of fear conditioning and extinction on dendritic spine remodelling. Nature 483, 87-91 (2012).
21. Munoz-Cuevas, F. J., Athilingam, J., Piscopo, D. & Wilbrecht, L. Cocaine-induced structural plasticity in frontal cortex correlates with conditioned place preference. Nat. Neurosci. 16, 1367-1369 (2013).
22. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT press, 1998).
23. Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should i go? How the human brain manages the trade-off between exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933-942 (2007).
24. Rushworth, M. F. S. & Behrens, T. E. J. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389-397 (2008).
25. Kayser, A. S., Mitchell, J. M., Weinstein, D. & Frank, M. J. Dopamine, locus of control, and the exploration-exploitation tradeoff. Neuropsychopharmacology 40, 454-462 (2015).
26. Hills, T. T. Animal foraging and the evolution of goal-directed cognition. Cogn.e Sci. 30, 3-41 (2006).
27. Pearson John, M., Watson Karli, K. & Platt Michael, L. Decision making: The neuroethological turn. Neuron 82, 950-965 (2014).
28. Beitz, K. M., Salthouse, T. A. & Davis, H. P. Performance on the iowa gambling task: From 5 to 89 years of age. J. Exp. Psychol. Gen. 143, 1677-1689 (2014).
29. Lee, D. Decision making: From neuroscience to psychiatry. Neuron 78, 233-248 (2013).
30. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876-879 (2006).
31. Donoso, M., Collins, A. G. E. & Koechlin, E. Foundations of human reasoning in the prefrontal cortex. Science 344, 1481-1486 (2014).
32. Quilodran, R., Rothé, M. & Procyk, E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron 57, 314-325 (2008).
33. Boorman, E. D., Behrens, T. E. J. & Woolrich, M. W. Rushworth MFS. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron 62, 733-743 (2009).
34. Hayden, B. Y., Pearson, J. M. & Platt, M. L. Neuronal basis of sequential foraging decisions in a patchy environment. Nat. Neurosci. 14, 933-939 (2011).
35. Donahue Christopher, H., Seo, H. & Lee, D. Cortical signals for rewarded actions and strategic exploration. Neuron 80, 223-234 (2013).
36. Tervo Dougal, G. R. et al. Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21-32 (2014).
37. Helmholtz, H. The Facts of Perception (1878).
38. Rescorla, R. A. & Wagner, A. R. in Classical Conditioning ii: Current Theory and Research. (eds Black, A. H. & Prokasy, W. F.) (Meredith Corporation, 1972).
39. Bush, R. R. & Mosteller, F. A mathematical model for simple learning. Psychol. Rev. 58, 313-323 (1951).
40. Holtmaat, A. et al. Long-term, high-resolution imaging in the mouse neocortex through a chronic cranial window. Nat. Protoc. 4, 1128-1144 (2009).
41. De Paola, V. et al. Cell type-specific structural plasticity of axonal branches and boutons in the adult neocortex. Neuron 49, 861-875 (2006).
42. Wilson Robert, C., Takahashi Yuji, K., Schoenbaum, G. & Niv, Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279 (2014).
43. Seamans, J., Lapish, C. & Durstewitz, D. Comparing the prefrontal cortex of rats and primates: Insights from electrophysiology. Neurotox. Res. 14, 249-262 (2008).
44. Knott, G. W., Holtmaat, A., Wilbrecht, L., Welker, E. & Svoboda, K. Spine growth precedes synapse formation in the adult neocortex in vivo. Nat. Neurosci. 9, 1117-1124 (2006).
45. Grillo, F. W. et al. Increased axonal bouton dynamics in the aging mouse cortex. Proc. Natl Acad. Sci. USA 110, E1514-E1523 (2013).
46. Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78-84 (2013).
47. Rigotti, M. et al. The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585-590 (2013).
48. Siegel, M., Buschman, T. J. & Miller, E. K. Cortical information flow during flexible sensorimotor decisions. Science 348, 1352-1355 (2015).
49. Kennerley, S. W. & Wallis, J. D. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061-2073 (2009).
50. Kennerley, S. W., Behrens, T. E. J. & Wallis, J. D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 1581-1589 (2011).
51. Knill, D. C. & Pouget, A. The bayesian brain: The role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712-719 (2004).
52. Friston, K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127-138 (2010).
53. Keller Georg, B., Bonhoeffer, T. & Hübener, M. Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron 74, 809-815 (2012).
54. Poort, J. et al. Learning enhances sensory and multiple non-sensory representations in primary visual cortex. Neuron 86, 1478-1490 (2015).
55. Wolpert, D. M., Ghahramani, Z. & Jordan, M. I. An internal model for sensorimotor integration. Science 269, 1880-1882 (1995).
56. Glimcher, P. W. Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. Proc. Natl Acad. Sci. USA 108, 15647-15654 (2011).
57. O'Neill, M. & Schultz, W. Risk prediction error coding in orbitofrontal neurons. J. Neurosci. 33, 15810-15814 (2013).
58. Hayden, B. Y., Heilbronner, S. R., Pearson, J. M. & Platt, M. L. Surprise signals in anterior cingulate cortex: Neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J. Neurosci. 31, 4178-4187 (2011).
59. Schoenbaum, G., Nugent, S. L., Saddoris, M. P. & Setlow, B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 13, 885-890 (2002).
60. Ghods-Sharifi, S., Haluk, D. M. & Floresco, S. B. Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning. Neurobiol. Learn. Mem. 89, 567-573 (2008).
61. Bussey, T. J., Muir, J. L., Everitt, B. J. & Robbins, T. W. Dissociable effects of anterior and posterior cingulate cortex lesions on the acquisition of a conditional visual discrimination: Facilitation of early learning vs impairment of late learning. Behav. Brain. Res. 82, 45-56 (1996).
62. Bussey, T. J., Muir, J. L., Everitt, B. J. & Robbins, T. W. Triple dissociation of anterior cingulate, posterior cingulate, and medial frontal cortices on visual discrimination tasks using a touchscreen testing procedure for the rat. Behav. Neurosci. 111, 920-936 (1997).
63. Holtmaat, A., Wilbrecht, L., Knott, G. W., Welker, E. & Svoboda, K. Experience-dependent and cell-type-specific spine growth in the neocortex. Nature 441, 979-983 (2006).
64. Keck, T. et al. Massive restructuring of neuronal circuits during functional reorganization of adult visual cortex. Nat. Neurosci. 11, 1162-1167 (2008).
65. Hofer, S. B., Mrsic-Flogel, T. D., Bonhoeffer, T. & Hubener, M. Experience leaves a lasting structural trace in cortical circuits. Nature 457, 313-317 (2009).
66. Hübener, M. & Bonhoeffer, T. Neuronal plasticity: Beyond the critical period. Cell 159, 727-737 (2014).
67. O'Doherty, J., Kringelbach, M. L., Rolls, E. T., Hornak, J. & Andrews, C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95-102 (2001).
68. Frank, M. J., Moustafa, A. A., Haughey, H. M., Curran, T. & Hutchison, K. E. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc. Natl Acad. Sci. USA 104, 16311-16316 (2007).
69. Khan, R. M. et al. Predicting odor pleasantness from odorant structure: Pleasantness as a reflection of the physical world. J. Neurosci. 27, 10015-10023 (2007).
70. Schönberg, T., Daw, N. D., Joel, D. & O'Doherty, J. P. Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J. Neurosci. 27, 12860-12867 (2007).