Behavioral variability through stochastic choice and its gating by anterior cingulate cortex

Cell

Volumn 159, Issue 1, 2014, Pages 21-32

  Tervo, Dougal G R ^a   Proskurin, Mikhail ^a   Manakov, Maxim ^a   Kabra, Mayank ^a   Vollmer, Alison ^a   Branson, Kristin ^a   Karpova, Alla Y ^a  

^a HOWARD HUGHES MEDICAL INSTITUTE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL EXPERIMENT; ANTERIOR CINGULATE; ARTICLE; BEHAVIOR; CONTROLLED STUDY; LOCUS CERULEUS; MALE; NONHUMAN; NORADRENERGIC SYSTEM; OUTCOME ASSESSMENT; RAT; TRANSGENIC RAT; ANIMAL; ANIMAL BEHAVIOR; CINGULATE GYRUS; COMPETITIVE BEHAVIOR; DECISION MAKING; DRUG EFFECTS; PHYSIOLOGY; STATISTICS;

ANIMALS; BEHAVIOR, ANIMAL; CHOICE BEHAVIOR; COMPETITIVE BEHAVIOR; GYRUS CINGULI; LOCUS COERULEUS; RATS; RATS, TRANSGENIC; STOCHASTIC PROCESSES;

EID: 84907486070     PISSN: 00928674     EISSN: 10974172     Source Type: Journal    
DOI: 10.1016/j.cell.2014.08.037     Document Type: Article

References (51)

1. H. Abe, and D. Lee Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex Neuron 70 2011 731 741
2. H. Abe, H. Seo, and D. Lee The prefrontal cortex and hybrid learning during iterative competitive games Ann. N Y Acad. Sci. 1239 2011 100 108
3. B.N. Armbruster, X. Li, M.H. Pausch, S. Herlitze, and B.L. Roth Evolving the lock to fit the key to create a family of G protein-coupled receptors potently activated by an inert ligand Proc. Natl. Acad. Sci. USA 104 2007 5163 5168
4. D.J. Barraclough, M.L. Conroy, and D. Lee Prefrontal cortex and decision making in a mixed-strategy game Nat. Neurosci. 7 2004 404 410
5. H. Bauer, J. Pripfl, C. Lamm, C. Prainsack, and N. Taylor Functional neuroanatomy of learned helplessness Neuroimage 20 2003 927 939
6. J.M. Beck, W.J. Ma, X. Pitkow, P.E. Latham, and A. Pouget Not noisy, just wrong: the role of suboptimal inference in behavioral variability Neuron 74 2012 30 39
7. T.J. Buschman, E.L. Denovellis, C. Diogo, D. Bullock, and E.K. Miller Synchronous oscillatory neural ensembles for rules in the prefrontal cortex Neuron 76 2012 838 846
8. J.D. Cohen, S.M. McClure, and A.J. Yu Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration Philos. Trans. R. Soc. Lond. B Biol. Sci. 362 2007 933 942
9. A.C. Courville, N.D. Daw, and D.S. Touretzky Bayesian theories of conditioning in a changing world Trends Cogn. Sci. 10 2006 294 300
10. N.D. Daw, J.P. O'Doherty, P. Dayan, B. Seymour, and R.J. Dolan Cortical substrates for exploratory decisions in humans Nature 441 2006 876 879
11. P. Devoto, G. Flore, L. Pani, and G.L. Gessa Evidence for co-release of noradrenaline and dopamine from noradrenergic neurons in the cerebral cortex Mol. Psychiatry 6 2001 657 664
12. S. Dong, J.A. Allen, M. Farrell, and B.L. Roth A chemical-genetic approach for precise spatio-temporal control of cellular signaling Mol. Biosyst. 6 2010 1376 1380
13. M.C. Dorris, and P.W. Glimcher Activity in posterior parietal cortex is correlated with the relative subjective desirability of action Neuron 44 2004 365 378
14. A.A. Faisal, L.P. Selen, and D.M. Wolpert Noise in the nervous system Nat. Rev. Neurosci. 9 2008 292 303
15. M.S. Fee, and J.H. Goldberg A hypothesis for basal ganglia-dependent reinforcement learning in the songbird Neuroscience 198 2011 152 170
16. J. Friedman, T. Hastie, and R. Tibshirani Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors) Ann. Stat. 28 2000 337 407
17. C.R. Gallistel, S. Fairhurst, and P. Balsam The learning curve: implications of a quantitative analysis Proc. Natl. Acad. Sci. USA 101 2004 13124 13131
18. J.I. Gold, and M.N. Shadlen The neural basis of decision making Annu. Rev. Neurosci. 30 2007 535 574
19. C.S. Green, C. Benson, D. Kersten, and P. Schrater Alterations in choice behavior by manipulations of world model Proc. Natl. Acad. Sci. USA 107 2010 16401 16406
20. B.Y. Hayden, J.M. Pearson, and M.L. Platt Neuronal basis of sequential foraging decisions in a patchy environment Nat. Neurosci. 14 2011 933 939
21. D.V. Hinkley Inference about the change-point from cumulative sum tests Biometrika 58 1971 509 523
22. C.B. Holroyd, and N. Yeung Motivation of extended behaviors by anterior cingulate cortex Trends Cogn. Sci. 16 2012 122 128
23. M. Jepma, and S. Nieuwenhuis Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory J. Cogn. Neurosci. 23 2011 1587 1596
24. M.H. Kao, A.J. Doupe, and M.S. Brainard Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song Nature 433 2005 638 643
25. M.P. Karlsson, D.G. Tervo, and A.Y. Karpova Network resets in medial prefrontal cortex mark the onset of behavioral uncertainty Science 338 2012 135 139
26. D. Lee, B.P. McGreevy, and D.J. Barraclough Learning and decision making in monkeys during a rock-paper-scissors game Brain Res. Cogn. Brain Res. 25 2005 416 430
27. D. Lee, M.L. Conroy, B.P. McGreevy, and D.J. Barraclough Reinforcement learning and decision making in monkeys during a competitive game Brain Res. Cogn. Brain Res. 22 2004 45 58
28. J.H. Martin Autoradiographic estimation of the extent of reversible inactivation produced by microinjection of lidocaine and muscimol in the rat Neurosci. Lett. 127 1991 160 164
29. K. Matsumoto, W. Suzuki, and K. Tanaka Neuronal correlates of goal-based motor selection in the prefrontal cortex Science 301 2003 229 232
30. A. Maximov, O.-H. Shin, X. Liu, and T.C. Südhof Synaptotagmin-12, a synaptic vesicle phosphoprotein that modulates spontaneous neurotransmitter release J. Cell Biol. 176 2007 113 124
31. J. Maynard Smith, and D.G. Harper The evolution of aggression: can selection generate variability? Philos. Trans. R. Soc. Lond. B Biol. Sci. 319 1988 557 570
32. J.F. Nash Equilibrium points in n-person games Proc. Natl. Acad. Sci. USA 36 1950 48 49
33. M.R. Nassar, R.C. Wilson, B. Heasly, and J.I. Gold An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment J. Neurosci. 30 2010 12366 12378
34. M.R. Nassar, K.M. Rumsey, R.C. Wilson, K. Parikh, B. Heasly, and J.I. Gold Rational regulation of learning dynamics by pupil-linked arousal systems Nat. Neurosci. 15 2012 1040 1046
35. J.X. O'Reilly, U. Schüffelgen, S.F. Cuell, T.E. Behrens, R.B. Mars, and M.F. Rushworth Dissociable effects of surprise and model update in parietal and anterior cingulate cortex Proc. Natl. Acad. Sci. USA 110 2013 E3660 E3669
36. B.P. Ölveczky, A.S. Andalman, and M.S. Fee Vocal experimentation in the juvenile songbird requires a basal ganglia circuit PLoS Biol. 3 2005 e153
37. C. Padoa-Schioppa Neuronal origins of choice variability in economic decisions Neuron 80 2013 1322 1336
38. S. Page, and A. Neuringer Variability is an operant J. Exp. Psychol. Anim. Behav. Process. 11 1985 429
39. E. Payzan-LeNestour, S. Dunne, P. Bossaerts, and J.P. O'Doherty The neural representation of unexpected uncertainty during value-based decision making Neuron 79 2013 191 201
40. J.J. Ribas-Fernandes, A. Solway, C. Diuk, J.T. McGuire, A.G. Barto, Y. Niv, and M.M. Botvinick A neural signature of hierarchical reinforcement learning Neuron 71 2011 370 379
41. Sutton, R.S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. Proceedings of the Seventh International Conference on Machine Learning 1990, 216-224.
42. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction Volume 1 1998 Cambridge University Press Cambridge
43. L.-E. Trudeau, D.G. Emery, and P.G. Haydon Direct modulation of the secretory machinery underlies PKA-dependent synaptic facilitation in hippocampal neurons Neuron 17 1996 789 797
44. T.J. Vickery, M.M. Chun, and D. Lee Ubiquity and specificity of reinforcement signals throughout the human brain Neuron 72 2011 166 177
45. X.-J. Wang Probabilistic decision making by slow reverberation in cortical circuits Neuron 36 2002 955 968
46. C.J. Watkins, and P. Dayan Q-learning Mach. Learn. 8 1992 279 292
47. I.B. Witten, E.E. Steinberg, S.Y. Lee, T.J. Davidson, K.A. Zalocusky, M. Brodsky, O. Yizhar, S.L. Cho, S. Gong, and C. Ramakrishnan Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement Neuron 72 2011 721 733
48. Z.Q.D. Xu, T.J.S. Shi, and T. Hökfelt Galanin/GMAP- and NPY-like immunoreactivities in locus coeruleus and noradrenergic nerve terminals in the hippocampal formation and cortex with notes on the galanin-R1 and -R2 receptors J. Comp. Neurol. 392 1998 227 251
49. W. Yoshida, and S. Ishii Resolution of uncertainty in prefrontal cortex Neuron 50 2006 781 789
50. A.J. Yu, and P. Dayan Uncertainty, neuromodulation, and attention Neuron 46 2005 681 692
51. L. Zhu, K.E. Mathewson, and M. Hsu Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning Proc. Natl. Acad. Sci. USA 109 2012 1419 1424