A simple network architecture accounts for diverse reward time responses in primary visual cortex

Journal of Neuroscience

Volumn 35, Issue 37, 2015, Pages 12659-12672

  Huertas, Marco A ^a   Hussain Shuler, Marshall G ^b   Shouval, Harel Z ^a  

^a UNIVERSITY OF TEXAS MEDICAL SCHOOL   (United States)

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Reinforcement learning; Reward; Synaptic plasticity; Timing; Visual cortex

Indexed keywords

ARTICLE; ASSOCIATION; LEARNING; MATHEMATICAL MODEL; NERVE POTENTIAL; PRIORITY JOURNAL; REWARD; STRIATE CORTEX; VISUAL CORTEX; ANIMAL; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION; MEMBRANE POTENTIAL; NERVE CELL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; PHYSIOLOGY; REINFORCEMENT; SYNAPTIC TRANSMISSION; ULTRASTRUCTURE;

ANIMALS; COMPUTER SIMULATION; LEARNING; MEMBRANE POTENTIALS; MODELS, NEUROLOGICAL; NERVE NET; NEURAL NETWORKS (COMPUTER); NEURONAL PLASTICITY; NEURONS; REINFORCEMENT (PSYCHOLOGY); REWARD; SYNAPTIC TRANSMISSION; VISUAL CORTEX;

EID: 84941796840     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0871-15.2015     Document Type: Article

References (44)

1. Alexander WH, Brown JW (2011) Medial prefrontal cortex as an actionoutcome predictor. Nat Neurosci 14:1338–1344. CrossRef Medline
2. Alexander WH, Brown JW (2014) A general role for medial prefrontal cortex in event prediction. Front Comput Neurosci 8:69. CrossRef Medline
3. Alitto HJ, Dan Y (2013) Cell-type-specific modulation of neocortical activity by basal forebrain input. Front Syst Neurosci 6:79. CrossRef Medline
4. Bekolay T, Laubach M, Eliasmith C (2014) A spiking neural integrator model of the adaptive control of action by the medial prefrontal cortex. J Neurosci 34:1892–1902. CrossRef Medline
5. Buhusi CV, Meck WH (2005) What makes us tick? Functional and neural mechanisms of interval timing. Nat Rev Neurosci 6:755–765. CrossRef Medline
6. Cassenaer S, Laurent G (2012) Conditional modulation of spike-timingdependent plasticity for olfactory learning. Nature 482:47–52. CrossRef Medline
7. Chubykin AA, Roach EB, Bear MF, Shuler MG (2013) A cholinergic mechanism for reward timing within primary visual cortex. Neuron 77:723–735. CrossRef Medline
8. Dayan P, Abbott LF (2005) Theoretical neuroscience: computational and mathematical modeling of neural systems. Cambridge, MA: Massachusetts Institute of Technology.
9. Froemke RC, Merzenich MM, Schreiner CE (2007) A synaptic memory trace for cortical receptive field plasticity. Nature 450:425–429. CrossRef Medline
10. Gavornik JP, Shouval HZ (2011) A network of spiking neurons that can represent interval timing: mean field analysis. J Comput Neurosci 30:501–513. CrossRef Medline
11. Gavornik JP, Shuler MG, Loewenstein Y, Bear MF, Shouval HZ (2009) Learning reward timing in cortex through reward dependent expression of synaptic plasticity. Proc Natl Acad Sci U S A 106:6826–6831. CrossRef Medline
12. Gershman SJ, Moustafa AA, Ludvig EA (2014) Time representation in reinforcement learning models of the basal ganglia. Front Comput Neurosci 7:194. CrossRef Medline
13. 2+ imaging reveals selective reward effects on stimulus-specific assemblies in mouse visual cortex. J Neurosci 33:11540–11555. CrossRef Medline
14. Gulledge AT, Park SB, Kawaguchi Y, Stuart GJ (2007) Heterogeneity of phasic cholinergic signaling in neocortical neurons. J Neurophysiol 97:2215–2229. CrossRef Medline
15. Haas JS, Nowotny T, Abarbanel HD (2006) Spike-timing-dependent plasticity of inhibitory synapses in the entorhinal cortex. J Neurophysiol 96: 3305–3313. CrossRef Medline
16. Holmgren CD, Zilberter Y (2001) Coincident spiking activity induces longterm changes in inhibition of neocortical pyramidal cells. J Neurosci 21: 8270–8277. Medline
17. Izhikevich EM (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17:2443–2452. CrossRef Medline
18. Kempter R, Gerstner W, van Hemmen JL (1999) Hebbian learning and spiking neurons. Phys Rev E 59:4498. CrossRef
19. Kuczewski N, Aztiria E, Gautam D, Wess J, Domenici L (2005) Acetylcholine modulates cortical synaptic transmission via different muscarinic receptors, as studied with receptor knockout mice: acetylcholine modulates synaptic transmission. J Physiol 566:907–919. CrossRef Medline
20. Lamme VA, Roelfsema PR (2000) The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci 23:571–579. CrossRef Medline
21. Liu CH, Coleman JE, Davoudi H, Zhang K, Hussain Shuler MG (2015) Selective activation of a putative reinforcement signal conditions cued interval timing in primary visual cortex. Curr Biol 25:1551–1561. CrossRef Medline
22. Machens CK, Romo R, Brody CD (2010) Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J Neurosci 30:350–360. CrossRef Medline
23. Mauk MD, Buonomano DV (2004) The neural basis of temporal processing. Annu Rev Neurosci 27:307–340. CrossRef Medline
24. Merchant H, Harrington DL, Meck WH (2013) Neural basis of the perception and estimation of time. Annu Rev Neurosci 36:313–336. CrossRef Medline
25. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936–1947. Medline
26. Moshitch D, Las L, Ulanovsky N, Bar-Yosef O, Nelken I (2006) Responses of neurons in primary auditory cortex (A1) to pure tones in the halothaneanesthetized cat. J Neurophysiol 95:3756–3769. CrossRef Medline
27. Namboodiri VM, Huertas MA, Monk KJ, Shouval HZ, Hussain Shuler MG (2015) Visually cued action timing in the primary visual cortex. Neuron 86:319–330. CrossRef Medline
28. Narayanan NS, Laubach M (2006) Top-down control of motor cortex ensembles by dorsomedial prefrontal cortex. Neuron 52:921–931. CrossRef Medline
29. Narayanan NS, Laubach M (2009) Delay activity in rodent frontal cortex during a simple reaction time task. J Neurophysiol 101:2859–2871. CrossRef Medline
30. Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235–6242. CrossRef Medline
31. Santos-Benitez H, Magariños-Ascone CM, Garcia-Austt E (1995) Nucleus basalis of Meynert cell responses in awake monkeys. Brain Res Bull 37: 507–511. CrossRef Medline
32. Shuler MG, Bear MF (2006) Reward timing in the primary visual cortex. Science 311:1606–1609. CrossRef Medline
33. Singh R, Eliasmith C (2006) Higher-dimensional neurons explain the tuning and dynamics of working memory cells. J Neurosci 26:3667–3678. CrossRef Medline
34. Supèr H, Spekreijse H, Lamme VA (2001) A neural correlate of working memory in the monkey primary visual cortex. Science 293:120–124. CrossRef Medline
35. Sutton RS, Barto AG (1998) Reinforcement learning. Cambridge, MA: Massachusetts Institute of Technology.
36. Terhune DB, Russo S, Near J, Stagg CJ, Cohen Kadosh R (2014) GABA predicts time perception. J Neurosci 34:4364–4370. CrossRef Medline
37. Vogels TP, Sprekeler H, Zenke F, Clopath C, Gerstner W (2011) Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334:1569–1573. CrossRef Medline
38. Wang L, Maffei A (2014) Inhibitory plasticity dictates the sign of plasticity at excitatory synapses. J Neurosci 34:1083–1093. CrossRef Medline
39. Wang M, Yang Y, Wang CJ, Gamo NJ, Jin LE, Mazer JA, Morrison JH, Wang XJ, Arnsten AF (2013) NMDA receptors subserve persistent neuronal firing during working memory in dorsolateral prefrontal cortex. Neuron 77:736–749. CrossRef Medline
40. Wang XJ (1999) Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. J Neurosci 19:9587–9603. Medline
41. Wilson FA, Rolls ET (1990) Neuronal responses related to reinforcement in the primate basal forebrain. Brain Res 509:213–231. CrossRef Medline
42. Wörgötter F, Porr B (2005) Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput 17:245–319. CrossRef Medline
43. Xue M, Atallah BV, Scanziani M (2014) Equalizing excitation–inhibition ratios across visual cortical neurons. Nature 511:596–600. CrossRef Medline
44. Yagishita S, Hayashi-Takagi A, Ellis-Davies GC, Urakubo H, Ishii S, Kasai H (2014) A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science 345:1616–1620. CrossRef Medline