Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework

PLoS Computational Biology

Volumn 12, Issue 2, 2016, Pages

  Song, H Francis ^a   Yang, Guangyu R ^a   Wang, Xiao Jing ^a,b  

^a NEW YORK UNIVERSITY   (United States)

^b NEW YORK UNIVERSITY SHANGHAI   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMALS; DECISION MAKING; ELECTRIC NETWORK ANALYSIS; LEARNING SYSTEMS; NEURONS; OPTIMIZATION;

COGNITIVE FUNCTIONS; COGNITIVE TASK; FLEXIBLE FRAMEWORK; MECHANISTICS; NETWORK PARAMETERS; NEURAL CIRCUITS; NEURAL COMPUTATIONS; OPTIMIZATION OF NETWORK; SIMPLE++; TESTING GROUNDS;

RECURRENT NEURAL NETWORKS;

COGNITION; CONSENSUS DEVELOPMENT; DECISION MAKING; DIFFERENTIATION; MACHINE LEARNING; NERVOUS SYSTEM; THEORETICAL MODEL; WORKING MEMORY; ALGORITHM; ANIMAL; BIOLOGICAL MODEL; BIOLOGY; COMPUTER SIMULATION; HAPLORHINI; NERVE CELL; PHYSIOLOGY; PROCEDURES;

ALGORITHMS; ANIMALS; COGNITION; COMPUTATIONAL BIOLOGY; COMPUTER SIMULATION; HAPLORHINI; MODELS, NEUROLOGICAL; NEURONS;

EID: 84959494188     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1004792     Document Type: Article

References (83)

1. Barak O, Tsodyks M, Romo R, Neuronal population coding of parametric working memory. 2010J Neurosci;30:9424–9430. doi: 10.1523/JNEUROSCI.1875-10.201020631171
2. Rigotti M, Rubin DBD, Wang XJ, Fusi S, Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. 2010Front Comput Neurosci;4:24. doi: 10.3389/fncom.2010.0002421048899
3. Rigotti M, Barak O, Warden MR, Wang XJ, Daw ND, Miller EK, et al. The importance of mixed selectivity in complex cognitive tasks. 2013Nature;497:585–590. doi: 10.1038/nature1216023685452
4. Yuste R, From the neuron doctrine to neural networks. 2015Nat Rev Neurosci;16:487–497. doi: 10.1038/nrn396226152865
5. Mante V, Sussillo D, Shenoy KV, Newsome WT, Context-dependent computation by recurrent dynamics in prefrontal cortex. 2013Nature;503:78–84. doi: 10.1038/nature1274224201281
6. Churchland MM, Cunningham JP, Kaufman MT, Foster JD, Nuyujukian P, Ryu SI, et al. Neural population dynamics during reaching. 2012Nature;487:51–56. doi: 10.1038/nature1112922722855
7. Carnevale F, de Lafuente V, Romo R, Barak O, Parga N, Dynamic control of response criterion in premotor cortex during perceptual detection under temporal uncertainty. 2015Neuron;86:1067–1077. doi: 10.1016/j.neuron.2015.04.01425959731
8. Sussillo D, Churchland MM, Kaufman MT, Shenoy KV, A neural network that finds a naturalistic solution for the production of muscle activity. 2015Nat Neurosci;18:1025–1033. doi: 10.1038/nn.404226075643
9. Siegel M, Buschman TJ, Miller EK, Cortical information flow during flexible sensorimotor decisions. 2015Science;348:1352–55. doi: 10.1126/science.aab055126089513
10. Stevenson IH, Kording KP, How advances in neural recording affect data analysis. 2011Nat Neurosci;14:139–142. doi: 10.1038/nn.273121270781
11. Cunningham JP, Yu BM, Dimensionality reduction for large-scale neural recordings. 2014Nat Neurosci;17:1500–1509. doi: 10.1038/nn.377625151264
12. Gao P, Ganguli S, On simplicity and complexity in the brave new world of large-scale neuroscience. 2015Curr Opin Neurobiol;32:148–155. doi: 10.1016/j.conb.2015.04.00325932978
13. Gerstner W, Kistler WM, Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge, UK: Cambridge University Press; 2002.
14. Funahashi K, Nakamura Y, Approximation of dynamical systems by continuous time recurrent neural networks. 1993Neural Networks;6:801–806. doi: 10.1016/S0893-6080(05)80125-X
15. Siegelmann HT, Sontag ED, On the Computational Power of Neural Nets. 1995J Comput Syst Sci;50:132–150. doi: 10.1006/jcss.1995.1013
16. Yamins DLK, Hong H, Cadieu CF, Solomon Ea, Seibert D, DiCarlo JJ, Performance-optimized hierarchical models predict neural responses in higher visual cortex. 2014Proc Natl Acad Sci U S A;111:8619–24. doi: 10.1073/pnas.140311211124812127
17. Sussillo D, Neural circuits as computational dynamical systems. 2014Curr Opin Neurobiol;25:156–163. doi: 10.1016/j.conb.2014.01.00824509098
18. Zipser D, Andersen RA, A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons. 1988Nature;331:679–684. doi: 10.1038/331679a03344044
19. Rumelhart DE, McClelland JL, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. Cambridge, MA: MIT Press; 1986.
20. Cohen JD, Dunbar K, McClelland JL, On the control of automatic processes: a parallel distributed processing account of the Stroop effect. 1990Psychol Rev;97:332–361. doi: 10.1037/0033-295X.97.3.3322200075
21. Eccles JC, Fatt P, Koketsu K, Cholinergic and inhibitory synapses in a pathway from motor-axon collaterals to motoneurones. 1954J Physiol;126:524–562. doi: 10.1113/jphysiol.1954.sp00522613222354
22. Murphy BK, Miller KD, Balanced amplification: A new mechanism of selective amplification of neural activity patterns. 2009Neuron;61:635–48. doi: 10.1016/j.neuron.2009.02.00519249282
23. Markram H, Toledo-Rodriguez M, Wang Y, Gupta A, Silberberg G, Wu C, Interneurons of the neocortical inhibitory system. 2004Nat Rev Neurosci;5:793–807. doi: 10.1038/nrn151915378039
24. Song S, Sjöström PJ, Reigl M, Nelson S, Chklovskii DB, Highly nonrandom features of synaptic connectivity in local cortical circuits. 2005PLoS Biol;3:e68. doi: 10.1371/journal.pbio.003006815737062
25. Pfeffer CK, Xue M, He M, Huang ZJ, Scanziani M, Inhibition of inhibition in visual cortex: the logic of connections between molecularly distinct interneurons. 2013Nat Neurosci;16:1068–76. doi: 10.1038/nn.344623817549
26. Potjans TC, Diesmann M, The cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking network model. 2014Cereb Cortex;24:785–806. doi: 10.1093/cercor/bhs35823203991
27. Jiang X, Shen S, Cadwell CR, Berens P, Sinz F, Ecker AS, et al. Principles of connectivity among morphologically defined cell types in adult neocortex. 2015Science;350:aac9462. doi: 10.1126/science.aac946226612957
28. Ercsey-Ravasz M, Markov NT, Lamy C, Van Essen DC, Knoblauch K, Toroczkai Z, et al. A predictive network model of cerebral cortical connectivity based on a distance rule. 2013Neuron;80:184–197. doi: 10.1016/j.neuron.2013.07.03624094111
29. Markov NT, Ercsey-Ravasz MM, Ribeiro Gomes aR, Lamy C, Magrou L, Vezoli J, et al. A weighted and directed interareal connectivity matrix for macaque cerebral cortex. 2014Cereb Cortex;24:17–36. doi: 10.1093/cercor/bhs27023010748
30. Song HF, Kennedy H, Wang XJ, Spatial embedding of structural similarity in the cerebral cortex. 2014Proc Natl Acad Sci;111:16580–16585. doi: 10.1073/pnas.141415311125368200
31. Sussillo D, Abbott LF, Generating coherent patterns of activity from chaotic neural networks. 2009Neuron;63:544–57. doi: 10.1016/j.neuron.2009.07.01819709635
32. Laje R, Buonomano DV, Robust timing and motor patterns by taming chaos in recurrent neural networks. 2013Nat Neurosci;16:925–33. doi: 10.1038/nn.340523708144
33. Martens J, Sutskever I. Learning recurrent neural networks with Hessian-free optimization. 2011 Proc 28th Int Conf Mach Learn;Available from: http://www.icml-2011.org/papers/532_icmlpaper.pdf.
34. Barak O, Sussillo D, Romo R, Tsodyks M, Abbott LF, rom fixed points to chaos: Three models of delayed discrimination. 2013Prog Neurobiol;103:214–22. doi: 10.1016/j.pneurobio.2013.02.00223438479
35. Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. 2013 Proc 30th Int Conf Mach Learn;Available from: http://jmlr.org/proceedings/papers/v28/pascanu13.pdf.
36. Bengio Y, Boulanger-Lewandowski N, Pascanu R. Advances in optimizing recurrent networks. In: Proc. Int. Conf. Acoust. Speech, Signal Process.; 2013. Available from: http://arxiv.org/pdf/1212.0901v2.pdf.
37. Hardt M, Recht B, Singer Y. 2015 Train faster, generalize better: Stability of stochastic gradient descent;Available from: http://arxiv.org/abs/1509.01240.
38. Dan Y, Poo Mm, Spike timing-dependent plasticity: From synapse to perception. 2006Physiol Rev;p. 1033–1048. doi: 10.1152/physrev.00030.200516816145
39. Bengio Y. 2015 Towards biologically plausible deep learning;Available from: http://arxiv.org/abs/1502.0415.
40. Bengio Y, Mesnard T, Fischer A, Zhang S, Wu Y. 2015 An objective function for STDP;Available from: http://arxiv.org/abs/1509.05936.
41. Bergstra J, Breuleux O, Bastien F, Lamblin P, Pascanu R, Desjardins G, et al. Theano: A CPU and GPU math compiler in Python. In: Proc. 9th Python Sci. Conf. 2010; 2010. Available from: http://conference.scipy.org/proceedings/scipy2010/pdfs/bergstra.pdf.
42. Bastien F, Lamblin P, Pascanu R, Bergstra J, Goodfellow I, Bergeron A, et al. 2012 Theano: new features and speed improvements;Available from: http://arxiv.org/abs/1211.5590.
43. Hahnloser RLT, On the piecewise analysis of networks of linear threshold neurons. 1998Neural Networks;11:691–697. doi: 10.1016/S0893-6080(98)00012-412662807
44. Pascanu R, Montúfar G, Bengio Y. On the number of response regions of deep feedforward networks with piecewise linear activations. 2014 Int Conf Learn Represent;Available from: http://arxiv.org/pdf/1312.6098v5.pdf.
45. Montúfar G, Pascanu R, Cho K, Bengio Y, On the number of linear regions of deep neural networks; 2014. Available from: http://arxiv.org/abs/1402.1869v1.
46. Gillespie DT, The mathematics of Brownian motion and Johnson noise. 1996Am J Phys;64:225. doi: 10.1119/1.18210
47. Brunton BW, Botvinick MM, Brody CD, Rats and humans can optimally accumulate evidence for decision-making. 2013Science;340:95–8. doi: 10.1126/science.123391223559254
48. Levy RB, Reyes AD, Spatial profile of excitatory and inhibitory synaptic connectivity in mouse primary auditory cortex. 2012J Neurosci;32:5609–5619. doi: 10.1523/JNEUROSCI.5158-11.201222514322
49. Thomson AM, West DC, Wang Y, Bannister AP, Synaptic connections and small circuits involving excitatory and inhibitory neurons in layers 2–5 of adult rat and cat neocortex: triple intracellular recordings and biocytin labelling in vitro. 2002Cereb Cortex;12:936–953. doi: 10.1093/cercor/12.9.93612183393
50. Binzegger T, Douglas RJ, Martin KAC, A quantitative map of the circuit of cat primary visual cortex. 2004J Neurosci;24:8441–53. doi: 10.1523/JNEUROSCI.1400-04.200415456817
51. Wang XJ, Tegnér J, Constantinidis C, Goldman-Rakic PS, Division of labor among distinct subtypes of inhibitory neurons in a cortical microcircuit of working memory. 2004Proc Natl Acad Sci U S A;101:1368–1373. doi: 10.1073/pnas.030533710114742867
52. Fino E, Packer AM, Yuste R, The logic of inhibitory connectivity in the neocortex. 2012Neurosci;19:228–237. Available from: http://dx.doi.org/10.1177/1073858412456743.
53. Karnani MM, Agetsuma M, Yuste R, A blanket of inhibition: Functional inferences from dense inhibitory connectivity. 2014Curr Opin Neurobiol;26:96–102. doi: 10.1016/j.conb.2013.12.01524440415
54. Festa D, Hennequin G, Analog memories in a balanced rate-based network of E-I neurons. In: Adv. Neural Inf. Process. Syst.; 2014. Available from: http://papers.nips.cc/paper/5336-analog-memories-in-a-balanced-rate-based-network-of-e-i-neurons.pdf.
55. Rajan K, Abbott LF, Eigenvalue Spectra of Random Matrices for Neural Networks. 2006Phys Rev Lett;97:188104. doi: 10.1103/PhysRevLett.97.18810417155583
56. Le QV, Jaitly N, Hinton GE, 2015A simple way to initialize recurrent networks of rectified linear units;p. 1–9. Available from: http://arxiv.org/abs/1504.00941.
57. Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. In: Proc. 30th Int. Conf. Mach. Learn. IEEE; 2013. Available from: http://www.cs.utoronto.ca/~ilya/pubs/2013/1051_2.pdf.
58. Rumelhart DE, Hinton GE, Williams RJ, Rumelhart DE, McClelland JL, Learning internal representations by error propagation. In: Parallel Distrib. Process. Vol. 1. Cambridge, MA: MIT Press; 1986. p. 318–362.
59. Dauphin Y, Pascanu R, Gulcehre C, Cho K, Ganguli S, Bengio Y, 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimization;Available from: http://arxiv.org/abs/1406.2572.
60. Bengio Y, Simard P, Frasconi P, Learning long-term dependencies with gradient descent is difficult. 1994IEEE Trans Neural Networks;5:157–166. doi: 10.1109/72.27918118267787
61. Newsome WT, Britten KH, Movshon Ja, Neuronal correlates of a perceptual decision. 1989Nature;341:52–54. doi: 10.1038/341052a02770878
62. Roitman JD, Shadlen MN, Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. 2002J Neurosci;22:9475–89. Available from: http://www.jneurosci.org/content/22/21/9475.long. 12417672
63. Kiani R, Hanks TD, Shadlen MN, Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. 2008J Neurosci;28:3017–3029. doi: 10.1523/JNEUROSCI.4761-07.200818354005
64. Raposo D, Kaufman MT, Churchland AK, A category-free neural population supports evolving demands during decision-making. 2014Nat Neurosci;17:1784–1792. doi: 10.1038/nn.386525383902
65. Romo R, Brody CD, Hernández A, Lemus L, Neuronal correlates of parametric working memory in the prefrontal cortex. 1999Nature;399:470–473. doi: 10.1038/2093910365959
66. Averbeck BB, Lee D, Prefrontal neural correlates of memory for sequences. 2007J Neurosci;27:2204–2211. doi: 10.1523/JNEUROSCI.4483-06.200717329417
67. Wong KF, Wang XJ, A recurrent network mechanism of time integration in perceptual decisions. 2006J Neurosci;26:1314–1328. doi: 10.1523/JNEUROSCI.3733-05.200616436619
68. Miller P, Brody CD, Romo R, Wang XJ, A recurrent network model of somatosensory parametric working memory in the prefrontal cortex. 2003Cereb Cortex;13:1208–1218. doi: 10.1093/cercor/bhg10114576212
69. Wang XJ, Synaptic basis of cortical persistent activity: The importance of NMDA receptors to working memory. 1999J Neurosci;19:9587–9603. Available from: http://www.jneurosci.org/content/19/21/9587.long. 10531461
70. Barak O, Tsodyks M, Working models of working memory. 2014Curr Opin Neurobiol;25:20–4. doi: 10.1016/j.conb.2013.10.00824709596
71. Funahashi S, Bruce CJ, Goldman-Rakic PS, Mnemonic coding of visual space in the monkey x2019;s dorsolateral prefrontal cortex. 1989J Neurophysiol;61:331–349. Available from: http://jn.physiology.org/content/61/2/331.long. 2918358
72. Hochreiter S, Schmidhuber J, Long Short-Term Memory. 1997Neural Comput;9:1735–1780. doi: 10.1162/neco.1997.9.8.17359377276
73. Renart A, de la Rocha J, Bartho P, Hollender L, Parga N, Reyes A, et al. The asynchronous state in cortical circuits. 2010Science;327:587–90. doi: 10.1126/science.117985020110507
74. Rubin DB, Hooser SDV, Miller KD, The stabilized supralinear network: A unifying circuit motif underlying multi-input integration in sensory cortex. 2013Neuron;85:1–51. Available from: http://dx.doi.org/10.1016/j.neuron.2014.12.026.
75. Sutton RS, Barto AG, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press; 1998.
76. Bakker B, Reinforcement learning with Long Short-Term Memory. 2002Adv Neural Inf Process Syst;Available from: http://papers.nips.cc/paper/1953-reinforcement-learning-with-long-short-term-memory.pdf.
77. Roelfsema PR, van Ooyen A, Attention-gated reinforcement learning of internal representations for classification. 2005Neural Comput;17:2176–2214. doi: 10.1162/089976605461569916105222
78. Izhikevich EM, Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling. 2007Cereb Cortex;17:2443–2452. doi: 10.1093/cercor/bhl15217220510
79. Potjans W, Morrison A, Diesmann M, A Spiking Neural Network Model of an Actor-Critic Agent. 2009Neural Comput;21:301–339. doi: 10.1162/neco.2008.08-07-59319196231
80. Neymotin SA, Chadderdon GL, Kerr CC, Francis JT, Lytton WW, Reinforcement learning of two-joint arm reaching in a computer model of sensorimotor cortex. 2013Neural Comput;25:3263–3293. Available from: http://dx.doi.org/10.1162/NECO. doi: 10.1162/NECO_a_0052124047323
81. Benda J, Herz AVM, A universal model for spike-frequency adaptation. 2003Neural Comput;15:2523–2564. doi: 10.1162/08997660332238506314577853
82. Engel TA, Wang XJ, Same or different? A neural circuit mechanism of similarity-based pattern match decision making. 2011J Neurosci;31:6982–6996. doi: 10.1523/JNEUROSCI.6150-10.201121562260
83. Sussillo D, Barak O, Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks. 2013Neural Comput;25:626–49. doi: 10.1162/NECO_a_0040923272922