Transition to chaos in random neuronal networks

Physical Review X

Volumn 5, Issue 4, 2015, Pages

  Kadmon, Jonathan ^a   Sompolinsky, Haim ^a,b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMICS; MEAN FIELD THEORY; MEMORY ARCHITECTURE; NETWORK ARCHITECTURE; NEURONS; RATE CONSTANTS; STATISTICAL MECHANICS; TRANSFER FUNCTIONS;

CENTRAL NERVOUS SYSTEMS; CHAOTIC FLUCTUATIONS; DYNAMIC MEAN FIELD THEORIES; IRREGULAR FLUCTUATIONS; NEURONAL CIRCUIT MODEL; NONLINEAR TRANSFER FUNCTIONS; SIGMOIDAL NONLINEARITY; SYNAPTIC CONNECTIONS;

NEURAL NETWORKS;

EID: 84951093613     PISSN: None     EISSN: 21603308     Source Type: Journal    
DOI: 10.1103/PhysRevX.5.041030     Document Type: Article

References (64)

1. M. Abeles, Corticonics: Neural Circuits of the Cerebral Cortex (Cambridge University Press, Cambridge, England, 1991).
2. W. Bair, C. Koch, W. Newsome, and K. Britten, Power Spectrum Analysis of Bursting Cells in Area MT in the Behaving Monkey, J. Neurosci. 14, 2870 (1994).
3. W. R. Softky and C. Koch, The Highly Irregular Firing of Cortical Cells Is Inconsistent with Temporal Integration of Random EPSPs, J. Neurosci. 13, 334 (1993).
4. G. R. Holt, W. R. Softky, C. Koch, and R. J. Douglas, A Comparison of Discharge Variability In Vitro and In Vivo in Cat Visual Cortex Neurons, J. Neurophysiol. 75, 1806 (1996).
5. K. Kang, R. M. Shapely, and H. Sompolinsky, Information Tuning of Populations of Neurons in Primary Visual Cortex, J. Neurosci. 24, 3726 (2004).
6. H. Sompolinsky, A. Crisanti, and H. J. Sommers, Chaos in Randon Neural Networks, Phys. Rev. Lett. 61, 259 (1988).
7. C. van Vreeswijk and H. Sompolinsky, Chaos in Neuronal Networks with Balanced Excitatory and Inhibitory Activity, Science 274, 1724 (1996).
8. C. van Vreeswijk and H. Sompolinsky, Chaotic Balanced State in a Model of Cortical Circuits, Neural Comput. 10, 1321 (1998).
9. A. Renart, J. de la Rocha, P. Bartho, L. Hollender, N. Parga, A. Reyes, and K. D. Harris, The Asynchronous State in Cortical Circuits, Science 327, 587 (2010).
10. N. Brunel, Phase Diagrams of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons, Neurocomputing;Variable Star Bulletin 32-33, 307 (2000).
11. L. Busing, B. Schrauwen, and R. Legenstein, Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons, Neural Comput. 22, 1272 (2010).
12. M. Lukoševicius and H. Jaeger, Reservoir Computing Approaches to Recurrent Neural Network Training, Comput. Sci. Rev. 3, 127 (2009).
13. D. Sussillo and L. F. Abbott, Generating Coherent Patterns of Activity from Chaotic Neural Networks, Neuron 63, 544 (2009).
14. O. Barak, D. Sussillo, R. Romo, M. Tsodyks, and L. F. Abbott, From Fixed Points to Chaos: Three Models of Delayed Discrimination, Prog. Neurobiol. 103, 214 (2013).
15. R. Legenstein andW. Maass, Edge of Chaos and Prediction of Computational Performance for Neural Circuit Models, Neural Netw. 20, 323 (2007).
16. W. Maass, T. Natschläger, and H. Markram, Real-Time Computing without Stable States: A New Framework for Neural Computation Based on Perturbations, Neural Comput. 14, 2531 (2002).
17. H. Jaeger and H. Haas, Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication, Science 304, 78 (2004).
18. T. Toyoizumi and L. F. Abbott, Beyond the Edge of Chaos: Amplification and Temporal Integration by Recurrent Networks in the Chaotic Regime, Phys. Rev. E 84, 051908 (2011).
19. N. Bertschinger and T. Natschlager, Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks, Neural Comput. 16, 1413 (2004).
20. A. M. Saxe, J. L. McClelland, and S. Ganguli, Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, arXiv:1312.6120.
21. A. Graves, A.-R. Mohamed, and G. Hinton, Speech Recognition with Deep Recurrent Neural Networks, in ICASSP 2013-IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, 2013) pp. 6645-6649.
22. G. Mesnil, Y. Dauphin, K. Yao, Y. Bengio, L. Deng, D. Hakkani-Tur, X. He, L. Heck, G. Tur, D. Yu, and G. Zweig, Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding, IEEE/ACM Transactions on Audio, Speech, and Language Processing 23, 530 (2015).
23. R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio, How to Construct Deep Recurrent Neural Networks, arXiv:1312.6026.
24. N. Brunel, Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons, J. Comput. Neurosci. 8, 183 (2000).
25. S. Ostojic, Two Types of Asynchronous Activity in Networks of Excitatory and Inhibitory Spiking Neurons, Nat. Neurosci. 17, 594 (2014).
26. M. Helias, T. Tetzlaff, and M. Diesmann, The Correlation Structure of Local Neuronal Networks Intrinsically Results from Recurrent Dynamics, PLoS Comput. Biol. 10, e1003428 (2014).
27. K. H. Fischer and J. A. Hertz, Spin Glasses (Cambridge University Press, Cambridge, England, 1991).
28. M. Mezard, G. Parisi, and M. A. Virasoro, Spin Glass Theory and Beyond, Lecture Notes in Physics Series (World Scientific, Singapore, 1987).
29. C. van Vreeswijk and H. Sompolinsky, Irregular Activity in Large Networks of Neurons, in Methods and Models in Neurophysics, Volume LXXX: Lecture Notes of the Les Houches Summer School, edited by C. Chow, B. Gutkin, D. Hansel, C. Meunier, and J. Dalibard (Elsevier, Amsterdam, 2005), Chap. 9, pp. 341-402.
30. T. Cabana and J. Touboul, Large Deviations, Dynamics and Phase Transitions in Large Stochastic and Disordered Neural Networks, J. Stat. Phys. 153, 211 (2013).
31. O. Faugeras, J. Touboul, and B. Cessac, A Constructive Mean-Field Analysis of Multi Population Neural Networks with Random Synaptic Weights, in Front. Syst. Neurosci. Conference Abstract: Computational and Systems Neuroscience (2009).
32. H. Sompolinsky and A. Zippelius, Relaxational Dynamics of the Edwards-Anderson Model and the Mean-Field Theory of Spin-Glasses, Phys. Rev. B 25, 6860 (1982).
33. M. A. Buice, J. D. Cowan, and C. C. Chow, Systematic Fluctuation Expansion for Neural Network Activity Equations, Neural Comput. 22, 377 (2010).
34. N. J. Priebe and D. Ferster, Inhibition, Spike Threshold, and Stimulus Selectivity in Primary Visual Cortex, Neuron 57, 482 (2008).
35. D. Hansel and C. Van Vreeswijk, How Noise Contributes to Contrast Invariance of Orientation Tuning in Cat Visual Cortex, J. Neurosci. 22, 51185 (2002).
36. A. Wolf, J. B. Swift, H. L. Swinney, and J. A. Vastan, Determining Lyapunov Exponents from a Time Series, Physica (Amsterdam) 16D, 285 (1985).
37. J. Aljadeff, M. Stern, and T. Sharpee, Transition to Chaos in Random Networks with Cell-Type-Specific Connectivity, Phys. Rev. Lett. 114, 088101 (2015).
38. G. Wainrib and J. Touboul, Topological and Dynamical Complexity of Random Neural Networks, Phys. Rev. Lett. 110, 118101 (2013).
39. D. Hansel and H. Sompolinsky, Chaos and Synchrony in a Model of a Hypercolumn in Visual Cortex, J. Comput. Neurosci. 3, 7 (1996).
40. P. Greengard, The Neurobiology of Slow Synaptic Transmission, Science 294, 1024 (2001).
41. I. Ginzburg and H. Sompolinsky, Theory of Correlations in Stochastic Neural Networks, Phys. Rev. E 50, 3171 (1994).
42. Yashar Ahmadian, Francesco Fumarola, and Kenneth D. Miller, Properties of Networks with Partially Structured and Partially Random Connectivity, Phys. Rev. E 91, 012820 (2015).
43. A. Litwin-Kumar and B. Doiron, Slow Dynamics and High Variability in Balanced Cortical Networks with Clustered Connections, Nat. Neurosci. 15, 1498 (2012).
44. P. Dayan and L. F. Abbott, Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems, Computational Neuroscience (Massachusetts Institute of Technology Press, Cambridge, MA, 2005).
45. T.W. Troyer, A. E. Krukowski, N. J. Priebe, and K. D. Miller, Contrast-Invariant Orientation Tuning in Cat Visual Cortex: Thalamocortical Input Tuning and Correlation-Based Intracortical Connectivity, J. Neurosci. 18, 5908 (1998).
46. D. J. Heeger, Normalization of Cell Responses in Cat Striate Cortex, Visual neuroscience 9, 181 (1992).
47. H. Sompolinsky and D. Hansel, Modeling Feature Selectivity in Local Cortical Circuits, in Methods in Neuronal Modeling: From Synapse to Networks, edited by C. Koch and I. Segev, 2nd ed. (MIT Press, Cambridge, MA, 1998), Chap. 13.
48. D. G. Albrecht, Visual Cortex Neurons in Monkey and Cat: Effect of Contrast on the Spatial and Temporal Phase Transfer Functions, Visual neuroscience 12, 1191 (1995).
49. H. C. Tuckwell, Introduction to Theoretical Neurobiology, Cambridge Books Online (Cambridge University Press, Cambridge, England, 1988), Vol. 1.
50. N. Brunel, V. Hakim, and M. J. E. Richardson, Single Neuron Dynamics and Computation, Curr. Opin. Neurobiol. 25, 149 (2014).
51. J. M. Rinzel and G. B. Ermentrout, Analysis of Neuronal Excitability, in Methods in Neuronal Modeling, edited by C. Koch and I. Segev (MIT Press, Cambridge, MA, 1989), pp. 135-170.
52. B. Ermentrout, Linearization of f-I Curves by Adaptation, Neural Comput. 10, 1721 (1998).
53. O. Shriki, D. Hansel, and H. Sompolinsky, Rate Models for Conductance-Based Cortical Neuronal Networks, Neural Comput. 15, 1809 (2003).
54. K. Rajan, L. F. Abbott, and H. Sompolinsky, Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks, Phys. Rev. E 82, 011903 (2010).
55. B. Ermentrout, Reduction of Conductance-Based Models with Slow Synapses to Neural Nets, Neural Comput. 6, 679 (1994).
56. E. Brown, R. Barbieri, V. Ventura, R. Kass, and L. Frank, The Time-Rescaling Theorem and Its Application to Neural Spike Train Data Analysis, Neural Comput. 14, 325 (2002).
57. R. Zillmer, N. Brunel, and D. Hansel, Very Long Transients, Irregular Firing, and Chaotic Dynamics in Networks of Randomly Connected Inhibitory Integrate-and-Fire Neurons, Phys. Rev. E 79, 031909 (2009).
58. S. Jahnke, R.-M. Memmesheimer, and M. Timme, Stable Irregular Dynamics in Complex Neural Networks, Phys. Rev. Lett. 100, 048102 (2008).
59. A. Zumdieck, M. Timme, T. Geisel, and F. Wolf, Long Chaotic Transients in Complex Networks, Phys. Rev. Lett. 93, 244103 (2004).
60. M. Monteforte and F. Wolf, Single Cell Dynamics Determine Strength of Chaos in Collective Network Dynamics, BMC Neurosci. 12, P225 (2011).
61. S. Ostojic and N. Brunel, From Spiking Neuron Models to Linear-Nonlinear Models, PLoS Comput. Biol. 7, e1001056 (2011).
62. E. S. Schaffer, S. Ostojic, and L. F. Abbott, A Complex-Valued Firing-Rate Model that Approximates the Dynamics of Spiking Networks, PLoS Comput. Biol. 9, e1003301 (2013).
63. V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome, Context-Dependent Computation by Recurrent Dynamics in Prefrontal Cortex, Nature (London) 503, 78 (2013).
64. O. Harish and D. Hansel, Asynchronous Rate Chaos in Spiking Neuronal Circuits, PLoS Comput. Biol. 11, e1004266 (2015).