Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks

Neural Computation

Volumn 25, Issue 3, 2013, Pages 626-649

  Sussillo, David ^a   Barak, Omri ^b  

^a Stanford University   (United States)

^b Department of Neurology and the Taub Institute for Research on Alzheimer's Disease and the Aging Brain   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK;

ARTIFICIAL INTELLIGENCE; NEURAL NETWORKS (COMPUTER);

EID: 84877827546     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00409     Document Type: Article

References (23)

1. Ahrens,M. B., Li, J. M., Orger, M. B., Robson, D. N., Schier, A. F., et al. (2012). Brainwide neuronal dynamics duringmotor adaptation in zebrafish. Nature, 485(7399), 471-477.
2. Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157-166.
3. Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press.
4. Hopfield, J. J. (1982). Neural networks and physical systemswith emergent collective computational abilities. PNAS, 79(8), 2554-2558.
5. Hopfield, J. J., & Tank, D. (1986). Computing with neural circuits: A model. Science, 233(4764), 625-633.
6. Huerta, R., & Rabinovich, M. (2004). Reproducible sequence generation in random neural ensembles. Physical Review Letters, 93(23), 238104.
7. Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304(5667), 78-80.
8. Jones, L. M., Fontanini, A., Sadacca, B. F., Miller, P., & Katz, D. B. (2007). Natural stimuli evoke dynamic sequences of states in sensory cortical ensembles. PNAS, 104(47), 18772-18777.
9. Liu, J. K., & Buonomano, D. V. (2009). Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neuroscience, 29(42), 13172-13181.
10. Maass, W., Joshi, P., & Sontag, E. D. (2007). Computational aspects of feedback in neural circuits. PLoS Comput. Biol., 3(1), e165.
11. Maass, W., Natschläger, T., & Markram, H. (2002). Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation, 14(11), 2531-2560.
12. Machens, C. K., Romo, R.,&Brody, C. D. (2005). Flexible control of mutual inhibition: A neural model of two-interval discrimination. Science, 307(5712), 1121-1124.
13. Martens, J. (2010). Deep learning via Hessian-free optimization. In Proceedings of the 27th International Conference on Machine Learning. Madison, WI:Omnipress.
14. Martens, J., & Sutskever, I. (2011). Learning recurrent neural networks with hessianfree optimization. In Proceedings of the 28th International Conference on Machine Learning. Madison, WI: Omnipress.
15. Ott, E. (2002). Chaos in dynamical systems. Cambridge: Cambridge University Press.
16. Ponzi, A., & Wickens, J. (2010). Sequentially switching cell assemblies in random inhibitory networks of spiking neurons in the striatum. Journal of Neuroscience, 30(17), 5894-5911.
17. Rabinovich, M. I., Huerta, R., Varona, P., & Afraimovich, V. S. (2008). Transient cognitive dynamics, metastability, and decision making. PLoS Comput. Biol., 4(5), e1000072.
18. Romo R., Brody, C. D., Hernández, A., & Lemus, L. (1999). Neuronal correlates of parametric working memory in the prefrontal cortex. Nature, 399(6735), 470-473.
19. Rumelhart, D., Hinton, G., & Williams, R. (1985). Learning internal representations by error propagation. Parallel Distributed Processing, 1, 319-362.
20. Stopfer, M., Jayaraman, V., & Laurent, G. (2003). Intensity versus identity coding in an olfactory system. Neuron, 39(6), 991-1004.
21. Strogatz, S.H. (1994). Nonlinear dynamics and chaos, with applications to physics, biology, chemistry, and engineering. Boulder, CO: Westview Press.
22. Sussillo, D., & Abbott, L. F. (2009). Generating coherent patterns of activity from chaotic neural networks. Neuron, 63(4), 544-557.
23. Wang, X. J. (2008). Decision making in recurrent neuronal circuits. Neuron, 60(2), 215-234.