Fast sampling-based inference in balanced neuronal networks

Advances in Neural Information Processing Systems

Volumn 3, Issue January, 2014, Pages 2240-2248

  Hennequin, Guillaume ^a   Aitchison, Laurence ^b   Lengyel, Maté ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; CHAINS; DECISION MAKING; INFORMATION SCIENCE; MARKOV PROCESSES; NEURAL NETWORKS; NEURONS; PROBABILITY DISTRIBUTIONS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

LATENT VARIABLE MODELING; MARKOV CHAIN MONTE CARLO ALGORITHMS; POSTERIOR DISTRIBUTIONS; PROBABILISTIC INFERENCE; PROBABILISTIC REPRESENTATION; SELECTIVE AMPLIFICATION; STATIONARY DISTRIBUTION; STOCHASTIC NEURAL NETWORK;

INFERENCE ENGINES;

EID: 84937937938     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. S. Thorpe, D. Fize, and C. Marlot. Speed of processing in the human visual system. Nature, 381:520-522, 1996.
2. D. MacKay. Information theory, inference, and learning algorithms. Cambridge University Press, 2003.
3. D. Knill and A. Pouget. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends in Neurosciences, 27:712-719, 2004.
4. J. Fiser, P. Berkes, G. Orbán, and M. Lengyel. Statistically optimal perception and learning: from behavior to neural representations. Trends in Cognitive Sciences, 14:119-130, 2010.
5. P. Berkes, G. Orbán, M. Lengyel, and J. Fiser. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science, 331:83-87, 2011.
6. R. Moreno-Bote, D. C. Knill, and A. Pouget. Bayesian sampling in visual perception. Proceedings of the National Academy of Sciences, 108:12491-12496, 2011.
7. L. N. Trefethen and M. Embree. Spectra and pseudospectra: the behavior of nonnormal matrices and operators. Princeton University Press, 2005.
8. G. Hennequin, T. P. Vogels, and W. Gerstner. Non-normal amplification in random balanced neuronal networks. Physical Review E, 86:011909 2012.
9. B. K. Murphy and K. D. Miller. Balanced amplification: A new mechanism of selective amplification of neural activity patterns. Neuron, 61:635-648, 2009.
10. C. W. Gardiner. Handbook of stochastic methods: for physics, chemistry, and the natural sciences. Berlin: Springer, 1985.
11. R. Neal. MCMC using Hamiltonian dynamics. Handbook of MCMC, pages 113-162, 2011.
12. M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the International Conference on Machine Learning, 2011.
13. A. Grabska-Barwinska, J. Beck, A. Pouget, and P. Latham. Demixing odors - fast inference in olfaction. In C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 26, pages 1968-1976. Curran Associates, Inc., 2013.
14. Mark Rudelson and Roman Vershynin. Smallest singular value of a random rectangular matrix. Communications on Pure and Applied Mathematics, 62:1707-1739, 2009.
15. M. Girolami and B. Calderhead. Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73:123-214, 2011.
16. J. Martin, L. C. Wilcox, C. Burstedde, and O. Ghattas. A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion. SIAM Journal on Scientific Computing, 34:A1460-A1487, 2012.
17. M. Mezard and A. Montanari. Information, physics, and computation. Oxford University Press, 2009.
18. G. E. Hinton and T. J. Sejnowski. Learning and relearning in Boltzmann machines. In D E Rumelhart, J L McClelland, and the PDP Research Group, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations, chapter 7, pages 282-317. MIT Press, Cambridge, MA, 1986.
19. G E Hinton, P Dayan, B J Frey, and R M Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):1158-1161, 1995.
20. L. Buesing, J. Bill, B. Nessler, and W. Maass. Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons. PLoS Computational Biology, 7:1-22, 2011.
21. J. Vanbiervliet, B. Vandereycken, W. Michiels, S. Vandewalle, and M. Diehl. The smoothed spectral abscissa for robust stability optimization. SIAM Journal on Optimization, 20:156-171, 2009.
22. G. Hennequin, T. P. Vogels, and W. Gerstner. Optimal control of transient dynamics in balanced networks supports generation of complex movements. Neuron, 82, 2014.
23. W. Keith Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57:97-109, 1970.
24. Akihisa Ichiki and Masayuki Ohzeki. Violation of detailed balance accelerates relaxation. Physical Review E, 88:020101 2013.
25. Y. Sun, J. Schmidhuber, and F. J. Gomez. Improving the asymptotic performance of Markov Chain Monte-Carlo by inserting vortices. In J.D. Lafferty, C.K.I. Williams, J. Shawe-Taylor, R.S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 2235-2243. 2010.
26. K. S. Turitsyn, M. Chertkov, and M. Vucelja. Irreversible Monte Carlo algorithms for efficient sampling. Physica D: Nonlinear Phenomena, 240:410-414, 2011.
27. Joris Bierkens. Non-reversible Metropolis-Hastings. arXiv:1401.8087 [math], 2014.