Searching for Collective Behavior in a Large Network of Sensory Neurons

PLoS Computational Biology

Volumn 10, Issue 1, 2014, Pages

  Tkačik, Gašper ^a   Marre, Olivier ^b,c   Amodei, Dario ^c   Schneidman, Elad ^d   Bialek, William ^c   Berry II, Michael J ^c  

^a INSTITUTE OF SCIENCE AND TECHNOLOGY AUSTRIA   (Austria)

^b CNRS   (France)

^c PRINCETON UNIVERSITY   (United States)

^d WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); ENTROPY; ERROR CORRECTION; ISING MODEL; POPULATION STATISTICS; PROBABILITY DISTRIBUTIONS;

COLLECTIVE BEHAVIOUR; GLOBAL INTERACTION; INDEPENDENT SYSTEMS; LARGER NETWORKS; MAXIMUM ENTROPY MODELS; PAIRWISE INTERACTION; PROBABILISTIC MODELS; PROBABILITY: DISTRIBUTIONS; SENSORY NEURONS; SMALL PERTURBATIONS;

NEURONS;

ACTION POTENTIAL; AMINO ACID SUBSTITUTION; ANIMAL CELL; ANIMAL TISSUE; ARTICLE; CELL FUNCTION; CELL INTERACTION; CELL POPULATION; ENTROPY; HISTOGRAM; INTERNEURON; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; NERVE CELL NETWORK; NONHUMAN; PROBABILITY; RETINA GANGLION CELL; SALAMANDER; SENSORY NERVE CELL; THERMODYNAMICS;

ACTION POTENTIALS; ANIMALS; COMPUTATIONAL BIOLOGY; ENTROPY; FISHES; MODELS, NEUROLOGICAL; MOVEMENT; NERVE NET; PROBABILITY; RETINA; SENSORY RECEPTOR CELLS; URODELA;

EID: 84896711622     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003408     Document Type: Article

References (113)

1. Hopfield JJ, (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci (USA) 79: 2554-8.
2. Amit DJ (1989) Modeling Brain Function: The World of Attractor Neural Networks. Cambridge: Cambridge University Press.
3. Hertz J, Krogh A & Palmer RG (1991) Introduction to the Theory of Neural Computation. Redwood City: Addison Wesley.
4. Schneidman E, Berry MJ II, Segev R, (2006) Bialek W, (2006) Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440: 1007-1012.
5. Shlens J, Field GD, Gaulthier JL, Grivich MI, Petrusca D, Sher A, Litke AM, (2006) Chichilnisky EJ, (2006) The structure of multi-neuron firing patterns in primate retina. J Neurosci 26: 8254-8266.
6. Tkačik G, Schneidman E, Berry MJ II & Bialek W (2006) Ising models for networks of real neurons. arXiv.org: q-bio/0611072.
7. Yu S, Huang D, Singer W, (2008) Nikolic D, (2008) A small world of neuronal synchrony. Cereb Cortex 18: 2891-2901.
8. Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, Prieto A, Petruscam D, Grivich MI, Sher A, Hottowy P, Dabrowski W, Litke AM, (2008) Beggs JM, (2008) A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. J Neurosci 28: 505-518.
9. Tkačik G, Schneidman E, Berry MJ II & Bialek W (2009) Spin-glass models for a network of real neurons. arXiv.org: 0912.5409.
10. Shlens J, Field GD, Gaulthier JL, Greschner M, Sher A, Litke AM, (2009) Chichilnisky EJ, (2009) The structure of large-scale synchronized firing in primate retina. J Neurosci 29: 5022-5031.
11. Ohiorhenuan IE, Mechler F, Purpura KP, Schmid AM, Hu Q, (2010) Victor JD, (2010) Sparse coding and higher-order correlations in fine-scale cortical networks. Nature 466: 617-621.
12. Ganmor E, Segev R, (2011) Schniedman E, (2011) Sparse low-order interaction network underlies a highly correlated and learnable neural population code. Proc Natl Acad Sci (USA) 108: 9679-9684.
13. Vasquez JC, Marre O, Palacios AG, Berry MJ II, (2012) Cessac B, (2012) Gibbs distribution analysis of temporal correlations structure in retina ganglion cells. J Physiol Paris 3-4: 120-127.
14. Granot-Atedgi E, Tkačik G, Segev R, (2013) Schneidman E, (2013) Stimulus-dependent maximum entropy models of neural population codes. PLoS Comput Biol 9: e1002922.
15. Ganmor E, Segev R, (2011) Schneidman E, (2011) The architecture of functional interaction networks in the retina. J Neurosci 31: 3044-3054.
16. Tkačik G, Prentice JS, Victor JD, (2010) Balasubramanian V, (2010) Local statistics in natural scenes predict the saliency of synthetic textures. Proc Natl Acad Sci (USA) 107: 18149-18154.
17. Stephens GJ, Mora T, Tkačik G, (2013) Bialek W, (2013) Statistical thermodynamics of natural images. Phys Rev Lett 110: 018701.
18. Saremi S, (2013) Sejnowski TJ, (2013) Hierarchical model of natural images and the origin of scale invariance. Proc Natl Acad Sci (USA) 110: 3071-3076.
19. Lezon TR, Banavar JR, Cieplak M, Maritan A, (2006) Federoff NV, (2006) Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns. Proc Natl Acad Sci (USA) 103: 19033-19038.
20. Tkačik G, (2007) Information Flow in Biological Networks. Dissertation, Princeton University.
21. Bialek W & Ranganathan R (2007) Rediscovering the power of pairwise interactions. arXiv.org: 0712.4397.
22. Seno F, Trovato A, Banavar JR, (2008) Maritan A, (2008) Maximum entropy approach for deducing amino acid interactions in proteins. Phys Rev Lett 100: 078102.
23. Weigt M, White RA, Szurmant H, Hoch JA, (2009) Hwa T, (2009) Identification of direct residue contacts in protein-protein interaction by message passing. Proc Natl Acad Sci (USA) 106: 67-72.
24. Halabi N, Rivoire O, Leibler S, (2009) Ranganathan R, (2009) Protein sectors: Evolutionary units of three-dimensional structure. Cell 138: 774-786.
25. Mora T, Walczak AM, Bialek W, (2010) Callan CG Jr, (2010) Maximum entropy models for antibody diversity. Proc Natl Acad Sci (USA) 107: 5405-5410.
26. Marks DS, Colwell LJ, Sheridan R, Hopf TA, Pagnani A, Zecchina R, (2011) Sander C, (2011) Protein 3D structure computed from evolutionary sequence variation. PLoS One 6: e28766.
27. Sulkowska JI, Morocos F, Weigt M, Hwa T, (2012) Onuchic JN, (2012) Genomics-aided structure prediction. Proc Natl Acad Sci (USA) 109: 10340-10345.
28. Stephens GJ, (2010) Bialek W, (2010) Statistical mechanics of letters in words. Phys Rev E 81: 066119.
29. Bialek W, Cavagna A, Giardina I, Mora T, Silvestri E, Viale M, (2012) Walczak A, (2012) Statistical mechanics for natural flocks of birds. Proc Natl Acad Sci (USA) 109: 4786-4791.
30. Shemesh Y, Sztainberg Y, Forkosh O, Shlapobersky T, Chen A, (2013) Schneidman E, (2013) High-order social interactions in groups of mice. Elife 2: e00759.
31. Mora T, (2010) Bialek W, (2010) Are biological systems poised at criticality? J Stat Phys 144: 268-302.
32. Nirenberg S, (2007) Victor J, (2007) Analyzing the activity of large populations of neurons: How tractable is the problem. Curr Opin Neurobiol 17: 397-400.
33. Roudi Y, Aurell E, (2009) Hertz JA, (2009) Statistical physics of pairwise probability models. Front Comput Neurosci 3: 22.
34. Roudi Y, Nirenberg S, (2009) Latham PE, (2009) Pairwise maximum entropy models for studying large biological systems: when they can work and when they can't. PLoS Comput Biol 5: e1000380.
35. Roudi Y, Trycha J, (2009) Hertz J, (2009) The Ising model for neural data: model quality and approximate methods for extracting functional connectivity. Phys Rev E 79: 051915.
36. Marre O, Amodei D, Deshmukh N, Sadeghi K, Soo F, Holy TE, (2012) Berry MJ II, (2012) Mapping a complete neural population in the retina. J Neurosci 32: 14859-14873.
37. Broderick T, Dudik M, Tkačik, G Schapire RE & Bialek W (2007) Faster solutions of the inverse pairwise Ising problem. arXiv.org: 0712.2437.
38. Treves A, Panzeri S, Rolls ET, Booth M, (1999) Wakeman EA, (1999) Firing rate distributions and efficiency of information transmission in inferior temporal cortex neurons to natural visual stimuli. Neural Comput 11: 601-631.
39. Tkačik G, Marre O, Mora T, Amodei D, Berry MJ II, (2013) Bialek W, (2013) The simplest maximum entropy model for collective behavior in a neural network. J Stat Mech pp. P03011.
40. Okun M, Yger P, Marguet SL, Gerard-Mercier F, Benucci A, Katzner S, Busse L, Carandini M, (2012) Harris KD, (2012) Population rate dynamics and multi neuron firing patterns in sensory cortex. J Neurosci 32: 17108-17119.
41. Jaynes ET, (1957) Information theory and statistical mechanics. Phys Rev 106: 620-630.
42. Cessac B, (2013) Cofré R, (2013) Spike train statistics and Gibbs distributions. J Physiol Paris pii: S0928-4257. arXiv.org:1302.5007.
43. Rieke F, Warland D, de Ruyter van Steveninck RR & Bialek W (1997) Spikes: Exploring the Neural Code. Cambridge: MIT Press.
44. Bazaraa MS, Sherali HD & Shetty CM (2005) Nonlinear programming: Theory and algorithms. Hoboken NJ, USA: Wiley & Sons.
45. Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chichilnisky EJ, (2008) Simoncelli EP, (2008) Spatio-temporal correlations and visual signalling in a complete neuronal population. Nature 454: 995-999.
46. Fairhall AL, Burlingame CA, Narasimhan R, Harris RA, Puchalla JL, (2006) Berry MJ II, (2006) Selectivity for multiple stimulus features in retinal ganglion cells. J Neurophysiol 96: 2724-2738.
47. Keat J, Reinagel P, Reid RC, (2001) Meister M, (2001) Predicting every spike: a model for the responses of visual neurons. Neuron 30: 803-817.
48. Sadeghi KS, (2009) Progress on Deciphering the Retinal Code. Dissertation, Princeton University.
49. Tkačik G, Prentice JS, Balasubramanian V, (2010) Schneidman E, (2010) Optimal population coding by noisy spiking neurons. Proc Natl Acad Sci (USA) 107: 14419-14424.
50. Tkačik G, Granot-Atedgi E, Segev R, (2013) Schneidman E, (2013) Retinal metric: a stimulus distance measure derived from population neural responses. Phys Rev Lett 110: 058104.
51. Ganmor E (2013) Noise, structure and adaptation in neural population codes. Thesis. Weizmann Institute of Science.
52. Perkel D, (1968) Bullock T, (1968) Neural coding. Neurosci Res Program Bull 6: 221-343.
53. Ginzburg I, (1994) Sompolinsky H, (1994) Theory of correlations in stochastic neural networks. Phys Rev E 50: 3171-3191.
54. Hopfield JJ, (2008) Searching for memories, sudoku, implicit check bits, and the iterative use of not-always-correct rapid neural computation. Neural Comp 20: 1119-1164.
55. Field D, (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc A 4: 2379-2394.
56. Dong DW, (1995) Atick JJ, (1995) Statistics of natural time-varying images. Network 6: 345-358.
57. Ruderman DL, (1994) Bialek W, (1994) Statistics of natural images: scaling in the woods. Phys Rev Lett 73: 814-817.
58. Schwartz O, (2001) Simoncelli EP, (2001) Natural signal statistics and sensory gain control. Nat Neurosci 4: 819-825 (2001).
59. Bethge M & Berens P (2008) Near-maximum entropy models for binary neural representations of natural images. In: Platt J et al. eds. Adv Neural Info Proc Sys 20: 97-104. Cambridge, MA: MIT Press.
60. Stephens GJ, Mora T, Tkačik G, (2013) Bialek W, (2013) Statistical thermodynamics of natural images. Phys Rev Lett 110: 018701.
61. Mastronarde DN, (1983) Interactions between ganglion cells in cat retina. J Neurophysiol 49: 350-365.
62. DeVries SH, (1999) Correlated firing in rabbit retinal ganglion cells. J Neurophysiol 81: 908-920.
63. Brivanlou IH, Warland DK, (1998) Meister M, (1998) Mechanisms of concerted firing among retinal ganglion cells. Neuron 20: 527-539.
64. Trong PK, (2008) Rieke F, (2008) Origin of correlated activity between parasol retinal ganglion cells. Nat Neurosci 11: 1343-1351.
65. Puchalla JL, Schneidman E, Harris RA, (2005) Berry MJ II, (2005) Redundancy in the population code of the retina. Neuron 46: 493-504.
66. Segev R, Puchalla J, (2006) Berry MJ II, (2006) Functional organization of retinal ganglion cells in the salamander retina. J Neurophysiol 95: 2277-2292.
67. Schneidman E, Still S, Berry MJ II, (2003) Bialek W, (2003) Network information and connected correlations. Phys Rev Lett 91: 238701.
68. Hopfield JJ, (1985) Tank DW, (1985) "Neural" computation of decisions in optimization problems. Biol Cybern 52: 141-152.
69. Hopfield JJ, (1986) Tank DW, (1986) Computing with neural circuits: a model. Science 233: 625-633.
70. Mezard M, Parisi G & Virasoro MA (1987) Spin Glass Theory and Beyond. Singapore: World Scientific.
71. Nemenman I, Shafee F, (2001) Bialek W, (2001) Entropy and inference, revisited. Adv Neural Info Proc Syst 14: 471-478.
72. Landau DP & Binder K (2000) Monte Carlo Simulations in Statistical Physics. Cambridge, UK: Cambridge University Press.
73. Wang F, (2001) Landau DP, (2001) Efficient, multiple-range random walk algorithm to calculate the density of states. Phys Rev Lett 86: 2050-2053.
74. Cover TM & Thomas JA (1991) Elements of Information Theory. New York: Wiley.
75. Lanford OE III (1973) Entropy and equilibrium states in classical statistical mechanics. Statistical Mechanics and Mathematical Problems, Lenard A ed., pp. 1-113. Berlin: Springer Verlag.
76. Landau LD and Lifshitz EM (1977) Statistical Physics. Oxford: Pergamon.
77. Berry MJ II, Warland DK, (1997) Meister M, (1997) The structure and precision of retinal spike train. Proc Natl Acad Sci (USA) 94: 5411-5416.
78. van Hateren JH, Rüttiger L, Sun H, (2002) Lee BB, (2002) Processing of natural temporal stimuli by macaque retinal ganglion cells. J Neurosci 22: 9945-9960.
79. Shapley RM, (1978) Victor JD, (1978) The effect of contrast on the transfer properties of cat retinal ganglion cells. J Physiol 285: 275-298.
80. Victor JD, (1979) Shapley RM, (1979) Receptive field mechanisms of cat X and Y retinal ganglion cells. J Gen Physiol 74: 275-298.
81. Mante V, Bonin V, (2008) Carandini M, (2008) Functional mechanisms shaping lateral geniculate responses to artificial and natural stimuli. Neuron 58: 625-638.
82. Chen EY, Marre O, Fisher C, Schwartz G, Levy J, da Silveira RA, (2013) Berry MJ II, (2013) Alert response to motion onset in the retina. J Neurosci 33: 120-132.
83. Dan Y, Atick JJ, (1996) Reid RC, (1996) Efficient coding of natural scenes in lateral geniculate nucleus: experimental test of a computational theory. J Neurosci 16: 3351-3362.
84. Theunissen FE, Sen K, Doupe AJ, (2000) Spectral-temporal receptive fields of nonlinear auditory neurons obtained using natural sounds. J Neurosci 20: 2135-2331.
85. Stopfer M, Jayaraman V, (2003) Laurent G, (2003) Intensity versus identity coding in an olfactory system. Neuron 39: 991-1004.
86. Machens CK, Romo R, (2010) Brody CD, (2010) Functional, but not anatomical, separation of "what" and "when" in prefrontal cortex. J Neurosci 30: 350-360.
87. Churchland MM, Cunningham JP, Kaufman MT, Ryu SI, (2010) Shenoy KV, (2010) Cortical preparatory activity: representation of movement or first cog in a dynamical machine? Neuron 68: 387-400.
88. Rutishauser U, Kotowicz A, (2013) Laurent G, (2013) A method for closed-loop presentation of sensory stimuli conditional on the internal brain-state of awake animals. J Neurosci Methods 215: 139-155.
89. Segev R, Goodhouse J, Puchalla JL, (2004) Berry MJ II, (2004) Recording spikes from a large fraction of the ganglion cells in a retinal patch. Nat Neurosci 7: 1155-1162.
90. Marre O, Boustani SE, Fregnac Y, (2009) Destexhe A, (2009) Prediction of spatio-temporal patterns of neural activity from pairwise correlations. Phys Rev Lett 102: 138101.
91. Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith W, ed. Sensory communication, pp 217-234. Cambridge, USA: MIT Press.
92. Attneave F, (1954) Some informational aspects of visual perception. Psychol Rev 61: 183-193.
93. Atick JJ, (1990) Redlich AN, (1990) Towards a theory of early visual processing. Neural Comput 2: 308-320.
94. van Hdateren JH, (1992) Real and optimal neural images in early vision. Nature 360: 68-70.
95. Barlow H, (2001) Redundancy reduction revisited. Network 12: 241-253.
96. Alivisatos AP, Chun M, Church GM, Greenspan RJ, Roukes ML, (2012) Yuste R, (2012) The brain activity map project and the challenge of functional connectomics. Neuron 74: 970-974.
97. Sourlas N, (1989) Spin-glass models as error-correcting codes. Nature 339: 693-695.
98. Bohte SM, Spekreijse, (2000) Roelfsema PR, (2000) The effects of pairwise and higher-order correlations in the firing rate of a postsynaptic neuron. Neural Comput 12: 153-179.
99. Amari S-I, Nakahara H, Wu S, (2003) Sakai Y, (2003) Synchronous firing and higher-order interactions in neuron pool. Neural Comput 15: 127-142.
100. Macke JH, Opper M, (2011) Bethge M, (2011) Common input explains higher-order correlations and entropy in a simple model of neural population activity. Phys Rev Lett 106: 1-4.
101. Santos GS, Gireesh ED, Plenz D, (2010) Nakahara H, (2010) Hierarchical interaction structure of neural activities in cortical slice cultures. J Neurosci 30: 8720-8733.
102. Shimazaki H, Sadeghi K, Ikegaya Y & Toyoizumi T (2012) The simultaneous silence of neurons explains higher-order interactions in ensemble spiking activity. Cosyne 2013, Salt Lake City, USA.
103. Köster U, Sohl-Dickstein J, Gray CM & Olshausen BA (2013) Higher order correlations within cortical layers dominate functional connectivity in microcolumns. arXiv.org:1301.0050.
104. Yu S, Yang H, Nakahara H, Santos GS, Nikolič D, (2011) Plenz D, (2011) Higher-order interactions characterized in cortical activity. J Neurosci 31: 17514-17526.
105. Amodei D, Schwartz G & Berry MJ II (2008) Correlations and the structure of the population code in a dense patch of the retina. In: Stett A ed. Proceedings MEA Meeting. BIOPRO Baden-Württemberg, 2008. pp. 197.
106. Dudik M, Phillips SJ & Schapire RE (2004) Performance guarantees for regularized maximum entropy density estimation. Proceedings 17th Annual conference on learning theory.
107. Ferrenberg AM, (1988) Swendsen RH, (1988) New Monte Carlo technique for studying phase transitions. Phys Rev Lett 61: 2635-2638.
108. Berry MJ II, Tkačik G, Dubuis J, Marre O, Azerado da Silveira R, (2013) A simple method for estimating the entropy of neural activity. J Stat Mech pp. P03015.
109. Cocco S, Leibler S, (2009) Monasson R, (2009) Neuronal couplings between retinal ganglion cells inferred by efficient inverse statistical physics methods. Proc Natl Acad Sci (USA) 106: 14058-14062.
110. Cocco S, (2011) Monasson R, (2011) Adaptive cluster expansion for inferring Boltzmann machines with noisy data. Phys Rev Lett 106: 090601.
111. Azhar F, (2008) An Information Theoretic Study of Neural Populations. Dissertation. University of California at Santa Barbara.
112. Azhar F & Bialek W (2010) When are correlations strong? arXiv.org: 1012.5987.
113. Sessak V, (2009) Monasson R, (2009) Small-correlation expansions for the inverse Ising problem. J Phys A 42: 055001.