Synergy, Redundancy, and Independence in Population Codes

Journal of Neuroscience

Volumn 23, Issue 37, 2003, Pages 11539-11553

  Schneidman, Elad ^a   Bialek, William ^a   Berry II, Michael J ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

Decoding; Encoding; Information theory; Neural code; Noise correlation; Signal correlation

Indexed keywords

ANIMAL BEHAVIOR; ARTICLE; ENTROPY; INFORMATION PROCESSING; MATHEMATICAL MODEL; MOTOR ACTIVITY; NERVE CELL; NOISE; PRIORITY JOURNAL; SENSORY STIMULATION; STIMULUS RESPONSE;

EID: 0347064157     PISSN: 02706474     EISSN: None     Source Type: Journal    
DOI: 10.1523/jneurosci.23-37-11539.2003     Document Type: Article

References (84)

1. Abbott L, Dayan P (1999) The effect of correlated variability on the accuracy of a population code. Neural Comput 11:91-101.
2. Abeles M, Lass Y (1975) Transmission of information by the axon: II. The channel capacity. Biol Cybern 19:121-125.
3. Abeles M, Bergman H, Margalit E, Vaadia E (1993) Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J Neurophysiol 70:1629-1638.
4. Aertsen A, Gerstein G, Habib M, Palm G (1989) Dynamics of neuronal firing correlation: modulation of "effective connectivity." J Neurophysiol 61:900-917.
5. Alonso J, Usrey W, Reid R (1996) Precisely correlated firing in cells of the lateral geniculate nucleus. Nature 383:815-819.
6. Atick J (1992) Could information theory provide an ecological theory of sensory processing? Netw Comput Neural Syst 3:213-251.
7. Attneave F (1954) Some informational aspects of visual perception. Psychol Rev 61:183-193.
8. Bair W, Koch C (1996) Temporal precision of spike trains in extrastriate cortex of the behaving macaque monkey. Neural Comp 8:1185-1202.
9. Barlow H (1961) Possible principles underlying the transformation of sensory messages. In: Sensory communication (Rosenblith WA, ed). Cambridge, MA: MIT.
10. Barlow H (2001) Redundancy reduction revisited. Network: Comput Neural Syst 12:241-253.
11. Bell A, Sejnowski T (1997) The "independent components" of natural scenes are edge filters. Vision Res 37:3327-3338.
12. Berry M, Meister M (1998) Refractoriness and neural precision. J Neurosci 16:2381-2396.
13. Bezzi M, Diamond M, Treves A (2002) Redundancy and synergy arising from pairwise correlations in neuronal ensembles. J Comput Neurosci 12:165-174.
14. Bialek W, Rieke F, de Ruyter van Steveninck R, Warland D (1991) Reading a neural code. Science 252:1854-1857.
15. Brenner N, Strong S, Koberle R, Bialek W, de Ruyter van Steveninck R (2000) Synergy in a neural code. Neural Comput 12:1531-1552.
16. Brody C (1999) Correlations without synchrony. Neural Comput 11:1537-1551.
17. Brown E, Frank L, Tang D, Quirk M, Wilson M (1998) A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J Neurosci 18:7411-7425.
18. Brunel N, Nadal J (1998) Mutual information, Fisher information, and population coding. Neural Comput 10:1731-1757.
19. Buracas G, Zador A, DeWeese M, Albright T (1998) Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20:959-969.
20. Chechik G, Globerson A, Anderson M, Young E, Nelken I, Tishby N (2002) Group redundancy measures reveal redundancy reduction along the auditory-pathway. In: Advances in neural information processing systems 14 (Dietterich TG, Becker S, Ghahramani Z, eds), pp 173-180. Cambridge, MA: MIT.
21. Cover T, Thomas J (1991) Elements of information theory. New York: Wiley Interscience.
22. Dan Y, Alonso J, Usrey W, Reid R (1998) Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nat Neurosci 1:501-507.
23. Dawid A (1979) Conditional independence in statistical theory. J R Star Soc B 41:1-31.
24. Dayan P, Abbott L (2001) Theoretical neuroscience. Cambridge, MA: MIT.
25. de Ruyter van Stevenick R, Bialek W (1988) Real-time performance of a movement sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc R Soc Lond B Biol Sci 234:379-414.
26. DeWeese M (1995) Optimization principles for the neural code. PhD thesis, Princeton University.
27. Eckhorn R, Popel B (1974) Rigorous and extended application of information theory to the afferent visual system of the cat: I. Basic concepts. Kybernetik 16:191-200.
28. Fitzhugh R (1957) The statistical detection of threshold signals in the retina. J Gen Physiol 40:925-948.
29. Gat I, Tishby N (1999) Synergy and redundancy among brain cells of behaving monkeys. In: Advances in neural processing systems 11 (Kearns M, Solla S, Cohn D, eds), pp 465-471. Cambridge, MA: MIT.
30. Gawne T, Richmond B (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? J Neurosci 13:2758-2771.
31. Georgopoulos A, Schwartz A, Kettner R (1986) Neuronal population coding of movement direction. Science 233:1416-1419.
32. Gray C, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proc Natl Acad Sci USA 86:1698-1702.
33. Hatsopoulos N, Ojakangas C, Paninski L, Donoghue I (1998) Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc Natl Acad Sci USA 95:15706-15711.
34. Hogg R, Craig A (1995) Introduction to mathematical statistics Ed 5. Englewood Cliffs, NJ: Prentice-Hall.
35. Hyvarinen A, Hoyer P (2001) A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images. Vision Res 41:2413-2423.
36. Johnson D, Gruner C, Baggrely K, Sesahgiri C (2001) Information-theoretic analysis of neural coding. J Comp Neurosci 16:2381-2396.
37. Kang K, Sompolinsky H (2001) Mutual information of population codes and distance measures in probability space. Phys Rev Lett 86:4958-4961.
38. Krahe R, Kreiman G, Gabbiani F, Koch C, Metzner W (2002) Stimulus encoding and feature extraction by multiple sensory neurons. J Neurosci 22:2374-2848.
39. Laurent G, Davidowitz H (1994) Encoding of olfactory information with oscillating neural assemblies. Science 265:1872-1875.
40. Lewen G, Bialek W, de Ruyter van Steveninck R (2001) Neural coding of naturalistic motion stimuli. Network: Comput Neural Syst 12:317-329.
41. Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37:145-151.
42. Mackay D, McCulloch W (1952) The limiting information capacity of a neuronal link. Bull Math Biophys 14:127-135.
43. Mainen Z, Sejnowski T (1995) Reliability of spike timing in neocortical neurons. Science 268:1503-1508.
44. Martignon L, Deco G, Laskey K, Diamond M, Freiwald W, Vaadia E (2000) Neural coding: higher-order temporal patterns in the neurostatistics of cell assemblies. Neural Comput 16:2381-2396.
45. Mastronarde D (1983) Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. J Neurophysiol 49:303-324.
46. Meister M, Hosoya T (2001) Are retinal ganglion cells independent encoders? Preprint. See also http://rhino.harvard.edu/Publications/Meister_2001_Encoders.pdf.
47. Meister M, Legando L, Baylor D (1995) Concerted signaling by retinal ganglion cells. Science 270:1207-1210.
48. Nemenman I, Bialek W, de Ruyter van Steveninck R (2003) Entropy and information in neural spike trains: progress on the sampling problem. arXiv:physics/0306063.
49. Nirenberg S, Latham P (2003) Decoding neuronal spike trains: how important are correlations? Proc Natl Acad Sci USA 100:7348-7353.
50. Nirenberg S, Carcieri S, Jacobs A, Latham P (2001) Retinal ganglion cells act largely as independent encoders. Nature 411:698-701.
51. Optican L, Richmond B (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex. III. Information theoretic analysis. J Neurophysiol 57:162-178.
52. Palm G, Aertsen A, Gerstein G (1988) On the significance of correlations among neuronal spike trains. Biol Cybern 59:1-11.
53. Paninski L (2003) Estimation of entropy and mutual information. Neural Comput 15:1191-1253.
54. Panzeri S, Schultz S (2001) A unified approach to the study of temporal, correlational, and rate coding. Neural Comput 13:1311-1349.
55. Panzeri S, Schultz S, Treves A, Rolls E (1999) Correlations and the encoding of information in the nervous system. Proc R Soc Lond B 266:1001-1012.
56. Perkel D, Bullock T (1968) Neural coding. Neurosci Res Program Bull 6:221-343.
57. Perkel D, Gerstein G, Moore G (1967) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys J 7:419-440.
58. Petersen R, Panzeri S, Diamond M (2001) Population coding of stimulus location in rat somatosensory cortex. Neuron 32:503-514.
59. Pola G, Thiele A, Hoffmann K-P, Panzeri S (2003) An exact method to quantify the information transmitted by different mechanisms of correlational coding. Network: Comput Neural Syst 14:35-60.
60. Reich D, Mechler K, Purpura F, Victor J (2000) Interspike intervals, receptive fields, and information encoding in primary visual cortex. J Neurosci 20:1964-1974.
61. Reinagel P, Reid R (2000) Temporal coding of visual information in the thalamus. J Neurosci 20:5392-5400.
62. Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1997) Spikes: exploring the neural code. Cambridge, MA: MIT.
63. Ruderman D (1994) The statistics of natural images. Network: Comput Neural Syst 5:517-548.
64. Salinas E, Abbott L (1994) Vector reconstruction from firing rates. J Comput Neurosci 1:89-107.
65. Samengo I (2002) Information loss in an optimal maximum likelihood decoding. Neural Comput 14:771-779.
66. Schneidman E, Brenner N, Tishby N, de Ruyter van Steveninck R, Bialek W (2001) Universality and individuality in a neural code. Adv Neural Inf Process Syst 13:159-165.
67. Schneidman E, Bialek W, Berry M (2003) An information theoretic approach to the functional classification of neurons. In: Advances in neural information processing systems 15 (Becker S, Thrun S, Obermayer K, eds), pp 197-204. Cambridge, MA: MIT.
68. Schneidman E, Still S, Berry M, Bialek W (2003) Network information and connected correlations. Phys Rev Lett, in press.
69. Schwartz O, Simoncelli E (2001) Natural signal statistics and sensory gain control. Nat Neurosci 4:819-825.
70. Seung H, Sompolinsky H (1993) Simple models for reading neuronal population codes. Proc Natl Acad Sci USA 90:10749-10753.
71. Shannon C, Weaver W (1949) The mathematical theory of communication. Urbana, IL: University of Illinois.
72. Sompolinsky H, Yoon H, Kang K, Shamir M (2001) Population coding in neuronal systems with correlated noise. Phys Rev E 64:8095-8100.
73. Srinivasan M, Lauglin S, Dubs A (1982) Predictive coding: a fresh view of inhibition in the retina. Proc R Soc Lond B 216:427-459.
74. Strong S, Koberle R, de Ruyter van Steveninck R, Bialek W (1998) Entropy and information in neural spike trains. Phys Rev Lett 80:197-200.
75. Treves A, Panzeri S (1995) The upward bias in measures of information derived from limited data samples. Neural Comput 7:399-407.
76. Vaadia E, Haalman I, Abeles M, Bergman H, Prut Y, Slovin H, Aertsen A (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature 373:515-518.
77. van Hateren J (1992) Real and optimal neural images in early vision. Nature 360:68-70.
78. Verveen A, Derksen H (1968) Fluctuation phenomena in nerve membrane. Proc IEEE 56:906-916.
79. Victor J (2002) Binless strategies for estimation of information from neural data. Phys Rev E 66:051903.
80. Warland D, Reinagel P, Meister M (1997) Decoding visual information from a population of retinal ganglion. J Neurophysiol 78:2336-2350.
81. Wu S, Chen D, Nirajan M, Amari S (2003) Sequential bayesian decoding with a population of neurons. Neural Comput 15:993-1012.
82. Zemel R, Dayan P, Pouget A (1998) Probabilistic interpretation of population codes. Neural Comput 10:403-430.
83. Zhang K, Ginzburg I, McNaughton B, Sejnowski T (1998) Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. J Neurophysiol 79:1017-1044.
84. Zohary E, Shadlen M, Newsome W (1994) Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370:140-143.