Approaches to Information-Theoretic Analysis of Neural Activity

Biological Theory

Volumn 1, Issue 3, 2006, Pages 302-316

  Victor, Jonathan D ^a  

^a WEILL CORNELL MEDICAL COLLEGE   (United States)

Author keywords

direct method; information theory; metric space method; neural coding; sensory; spike trains; systems

Indexed keywords

EID: 38649136181     PISSN: 15555542     EISSN: 15555550     Source Type: Journal    
DOI: 10.1162/biot.2006.1.3.302     Document Type: Article

References (115)

1. Abbott LF, Varela JA, Sen K, Nelson SB (1997) Synaptic depression and cortical gain control. Science 275: 220–224.
2. Abeles M, Prut Y (1996) Spatio-temporal firing patterns in the frontal cortex of behaving monkeys. Journal of Physiology Paris 90: 249–250.
3. Amari S-I (2001) Information geometry on hierarchy of probability distributions. IEEE Transaction Information Theory 47: 1701–1711.
4. Amigo JM, Szczepanski J, Wajnryb E, Sanchez-Vives M (2004) Estimating the entropy rate of spike trains via Lempel-Ziv complexity. Neural Computation 16: 717–736.
5. Aronov D (2003) Fast algorithm for the metric-space analysis of simultaneous responses of multiple single neurons. Journal of Neuroscientific Methods 124: 175–179.
6. Aronov D, Reich DS, Mechler F, Victor JD (2001) Multidimensional representation of spatial phase in V1. Investigations in Ophthalmology and Vision Science 42: 405.
7. Baker SN, Lemon RN (2000) Precise spatiotemporal repeating patterns in monkey primary and supplementary motor areas occur at chance levels. Journal of Neurophysiology 84: 1770–1780.
8. Berry MJ, Warland DK, Meister M (1997) The structure and precision of retinal spike trains. Proceedings of the National Academy of Sciences USA 94: 5411–5416.
9. Bialek W, Rieke F, de Ruyter van Steveninck RR, Warland D (1991) Reading a neural code. Science 252: 1854–1857.
10. Bliss TV, Collingridge GL (1993) A synaptic model of memory: long-term potentiation in the hippocampus. Nature 361: 31–39.
11. Bourne H, Nicoll R (1993). Molecular machines integrate coincident synaptic signals. Cell 72/Neuron 10 (Suppl), 65–85.
12. Buracas GT, Zador AM, DeWeese MR, Albright TD (1998). Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20: 959–969.
13. Carlton AG (1969). On the bias of information estimates. Psychological Bulletin 71: 108–109.
14. Chao A, Shen TJ (2003) Nonparametric estimate of Shannon’s index of diversity when there are unseen species in a sample. Environmental and Ecological Statistics 10: 429–443.
15. Chee-Orts MN, Optican LM (1993) Clustermethod for analysis of transmitted information in multivariate neuronal data. Biological Cybernetics 69: 29–35.
16. Cline H (1997) Coincidence detection in the nervous system. Trends in Neurosciences 19: 566–567.
17. Dan Y, Alonso JM, Usrey WM, Reid RC (1998) Coding of visual information by precisely correlated spikes in the lateral geniculate nucleus. Nature Neuroscience 1: 501–507.
18. de Ruyter van Steveninck RR, Lewen GD, Strong SP, Koberle R, Bialek W (1997). Reproducibility and variability in neural spike trains. Science 275: 1805–1808.
19. Di Lorenzo PM, Victor JD (2003) Taste response variability and temporal coding in the nucleus of the solitary tract of the rat. Journal of Neurophysiology 90: 1418–1431.
20. Efron B (1982) The Jackknife, the Bootstrap and Other Resampling Plans.Philadelphia: SIAM.
21. Efron B, Tibshirani RJ (1998) An Introduction to the Bootstrap. Monographs on Statistics and Applied Probability, Vol. 57, 436. Boca Raton, FL: Chapman and Hall/CRC Press.
22. Farach M, Noordewier M, Savari S, Shepp L, Wyner A, Ziv J (1995) Ontheentropy of DNA: Algorithms and measurements based on memory and rapid convergence. Proceedings of the 6th Annual ACM-SIAM Symposium on Discrete Algorithms, 48–57.
23. Gawne TJ (2000) The simultaneous coding of orientation and contrast in the responses of V1 complex cells. Experimental Brain Research 133: 293–302.
24. Grassberger P (1988) Finite sample corrections to entropy and dimension estimates. Physics Letters A 128: 369–373.
25. Grassberger P, Procaccia I (1983) Measuring the strangeness of strange attractors. Physica D 9: 189–208.
26. Gray CM, Singer W (1989) Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences the USA 86: 1698–1702.
27. Hirata Y, Mees AI (2003) Estimating topological entropy via a symbolic data compression technique. Physical Review E 67(2 Pt 2): 026205.
28. Jarvis MR, Mitra PP (2001) Sampling properties of the spectrum and coherency of sequences of action potentials. Neural Computation 13: 717–749.
29. Johnson DH, Gruner CM, Baggerly K, Seshagiri C (2001) Information-theoretic analysis of neural coding. Journal of Computational Neuroscience 10: 47–69.
30. Kennel MB, Shlens J, Abarbanel HD, Chichilnisky EJ (2005) Estimating entropy rates with Bayesian confidence intervals. Neural Computation 17: 1531–1576.
31. Kontoyiannis I, Algoet PH, Suhov YM, Wyner AJ (1998) Nonparametric entropy estimation for stationary processes and random fields, with applications to English text. IEEE Transactions on Information Theory 44: 1319–1327.
32. Kozachenko LF, Leonenko NN (1987) Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii 23: 9–16.
33. Kraskov A, Stogbauer H, Grassberger P (2004) Estimating mutual information. Physical Review E 69(6 Pt 2): 066138.
34. Kreiman G, Krahe R, Metzner W, Koch C, Gabbiani F (2000) Robustness and variability of neuronal coding by amplitude-sensitive afferents in the weakly electric fish eigenmannia. Journal of Neurophysiology 84: 189–204.
35. Krichevsky R, Trofimov V (1981) The performance of universal coding. IEEE Transactions on Information Theory 27: 199–207.
36. Lestienne R, Tuckwell HC (1997) The significance of precisely replicating patterns in mammalian CNS spike trains. Neuroscience 82: 315–336.
37. Levy WB (2000) Experiences, thoughts, and conjectures on implementing a Lempel-Ziv-type algorithm to measure information in a spike train. In: Neural Information Processing Systems Workshop on Information and Statistical Structure in Spike Trains. Breckenridge, CO December 1–2, 2000.
38. London M, Schreibman A, Hausser M, Larkum ME, Segev I (2002) The information efficacy of a synapse. Nature Neuroscience 5: 332–340.
39. Luck SJ, Chelazzi L, Hillyard SA, Desimone R (1997) Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. Journal of Neurophysiology 77: 24–42.
40. Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275: 213–215.
41. Marmarelis PZ, Marmarelis VZ (1978) Analysis of physiological systems: The white-noise approach. In: Computers in Biology and Medicine, 487. New York: Plenum.
42. McClurkin JW, Optican LM, Richmond BJ, Gawne TJ (1991) Concurrent processing and complexity of temporally encoded neuronal messages in visual perception. Science 253: 675–677.
43. McFadden JA (1965) The entropy of a point process. Journal of the Society for Industrial and Applied Mathematics 13: 988–994.
44. Meister M, Lagnado L, Baylor DA (1995) Concerted signaling by retinal ganglion cells. Science 270: 1207–1210.
45. Mel BW (1993) Synaptic integration in an excitable dendritic tree. Journal of Neurophysiology 70: 1086–1101.
46. Middlebrooks JC, Clock AE, Xu L, Green DM (1994) A panoramic code for sound location by cortical neurons. Science 264: 842–844.
47. Miller GA (1955) Note on the bias on information estimates. In: Information Theory in Psychology: Problems and Methods (Quastler H, ed), II-B, Glencoe, IL: Free Press, 95–100.
48. Mitra PP, Pesaran B (1999) Analysis of dynamic brain imaging data. Biophysical Journal 76: 691–708.
49. Nakahara H, Amari S (2002) Information-geometric measure for neural spikes. Neural Computation 14: 2269–2316.
50. Nemenman I, Bialek W, de Ruyter van Steveninck R (2004) Entropy and information in neural spike trains: Progress on the sampling problem. Physical Review E 69 (5 Pt 2): 056111.
51. Nirenberg S, Carcieri SM, Jacobs AL, Latham PE (2001) Retinal ganglion cells act largely as independent encoders. Nature 411: 698–701.
52. Nirenberg S, Jacobs A, Fridman G, Latham P, Douglas R, Alam N, Prusky G (2006) Ruling out and ruling in neural codes. Journal of Vision 6: 889a.
53. Nirenberg S, Latham PE (2003) Decoding neuronal spike trains: how important are correlations? Proceedings of the National Academy of Sciences USA 100: 7348–7353.
54. Optican LM, Richmond BJ (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex: III. Information theoretic analysis. Journal of Neurophysiology 57: 162–178.
55. Oram MW, Wiener MC, Lestienne R, Richmond BJ (1999). Stochastic nature of precisely timed spike patterns in visual system neuronal responses. Journal of Neurophysiology 81: 3021–3033.
56. Paninski L (2003) Estimation of entropy and mutual information. Neural Computation 15: 1191.
57. Panzeri S, Petersen RS, Schultz SR, Lebedev M, Diamond ME (2001) The role of spike timing in the coding of stimulus location in rat somatosensory cortex. Neuron 29: 769–777.
58. Panzeri S, Schultz SR (2001) A unified approach to the study of temporal, correlational, and rate coding. Neural Computation 13: 1311–1349.
59. Panzeri S, Schultz SR, Treves A, Rolls ET (1999) Correlations and the encoding of information in the nervous system. Proceedings of the Royal Society of London B 266: 1001–1012.
60. Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophysical Journal 7: 419–440.
61. Petersen RS, Panzeri S, Diamond ME (2001) Population coding of stimulus location in rat somatosensory cortex. Neuron 32: 503–514.
62. Pfurtscheller G, Andrew C (1999) Event-related changes of band power and coherence: methodology and interpretation. Journal of Clinical Neurophysiology 16: 512–519.
63. Quiroga RQ, Rosso OA, Basar E, Schurmann M (2001) Wavelet entropy in event-related potentials: A new method shows ordering of EEG oscillations. Biological Cybernetics 84: 291–299.
64. Rapp PE, Zimmerman ID, Vining EP, Cohen N, Albano AM, Jimenez-Montano MA (1994) The algorithmic complexity of neural spike trains increases during focal seizures. Journal of Neuroscience 14: 4731–4739.
65. Reich DS, Mechler F, Victor JD (2001a) Formal and attribute-specific information in primary visual cortex. Journal of Neurophysiology 85: 305–318.
66. Reich DS, Mechler F, Victor JD (2001b) Independent and redundant information in nearby cortical neurons. Science 294: 2566–2568.
67. Reich DS, Mechler F, Victor JD (2001c) Temporal coding of contrast in primary visual cortex: When, what, and why. Journal of Neurophysiology 85: 1039–1041.
68. Reinagel P, Reid RC (2000) Temporal coding of visual information in the thalamus. Journal of Neuroscience 20: 5392–5400.
69. Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4 neurons. Neuron 26: 703–714.
70. Richmond BJ, Optican LM (1987) Temporal encoding of two-dimensional patterns by single units in primate inferior temporal cortex: II. Quantification of response waveform. Journal of Neurophysiology 57: 147–161.
71. Rieke F, Warland D, de Ruyter van Steveninck R, Bialek W (1997) Spikes: Exploring the Neural Code. Cambridge, MA: MIT Press.
72. Rissanen J (1989) Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific.
73. Rodriguez E, George N, Lachaux JP, Martinerie J, Renault B, Varela FJ (1999) Perception’s shadow: long-distance synchronization of human brain activity. Nature 397: 430–433.
74. Roelfsema PR, Lamme VA, Spekreijse H (2004) Synchrony and covariation of firing rates in the primary visual cortex during contour grouping. Nature Neuroscience 7: 982–991.
75. Salzman CD, Newsome WT (1994) Neural mechanisms for forming a perceptual decision. Science 264: 231–237.
76. Samonds JM, Bonds AB (2004) From another angle: Differences in cortical coding between fine and coarse discrimination of orientation. Journal of Neurophysiology 91: 1193–1202.
77. Samonds JM, Zhou Z, Bernard MR, Bonds AB (2006) Synchronous activity in cat visual cortex encodes collinear and cocircular contours. Journal of Neurophysiology 95: 2602–2616.
78. Schiff ND, Kalik SF, Purpura KP (2000) Episodic dynamics of cortical processing in the ventral stream during free-viewing: Analysis of local field potentials in striate/extrastriate and inferotemporal cortices. Society for Neuroscience Abstracts 26: 1199.
79. Schiff ND, Kalik SF, Purpura KP (2001) Sustained activity in the central thalamus and extrastriate areas during attentive visuomotor behavior: Correlation of single unit activity and local field potentials. Society for Neuroscience Abstracts 27: 1910.
80. Schiff SJ, Aldroubi A, Unser M, Sato S (1994) Fast wavelet transformation of EEG. Electroencephalography and Clinical Neurophysiology 91: 442–455.
81. Schiff SJ, So P, Chang T, Burke RE, Sauer T (1996) Detecting dynamical interdependence and generalized synchrony through mutual prediction in a neural ensemble. Physical Review E 54: 6708–6724.
82. Schlens J, Kennel M, Abarbanel H, Chichilnisky EJ (2006) Estimating information rates in neural spike trans with confidence intervals. Neural Computation.
83. Schneidman E, Berry MJ II, Segev R, Bialek W (2006) Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440: 1007–1012.
84. Schultz SR, Panzeri S (2001) Temporal correlations and neural spike train entropy. Physical Review Letters 86: 5823–5826.
85. Schurmann T, Grassberger P (1996) Entropy estimation of symbol sequences. Chaos 6: 414–427.
86. Sellers P (1974) On the theory and computation of evolutionary distances. SIAM Journal of Applied Mathematics 26: 787–793.
87. Sen K, Jorge-Rivera JC, Marder E, Abbott LF (1996) Decoding synapses. Journal of Neuroscience 16: 6307–6318.
88. Shadlen MN, Newsome WT (1998) The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. Journal of Neuroscience 18: 3870–3896.
89. Shannon CE, Weaver W (1949) The Mathematical Theory of Communication. Urbana: University of Illinois Press.
90. Shlens J, Field GD, Gauthier JL, Grivich MI, Petrusca D, Sher A, Litke AM, Chichilnisky EJ (2006) Probing the structure of multi-neuron firing patterns in the primate retina using maximum entropy methods. CoSyNe, Salt Lake City, UT.
91. Shlens J, Kennel MB, Abarbanel HD, Chichilnisky EJ (in press 2006) Estimating information rates with confidence intervals in neural spike trains. Neural Computation.
92. Slepian D (1976) On bandwidth. Proceedings of the IEEE 64: 292–300.
93. Softky W (1994) Sub-millisecond coincidence detection in active dendritic trees. Neuroscience 58: 13–41.
94. Softky WR, Koch C (1993) The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. Journal of Neuroscience 13: 334–350.
95. Solomonoff R (1964) A formal theory of inductive inference: Part I. Information and Control 7: 1–22.
96. Stopfer M, Bhagavan S, Smith BH, Laurent G (1997) Impaired odour discrimination on desynchronization of odour-encoding neural assemblies. Nature 390: 70–74.
97. Strong SP, Koberle R, Ruyter van Steveninck RR, Bialek W (1998) Entropy and information in neural spike trains. Physical Review Letters 80: 197–200.
98. Tallon C, Bertrand O, Bouchet P, Pernier J (1995) Gamma-range activity evoked by coherent visual stimuli in humans. European Journal of Neuroscience 7: 1285–1291.
99. Theiler J, Galdrikian B, Longtin A, Farmer J (1991) Testing for nonlinearity in time series: The method of surrogate data. Los Alamos National Laboratory Preprint, LA-UR-91-3343.
100. Theiler J, Rapp PE (1996) Re-examination of the evidence for low-dimensional, nonlinear structure in the human electroencephalogram. Electroencephalography and Clinical Neurophysiology 98: 213–222.
101. Théunissen F, Miller JP (1995) Temporal encoding in nervous systems: A rigorous definition. Journal of Computational Neuroscience 2: 149–162.
102. Theunissen F, Roddey JC, Stufflebeam S, Clague H, Miller JP (1996) Information theoretic analysis of dynamical encoding by four identified primary sensory interneurons in the cricket cercal system. Journal of Neurophysiology 75: 1345–1364.
103. Thomson DJ (1982) Spectrum estimation and harmonic analysis. Proceedings of the IEEE 70: 1055–1096.
104. Treves A, Panzeri S (1995). The upward bias in measures of information derived from limited data samples. Neural Computation 7: 399–407.
105. Usrey WM, Reppas JB, Reid RC (1998) Paired-spike interactions and synaptic efficacy of retinal inputs to the thalamus. Nature 395: 384–387.
106. van der Togt C, Kalitzin S, Spekreijse H, Lamme VA, Super H (2006) Synchrony dynamics in monkey V1 predict success in visual detection. Cerebral Cortex 16: 136–148.
107. Victor JD (2002). Binless strategies for estimation of information from neural data. Physical Review E 66: 51903.
108. Victor JD (2005) Spike train metrics. Current Opinion in Neurobiology 15: 585–592.
109. Victor JD, Purpura KP (1996a) Nature and precision of temporal coding in visual cortex: A metric-space analysis. Journal of Neurophysiology 76: 1310–1326.
110. Victor JD, Purpura KP (1996b) Nature and precision of temporal coding in visual cortex: A metric-space analysis. Journal of Neurophysiology 76: 1310–1326.
111. Victor JD, Purpura KP (1997) Metric-space analysis of spike trains: theory, algorithms and application. Network 8: 127–164.
112. Willems FMJ, Shtarkov YM, Tjalkens TJ (1995) The context-tree weighting method: basic properties. IEEE Transactions on Information Theory 41: 653–664.
113. Wolpert DH, Wolf DR (1995) Estimating functions of probability distributions from a finite set of samples. Physical Review E 52: 6841–6854.
114. Wu M, David SV, Gallant J (2006) Complete functional characterization of sensory neurons by system identification. Annual Review of Neuroscience 29: 477–505.
115. Wyner AD, Ziv J (1989) Some asymptotic properties of entropy of a stationary ergodic data source with applications to data compression. IEEE Transactions on Information Theory 35: 1250–1258.