Entropy and information in neural spike trains: Progress on the sampling problem

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 69, Issue 5 1, 2004, Pages

  Nemenman, Ilya ^a,c   Bialek, William ^b   De Ruyter Van Steveninck, Rob ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CODES (SYMBOLS); DATA PROCESSING; INFORMATION THEORY; INTEGRAL EQUATIONS; MATHEMATICAL MODELS; NATURAL SCIENCES COMPUTING; PROBABILITY DISTRIBUTIONS;

BAYESIAN ENTROPY ESTIMATORS; INFORMATION THEORETIC ANALYSIS; NEURAL SPIKE TRAINS; SENSORY INPUTS;

NEURAL NETWORKS;

EID: 42749107407     PISSN: 15393755     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (37)

1. W. Bialek, F. Rieke, R. R. de Ruyter van Steveninck, and D. Warland, Science 252, 1854 (1991).
2. E. Theunissen and J. P. Miller, J. Neurophysiol. 66, 1690 (1991).
3. M. J. Berry, D. K. Warland, and M. Meister, Proc. Natl. Acad. Sci. U.S.A. 94, 5411 (1997).
4. S. P. Strong, R. Koberle, R. R. de Ruyter van Steveninck, and W. Bialek, Phys. Rev. Lett. 80, 197 (1998).
5. A. Borst and F. E. Theunissen, Nat. Neurosci. 2, 947 (1999).
6. N. Brenner, S. P. Strong, R. Koberle, W. Bialek, and R. R. de Ruyter van Steveninck, Neural Comput. 12, 1531 (2000).
7. P. Reinagel and R. C. Reid, J. Neurosci. 20, 5392 (2000).
8. D. S. Reich, F. Mechler, and J. D. Victor, J. Neurophysiol. 85, 1039 (2001).
9. F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, Spikes: Exploring the Neural Code (MIT Press, Cambridge, MA, 1997).
10. G. D. Lewen, W. Bialek, and R. R. de Ruyter van Steveninck, Network Comput. Neural Syst. 12, 312 (2001).
11. H. B. Barlow, in Proceedings of the Symposium on the Mechanization of Thought Processes, edited by D. V. Blake and A. M. Uttley (H. M. Stationery Office, London, 1959), Vol. 2, pp. 537-574.
12. H. B. Barlow, in Sensory Communication, edited by W. Rosenblith (MIT Press, Cambridge, MA, 1961), pp. 217-234.
13. S. B. Laughlin, Z. Naturforsch. C 36c, 910 (1981).
14. N. Brenner, W. Bialek, and R. R. de Ruyter van Steveninck, Neuron 26, 695 (2000).
15. A. L. Fairhall, G. D. Lewen, W. Bialek, and R. R. de Ruyter van Steveninck, Nature (London) 412, 787 (2001).
16. Z. F. Mainen and T. J. Sejnowski, Science 268, 1503 (1995).
17. R. R. de Ruyter van Steveninck, G. D. Lewen, S. P. Strong, R. Koberle, and W. Bialek, Science 275, 1805 (1997).
18. J. D. Victor, Phys. Rev. E 66, 051903 (2002).
19. J. D. Victor and K. Purpura, Network Comput. Neural Syst. 8, 127 (1997).
20. S. Ma, J. Stat. Phys. 26, 221 (1981).
21. I. Nemenman, F. Shafee, and W. Bialek, in Advances in Neural Information Processing Systems 14, edited by T. G. Dietterich, S. Becker, and Z. Ghahramani (MIT Press, Cambridge, MA, 2002).
22. L. Paninski, Neural Comput. 15, 1191 (2003).
23. T. Batu, S. Dasgupta, R. Kumar, and R. Rubinfeld, in Proceedings of the 34th Symposium on Theory Computing, ACM, 2002.
24. G. A. Miller, in Information Theory in Psychology; Problems and Methods II-B, edited by H. Quastler (Free Press, Glencoe, IL, 1955), pp. 95-100.
25. S. Panzeri and A. Treves, Network Comput. Neural Syst. 7, 87 (1996).
26. P. Grassberger, Phys. Lett. A 128, 369 (1988).
27. P. Grassberger, physics/0307138.
28. D. Wolpert and D. Wolf, Phys. Rev. E 52, 6841 (1995).
29. I. Nemenman, e-print physics/0207009.
30. M. F. Land and T. S. Collett, J. Comp. Physiol. 89, 331 (1974).
31. B. S. Clarke and A. R. Barron, IEEE Trans. Inf. Theory 36, 453 (1990).
32. I. Nemenman, Ph.D. thesis, Princeton University, 2000.
33. W. Bialek, I. Nemenman, and N. Tishby, Neural Comput. 13, 2409 (2001).
34. R. de Ruyter van Steveninck and W. Bialek, in Methods in Neural Networks IV, edited by J. van Hemmen, J. D. Cowan, and E. Domany (Springer-Verlag, Heidelberg, New York, 2001), pp. 313-371 (see Fig. 17).
35. In reference to Bayesian estimators, consistency usually means that, as N grows, the posterior probability concentrates around unknown parameters of the true model that generated the data. For finite parameter models, such as the one considered here, only technical assumptions like positivity of the prior for all parameter values, soundness (different parameters always correspond to different distributions) [31], and a few others are needed for consistency. For nonparametric models, the situation is more complicated. There one also needs ultraviolet convergence of the functional integrals defined by the prior [32,33].
36. It may happen that information is a small difference between two large entropies. Then, due to statistical errors, methods that estimate information directly will have an advantage over NSB, which estimates entropies first. In our case, this is not a problem since the information is roughly a half of the total available entropy [4].
37. For our and many other neural systems, the spike timing can be more accurate than the refractory period of roughly 2 ms [6,10,34]. For the current amount of data, discretization of l ms and large enough T will push the limits of all estimation methods, including ours, that do not make explicit assumptions about properties of the spike trains. Thus, to have enough statistics to convincingly show validity of the NSB approach, in this paper we choose τ=0.75...2 ms, which is still much shorter than other methods can handle. We leave open the possibility that more information is contained in timing precision at finer scales.