Entropy and information in neural spike trains: Progress on the sampling problem

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 69, Issue 5, 2004, Pages 6-

  Nemenman, Ilya ^a   Bialek, William ^b   de Ruyter van Steveninck, Rob ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b PRINCETON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85036164237     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.69.056111     Document Type: Article

References (37)

1. W. Bialek, F. Rieke, R. R. de Ruyter van Steveninck, and D. Warland, ScienceSCIEAS0036-8075 252, 1854 (1991).
2. E. Theunissen and J. P. Miller, J. Neurophysiol.JONEA40022-3077 66, 1690 (1991).
3. M. J. Berry, D. K. Warland, and M. Meister, Proc. Natl. Acad. Sci. U.S.A.PNASA60027-842410.1073/pnas.94.10.5411 94, 5411 (1997).
4. S. P. Strong, R. Koberle, R. R. de Ruyter van Steveninck, and W. Bialek, Phys. Rev. Lett.PRLTAO0031-900710.1103/PhysRevLett.80.197 80, 197 (1998).
5. A. Borst and F. E. Theunissen, Nat. Neurosci.NANEFN1097-625610.1038/14731 2, 947 (1999).
6. N. Brenner, S. P. Strong, R. Koberle, W. Bialek, and R. R. de Ruyter van Steveninck, Neural Comput.NEUCEB0899-766710.1162/089976600300015259 12, 1531 (2000).
7. P. Reinagel and R. C. Reid, J. Neurosci.JNRSDS0270-6474 20, 5392 (2000).
8. D. S. Reich, F. Mechler, and J. D. Victor, J. Neurophysiol.JONEA40022-3077 85, 1039 (2001).
9. F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, Spikes: Exploring the Neural Code (MIT Press, Cambridge, MA, 1997).
10. G. D. Lewen, W. Bialek, and R. R. de Ruyter van Steveninck, Network Comput. Neural Syst.NTWREZ0954-898X10.1088/0954-898X/12/3/305 12, 312 (2001).
11. H. B. Barlow, in Proceedings of the Symposium on the Mechanization of Thought Processes, edited by D. V. Blake and A. M. Uttley (H. M. Stationery Office, London, 1959), Vol. 2, pp. 537–574.
12. H. B. Barlow, in Sensory Communication, edited by W. Rosenblith (MIT Press, Cambridge, MA, 1961), pp. 217–234.
13. S. B. Laughlin, Z. Naturforsch. CZNFCAP0341-0471 36c, 910 (1981).
14. N. Brenner, W. Bialek, and R. R. de Ruyter van Steveninck, NeuronNERNET0896-6273 26, 695 (2000).
15. A. L. Fairhall, G. D. Lewen, W. Bialek, and R. R. de Ruyter van Steveninck, Nature (London)NATUAS0028-083610.1038/35090500 412, 787 (2001).
16. Z. F. Mainen and T. J. Sejnowski, ScienceSCIEAS0036-8075 268, 1503 (1995).
17. R. R. de Ruyter van Steveninck, G. D. Lewen, S. P. Strong, R. Koberle, and W. Bialek, ScienceSCIEAS0036-807510.1126/science.275.5307.1805 275, 1805 (1997).
18. J. D. Victor, Phys. Rev. EPLEEE81063-651X10.1103/PhysRevE.66.051903 66, 051903 (2002).
19. J. D. Victor and K. Purpura, Network Comput. Neural Syst.NTWREZ0954-898X10.1088/0954-898X/8/2/003 8, 127 (1997).
20. S. Ma, J. Stat. Phys.JSTPBS0022-4715 26, 221 (1981).
21. I. Nemenman, F. Shafee, and W. Bialek, in Advances in Neural Information Processing Systems 14, edited by T. G. Dietterich, S. Becker, and Z. Ghahramani (MIT Press, Cambridge, MA, 2002).
22. L. Paninski, Neural Comput.NEUCEB0899-766710.1162/089976603321780272 15, 1191 (2003).
23. T. Batu, S. Dasgupta, R. Kumar, and R. Rubinfeld, in Proceedings of the 34th Symposium on Theory Computing, ACM, 2002.
24. G. A. Miller, in Information Theory in Psychology; Problems and Methods II-B, edited by H. Quastler (Free Press, Glencoe, IL, 1955), pp. 95–100.
25. S. Panzeri and A. Treves, Network Comput. Neural Syst.NTWREZ0954-898X10.1088/0954-898X/7/1/006 7, 87 (1996).
26. P. Grassberger, Phys. Lett. APYLAAG0375-960110.1016/0375-9601(88)90193-4 128, 369 (1988).
27. P. Grassberger, physics/0307138.
28. D. Wolpert and D. Wolf, Phys. Rev. EPLEEE81063-651X10.1103/PhysRevE.52.6841 52, 6841 (1995).
29. I. Nemenman, e-print physics/0207009.
30. M. F. Land and T. S. Collett, J. Comp. Physiol.JRCPA30373-0859 89, 331 (1974).
31. B. S. Clarke and A. R. Barron, IEEE Trans. Inf. TheoryIETTAW0018-944810.1109/18.54897 36, 453 (1990).
32. I. Nemenman, Ph.D. thesis, Princeton University, 2000.
33. W. Bialek, I. Nemenman, and N. Tishby, Neural Comput.NEUCEB0899-766710.1162/089976601753195969 13, 2409 (2001).
34. R. de Ruyter van Steveninck and W. Bialek, in Methods in Neural Networks IV, edited by J. van Hemmen, J. D. Cowan, and E. Domany (Springer-Verlag, Heidelberg, New York, 2001), pp. 313–371 (see Fig. 17).
35. In reference to Bayesian estimators, consistency usually means that, as (Formula presented) grows, the posterior probability concentrates around unknown parameters of the true model that generated the data. For finite parameter models, such as the one considered here, only technical assumptions like positivity of the prior for all parameter values, soundness (different parameters always correspond to different distributions) c31, and a few others are needed for consistency. For nonparametric models, the situation is more complicated. There one also needs ultraviolet convergence of the functional integrals defined by the prior c32 c33.
36. It may happen that information is a small difference between two large entropies. Then, due to statistical errors, methods that estimate information directly will have an advantage over NSB, which estimates entropies first. In our case, this is not a problem since the information is roughly a half of the total available entropy c4.
37. For our and many other neural systems, the spike timing can be more accurate than the refractory period of roughly 2 ms c6 c10 c34. For the current amount of data, discretization of (Formula presented) ms and large enough (Formula presented) will push the limits of all estimation methods, including ours, that do not make explicit assumptions about properties of the spike trains. Thus, to have enough statistics to convincingly show validity of the NSB approach, in this paper we choose (Formula presented) ms, which is still much shorter than other methods can handle. We leave open the possibility that more information is contained in timing precision at finer scales.