Information theory and neural information processing

IEEE Transactions on Information Theory

Volumn 56, Issue 2, 2010, Pages 653-666

  Johnson, Don H ^a  

^a Rice University   (United States)

Author keywords

Capacity of multichannel Poisson channels; Kullback Leibler divergence; Neural processing models; Non Poisson channels

Indexed keywords

CLASSIC INFORMATION; COMMUNICATION MEDIUM; DATA ANALYSIS; ENCODED INFORMATION; EXISTING SYSTEMS; INPUT SIGNAL; KULLBACK LEIBLER DIVERGENCE; MULTI-CHANNEL; NEURAL INFORMATION PROCESSING; NEURAL POPULATIONS; NEURAL SIGNALS; NEURAL-PROCESSING; OPTIMAL PERFORMANCE; POISSON CHANNEL; PROCESS INFORMATION;

DATA PROCESSING; INDEPENDENT COMPONENT ANALYSIS; NEURAL NETWORKS; NEURONS;

CHANNEL CAPACITY;

EID: 77749277069     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2037047     Document Type: Article

References (51)

1. W. Weaver, "The mathematics of communication," Scientif. Amer., vol. 181, pp. 11-15, 1949.
2. D. H. Hubel and T. N. Wiesel, "Receptive fields and functional architecture of monkey striate cortex.," J. Physiol, vol. 195, no. 1, pp. 215-243, Mar. 1968.
3. D. Snyder, Random Point Processes. New York: Wiley, 1975.
4. J. Csicsvari, D. Henze, B. Jamieson, K. D. Harris, A. Sirota, P. Barthó, K. Wise, and G. Buzsáki, "Massively parallel recording of unit and local field potentials with silicon-based electrodes," J. Neurophysiol., vol. 90, pp. 1314-1323, 2003.
5. M. Nicolelis, Methods for Neural Ensemble Recordings. New York: CRC, 2007.
6. H. J. Koester and B. Sakmann, "Calcium dynamics associated with action potentials in single nerve terminals of pyramidal cells in layer 2/3 of the young rat neocortex," J. Physiol., vol. 529, pp. 625-646, Dec. 2000.
7. D. Johnson, "Point process models of single-neuron discharges," J. Comp. Neurosci., vol. 3, pp. 275-299, 1996.
8. D. Daley and D. Vere-Jones, An Introduction to the Theory of Point Processes. New York: Springer-Verlag, 1988.
9. R. Milne, "Infinitely divisible bivariate Poisson processes," Adv. Applied Prob., vol. 6, pp. 226-227, 1974.
10. J. McFadden, "The entropy of a point process," SIAM J. Appl. Math., vol. 13, pp. 988-994, 1965.
11. S. Verdú, "Teaching it," IEEE Inf. Soc. Newslett., vol. 57, pp. 1,8-9, Dec. 2007, XXVIII Shannon Lecture.
12. T. Cover and J. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
13. R. Stein, "The information capacity of nerve cells using a frequency code," Biophysical J., vol. 7, pp. 67-82, 1967.
14. D. Johnson, "Four top reasons mutual information does not quantify neural information processing," in Proc. Computat. Neurosci.'02, Chicago, IL, 2002.
15. Y. Kabanov, "The capacity of a channel of the Poisson type," Theory Prob. Applicat., vol. 23, pp. 143-147, 1978.
16. P. Brémaud, Point Processes and Queues. New York: Springer-Verlag, 1981.
17. M. Davis, "Capacity and cutoff rate for Poisson-type channels," IEEE Trans. Inf. Theory, vol. IT-26, pp. 710-715, 1980.
18. M. Frey, "Information capacity of the Poisson channel," IEEE Trans. Inf. Theory, vol. 37, pp. 244-256, 1991.
19. E. Brown, R. Barbieri, V. Ventura, R. Kass, and L. Frank, "The timerescaling theorem and its application to neural spike train data analysis," Neural Comp., vol. 14, pp. 325-346, 2002.
20. D. Johnson and I. Goodman, "Inferring the capacity of the vector Poisson channel with a Bernoulli model," Network: Computat. Neural Syst., vol. 19, pp. 13-33, 2008.
21. P. Holgate, "Estimation for the bivariate Poisson distribution," Biometrika, vol. 51, pp. 241-245, 1964.
22. R. Griffiths, R. Milne, and R. Wood, "Aspects of correlation in bivariate Poisson distributions and processes," Australian J. Statist., vol. 21, pp. 238-255, 1979.
23. D. Johnson, The Correlation Function of Multiple Dependent Poisson Processes Generated by the Alternating Renewal Process Method arXiv:0811.3713, 2007.
24. P. Latham and S. Nirenberg, "Synergy, redundancy, and independence in population codes, revisited," J. Neurosci., vol. 25, pp. 5195-5206, 2005.
25. N. Brenner, S. Strong, R. Koberle, W. Bialek, and R. de Ruyter van Steveninck, "Synergy in a neural code," Neur. Computat., vol. 12, pp. 1531-1552, 2000.
26. E. Schneidman, W. Bialek, and M. B. II, "Synergy, redundancy, and independence in population codes," J. Neurosci., vol. 23, pp. 11 539-11 553, 2003.
27. D. Reich, F. Mechler, and J. Victor, "Independent and redundant information in nearby cortical neurons," Science, vol. 294, pp. 2566-2568, 2001.
28. I. Goodman and D. Johnson, "Information theoretic bounds on neural prosthesis effectiveness: The importance of spike sorting," in Proc. ICASSP, 2008.
29. S. Panzeri, S. Schultz, A. Treves, and E. Rolls, "Correlations and the encoding of information in the nervous system," Proc. Royal Soc. Lond., vol. 266, pp. 1001-1012, 1999.
30. S. Panzeri and S. Schultz, "A unified approach to the study of temporal, correlational, and rate coding," Neur. Computat., vol. 13, pp. 1311-1349, 2001.
31. S. Sinanovic ́ and D. Johnson, "Toward a theory of information processing," Signal Processing, vol. 87, pp. 1326-1344, 2007.
32. D. Johnson, C. Gruner, K. Baggerly, and C. Seshagiri, "Informationtheoretic analysis of neural coding," J. Comp. Neurosci., vol. 10, pp. 47-69, 2001.
33. S. Ali and S. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Stat. Soc., vol. 28, pp. 131-142, 1966.
34. M. Basseville, "Distance measures for signal processing and pattern recognition," Signal Processing, vol. 18, pp. 349-369, 1989.
35. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
36. C. Rozell, D. Johnson, and R. Glantz, "Measuring information transfer in crayfish sustaining fiber spike generators: Methods and analysis," Biol. Cybernet., vol. 90, pp. 89-97, 2004.
37. D. Johnson, "Neural population structures and consequences for neural coding," J. Comp. Neurosci., vol. 16, pp. 69-80, 2004.
38. A. Carlton, "On the bias of information estimates," Psychol. Bull., vol. 71, pp. 108-109, 1969.
39. L. Paninski, "Estimation of entropy and mutual information," Neur. Computat., vol. 15, pp. 1191-1253, 2003.
40. F. Rieke, D. Warland, R. de Ruyter van Steveninck, and W. Bialek, Spikes: Exploring the Neural Code. Cambridge, MA: MIT Press, 1999.
41. W. Bialek, F. Rieke, R. de Ruyter van Steveninck, and D. Warland, "Reading a neural code," Science, vol. 252, pp. 1852-1856, 1991.
42. J. Victor and K. Purpura, "Metric-space analysis of spike trains: Theory, algorithms and applications," Network: Comput. Neur. Syst., vol. 8, pp. 127-164, 1997.
43. J. D. Victor, "Asymptotic bias in information estimates and the exponential (Bell) polynomials," Neur. Computat., vol. 12, pp. 2797-2804, 2000.
44. O. Shayevitz and M. Feder, "Communication with feedback via posterior matching," in Proc. Int. Symp. Inf. Theory (ISIT), Nice, France, 2007.
45. J. Massey, "Causality, feedback and directed information," in Proc. 1990 Int. Symp. Inf. Theory Its Applicat., Waikiki, Hawaii, 1990.
46. D. Johnson and D. Linebarger, "Signal processing models for point processes," in Proc. ICASSP, San Diego, CA, 1984.
47. T. Ozaki, "Maximum likelihood estimation of Hawkes' self-exciting point processes," Ann. Inst. Statist. Math., vol. 31, pt. B, pp. 145-155, 1979.
48. D. Cox, Renewal Theory. London, Methuen, 1962.
49. D. Johnson, C. Tsuchitani, D. Linebarger, and M. Johnson, "The application of a point process model to the single unit responses of the cat lateral superior olive to ipsilaterally presented tones," Hearing Res., vol. 21, pp. 135-159, 1986.
50. C. Leang and D. Johnson, "On the asymptotics of M-hypothesis Bayesian detection," IEEE Trans. Inf. Theory, vol. 43, pp. 280-282, 1997.
51. H. Cramér, Mathematical Methods of Statistics. Princeton, NJ: Princeton University Press, 1946.