Bayesian bin distribution inference and mutual information

IEEE Transactions on Information Theory

Volumn 51, Issue 11, 2005, Pages 3766-3779

  Endres, Dominik ^a   Földiák, Peter ^a  

^a UNIVERSITY OF ST ANDREWS   (United Kingdom)

Author keywords

Bayesian inference; Entropy; Model selection; Mutual information

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTATIONAL METHODS; MATHEMATICAL MODELS; MATHEMATICAL TRANSFORMATIONS; POLYNOMIALS; PROBABILITY DISTRIBUTIONS;

BAYESIAN BIN DISTRIBUTION; MODEL SELECTION; MUTUAL INFORMATION; PROBABILITY DISTRIBUTION MODEL;

INFORMATION THEORY;

EID: 27744440784     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.856954     Document Type: Article

References (20)

1. M. Rosenblatt, "Remarks on some nonparametric estimates of a density function," Ann. Math. Statist., vol. 27, pp. 832-837, 1956.
2. M. Stone, "Cross-validatory choice and assessment of statistical predictions," J. Roy. Statist. Soc., Ser. B, vol. 36, no. 1, pp. 111-147, 1974.
3. N. Ueda, R. Nakano, Z. Ghahramani, and G. E. Hinton, "SMEM algorithm for mixture models," Neur Comput., vol. 12, no. 9, pp. 2109-2128, 2000.
4. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood for incomplete data via the EM algorithm," J. Roy. Statist. Soc., Ser B, vol. 39, no. 1, pp. 1-38, 1977.
5. R. Cox, "Probability, frequency and reasonable expectation," Amer. J. Phys., vol. 14, no. 1, pp. 1-13, 1946.
6. M. Jordan, Z. Ghahramani, T. Jaakkola, and L. Saul, "An introduction to variational methods for graphical models," Mach. Learn., vol. 37, no. 2, pp. 183-233, 1999.
7. C. Wallace and D. Boulton, "An information measure for classification," Comput. J., vol. 11, pp. 185-194, 1968.
8. J. Rissanen, "Modeling by shortest data description," Automatica, Vol. 14, pp. 465-471, 1978.
9. M. Hutter, "Distribution of mutual information," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2002, vol. 14, pp. 339-406.
10. F. Kschischang, B. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
11. P. Davis, "Gamma function and related functions," in Handbook of Mathematical Functions, M. Abramowitz and I. Stegun, Eds. New York: Dover, 1972.
12. W. Press, B. Flannery, S. Teukolsky, and W. Vetterling, Numerical Recipes in C: The Art of Scientific Computing. New York: Cambridge Univ. Press, 1986.
13. D. Endres and J. Schindelin, "A new metric for probability distributions," IEEE Trans. Inf. Theory, vol. 49, no. 7, pp. 1858-1860, Jul. 2003.
14. T. S. Ferguson, "Prior distributions on spaces of probability measures," Ann. Statist., vol. 2, no. 4, pp. 615-629, 1974.
15. S. Panzeri and A. Treves, "Analytical estimates of limited sampling biases in different information measures," Network: Comp. Neur. Syst., vol. 7, pp. 87-107, 1996.
16. L. Optican, T. Gawne, B. Richmond, and P. Joseph, "Unbiased measures of transmitted information and channel capacity from multivariate neuronal data," Biol. Cybern., vol. 65, pp. 305-310, 1991.
17. E. Rolls, H. Critchley, and A. Treves, "The representation of olfactory information in the primate orbitofrontal cortex," J. Neumphys., vol. 75, pp. 1982-1996, 1995.
18. E. Arabzadeh, S. Panzeri, and M. Diamond, "Whisker vibration information carried by rat barrel cortex neurons," J. Neurosci., vol. 24, no. 26, pp. 6011-6020, 2004.
19. I. Nemenman, W. Bialek, and R. van Steveninck, "Entropy and information in neural spike trains: Progress on the sampling problem," Phys. Rev. E, vol. 69, no. 5, 2004.
20. L. Paninski, "Estimating entropy on m bins given fewer than m samples," IEEE Trans. Inf. Theory, vol. 50, no. 9, pp. 2200-2203, Sep. 2004.