Estimating entropy on m bins given fewer than m samples

IEEE Transactions on Information Theory

Volumn 50, Issue 9, 2004, Pages 2200-2203

  Paninski, Liam ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; MAXIMUM LIKELIHOOD ESTIMATION; POISSON DISTRIBUTION; POLYNOMIAL APPROXIMATION; PROBABILITY; THEOREM PROVING;

DISTRIBUTION FREE BOUNDS; ENTROPY FUNCTION; POISSON RANDOM VARIABLE; PROBABILITY MEASURES; RESCALED BIAS FUNCTION; STIRLING APPROXIMATION;

INFORMATION THEORY;

EID: 4544287683     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.833360     Document Type: Article

References (10)

1. G. Miller, "Note on the bias of information estimates," in Information Theory in Psychology II-B. Glencoe, IL: Free Press, 1955, pp. 95-100.
2. L. Paninski, "Estimation of entropy and mutual information," Neural Comput., vol. 15, pp. 1191-1253, 2003.
3. A. Antos and I. Kontoyiannis, "Convergence properties of functional estimates for discrete distributions," Random Structures and Algorithms, vol. 19, pp. 163-193, 2001.
4. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
5. H. Chernoff, "A measure of asymptotic efficiency for tests of hypothesis based on the sum of observations," Ann. Math. Statist., vol. 23, pp. 493-509, 1952.
6. W. Rudin, Functional Analysis. New York: McGraw-Hill, 1973.
7. P. Koosis, The Logarithmic Integral. Cambridge, U.K.: Cambridge Univ. Press, 1988.
8. L. LeCam, Asymptotic Methods in Statistical Decision Theory. New York: Springer-Verlag, 1986.
9. D. Donoho and R. Liu, "Geometrizing rates of convergence," Ann. Statist., vol. 19, pp. 633-701, 1991.
10. A. Renyi, "On measures of entropy and information," in Proc. 4th Berkley Symp. Mathematical Statistics and Probability, vol. 1, 1961, pp. 547-561.