Sharp bounds on the entropy of the poisson law and related quantities

IEEE Transactions on Information Theory

Volumn 56, Issue 5, 2010, Pages 2299-2306

  Adell, José A ^a   Lekuona, Alberto ^a   Yu, Yaming ^b  

^a UNIVERSITY OF ZARAGOZA   (Spain)

^b 5226 Donald Bren Hall   (United States)

Author keywords

Asymptotic expansion; Binomial distribution; Central moments; Complete monotonicity; Entropy bounds; Integral representation; Poisson channel; Poisson distribution

Indexed keywords

ASYMPTOTIC EXPANSION; BINOMIAL DISTRIBUTION; CENTRAL MOMENTS; ENTROPY BOUND; INTEGRAL REPRESENTATION; MONOTONICITY; POISSON CHANNEL;

ASYMPTOTIC ANALYSIS; COMMUNICATION CHANNELS (INFORMATION THEORY); ENTROPY;

POISSON DISTRIBUTION;

EID: 77951542234     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2044057     Document Type: Article

References (33)

1. M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions. New York: Dover, 1964.
2. J. A. Adell and J. M. Anoz, "Signed binomial approximation of binomial mixtures via differential calculus for linear operators, " J. Statist. Plann. Inference, vol. 138, no. 12, pp. 3687-3695, 2008.
3. J. A. Adell and A. Lekuona, "Sharp estimates in signed Poisson approximation of Poisson mixtures, " Bernoulli, vol. 11, no. 1, pp. 47-65, 2005.
4. A. D. Barbour, L. Holst, and S. Janson, Poisson Approximation, Oxford Studies in Probability. Oxford, U. K.: Clarendon, 1992, vol. 2.
5. B. C. Berndt, Ramanujan's Notebooks. New York: Springer-Verlag, 1985, pt. Part I.
6. D. Brady and S. Verdú, "The asymptotic capacity of the direct detection photon channel with a bandwidth constraint, " in Proc. 28th Allerton Conf. Communication, Control and Computing, Monticello, IL, Oct. 1990, pp. 691-700.
7. S. Chang and E. Weldon, "Coding for T-user multiple-access channels, " IEEE Trans. Inf. Theory, vol. IT-25, no. 6, pp. 684-691, Nov. 1979.
8. T. Cover and J. Thomas, Elements of Information Theory, 2nd ed. New York: Wiley, 2006.
9. A. D'yachkov and P. Vilenkin, "Asymptotics of the Shannon and Rényi entropies for sums of independent random variables, " in Proc. ISIT, Cambridge, MA, 1998, p. 376.
10. R. J. Evans, J. Boersma, N. M. Blachman, and A. A. Jagers, "The entropy of a Poisson distribution: Problem 87-6, " SIAMRev., vol. 30, pp. 314-317, 1988.
11. P. Flajolet, "Singularity analysis and asymptotics of Bernoulli sums, " Theoret. Comput. Sci., vol. 275, pp. 371-387, 1999.
12. M. R. Frey, "Information capacity of the Poisson channel, " IEEE Trans. Inf. Theory, vol. 37, no. 2, pp. 244-256, Mar. 1991.
13. D. Guo, S. Shamai, and S. Verdú, "Mutual information and conditional mean estimation in Poisson channels, " IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 1837-1849, May 2008.
14. P. Harremoës, "Binomial and Poisson distributions as maximum entropy distributions, " IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 2039-2041, Jul. 2001.
15. P. Harremoës and P. S. Ruzankin, "Rate of convergence to Poisson law in terms of information divergence, " IEEE Trans. Inf. Theory, vol. 50, no. 9, pp. 2145-2149, Sep. 2004.
16. P. Jacquet and W. Szpankowski, "Analytical depoissonization and its applications, " Theoret. Comput. Sci., vol. 201, pp. 1-62, 1998.
17. P. Jacquet and W. Szpankowski, "Entropy computations via analytic depoissonization, " IEEE Trans. Inf. Theory, vol. 45, no. 4, pp. 1072-1081, Jul. 1999.
18. O. Johnson, "Log-concavity and the maximum entropy property of the Poisson distribution, " Stochastic Process. Appl., vol. 117, no. 6, pp. 791-802, Jun. 2007.
19. N. L. Johnson, A. W. Kemp, and S. Kotz, Univariate Discrete Distributions, 3rd ed. Hoboken, NJ: Wiley, 2005.
20. C. Knessl, "Integral representations and asymptotic expansions for Shannon and Renyi entropies, " Appl. Math. Lett., vol. 11, pp. 69-74, 1998.
21. I. Kontoyiannis, P. Harremoës, and O. T. Johnson, "Entropy and the law of small numbers, " IEEE Trans. Inf. Theory, vol. 51, no. 2, pp. 466-472, Feb. 2005.
22. A. Lapidoth and S. M. Moser, "On the capacity of the discrete-time Poisson channel, " IEEE Trans. Inf. Theory, vol. 55, no. 1, pp. 303-322, Jan., 2009.
23. M. Madiman, O. Johnson, and I. Kontoyiannis, "Fisher information, compound Poisson approximation and the Poisson channel, " presented at the IEEE Int. Symp. Information Theory, Nice, France, Jun. 2007.
24. A. Martinez, "Spectral efficiency of optical direct detection, " J. Opt. Soc. Amer. B, vol. 24, no. 4, pp. 739-749, Apr. 2007.
25. P. Mateev, "The entropy of the multinomial distribution, " Teor. Verojatnost. i Primenen., vol. 23, no. 1, pp. 196-198, 1978.
26. S. Shamai (Shitz), "Capacity of a pulse amplitude modulated direct detection photon channel, " Proc. Inst. Elec. Eng., vol. 137, no. 6, pt. I (Communications, Speech, and Vision), pp. 424-430, Dec. 1990.
27. S. Shamai (Shitz) and A. Lapidoth, "Bounds on the capacity of a spectrally constrained Poisson channel, " IEEE Trans. Inf. Theory, vol. 39, no. 1, pp. 19-29, Jan. 1993.
28. L. A. Shepp and I. Olkin, "Entropy of the sum of independent bernoulli random variables and of the multinomial distribution, " in Proc. Contributions to Probability, New York, 1981, pp. 201-206.
29. F. Topsøe, "Maximum entropy versus minimum risk and applications to some classical discrete distributions, " IEEE Trans. Inf. Theory, vol. 48, no. 8, pp. 2368-2376, Aug. 2002.
30. Y. Yu, "On the maximum entropy properties of the binomial distribution, " IEEE Trans. Inf. Theory, vol. 54, no. 7, pp. 3351-3353, Jul. 2008.
31. Y. Yu, "On an asymptotic series of Ramanujan, " Ramanujan J., vol. 20, pp. 179-188, 2009.
32. Y. Yu, "Complete monotonicity of the entropy in the central limit theorem for gamma and inverse Gaussian distributions, " Stat. Prob. Lett., vol. 79, pp. 270-274, 2009.
33. Y. Yu, "Monotonic convergence in an information-theoretic law of small numbers, " IEEE Trans. Inf. Theory, vol. 55, no. 12, pp. 5412-5422, Dec. 2009.