On the entropy of compound distributions on nonnegative integers

IEEE Transactions on Information Theory

Volumn 55, Issue 8, 2009, Pages 3645-3650

  Yu, Yaming ^a  

^a 5226 Donald Bren Hall   (United States)

Author keywords

Compound binomial; Compound Poisson; Convex order; Infinite divisibility; Log concavity; Maximum entropy; Minimum entropy; Random sum; Stochastic orders

Indexed keywords

COMPOUND BINOMIAL; COMPOUND POISSON; CONVEX ORDER; INFINITE DIVISIBILITY; LOG-CONCAVITY; MAXIMUM ENTROPY; MINIMUM ENTROPY; RANDOM SUM; STOCHASTIC ORDERS;

ENTROPY; POISSON EQUATION;

POISSON DISTRIBUTION;

EID: 68249133772     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2009.2023725     Document Type: Article

References (32)

1. S. Artstein, K. M. Ball, F. Barthe, and A. Naor, "Solution of Shannon's problem on the monotonicity of entropy," J. Amer. Math. Soc., vol. 17, no. 4, pp. 975-982, 2004.
2. A. D. Barbour, L. Chen, and W.-L. Loh, "Compound Poisson approximation for nonnegative random variables via Steins method," Ann. Prob., vol. 20, no. 4, pp. 1843-1866, 1992.
3. A. D. Barbour and O. Chryssaphinou, "Compound Poisson approximation: A user's guide," Ann. Appl. Prob., vol. 11, pp. 964-1002, 2001.
4. A. R. Barron, "Entropy and the central limit theorem," Ann. Probab. vol. 14, pp. 336-342, 1986.
5. W. Feller, An Introduction to Probability Theory and Its Applications New York: Wiley, 1966, vol. 1.
6. B. G. Hansen, "On log-concave and log-convex infinitely divisible sequences and densities," Ann. Prob., vol. 16, no. 4, pp. 1832-1839, Oct. 1988.
7. P. Harremoës, "Binomial and Poisson distributions as maximum entropy distributions," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 2039-2041, Jul. 2001.
8. P. Harremoës, O. Johnson, and I. Kontoyiannis, "Thinning and the law of small numbers," in Proc. IEEE Int. Symp. Inf. Theory, Nice, France, Jun. 2007.
9. P. Harremoës, O. Johnson, and I. Kontoyiannis, "Thinning and information projections," in Proc. IEEE Int. Symp. Inf. Theory, Toronto, Canada, Jul. 2008.
10. W. Hoeffding, "On the distribution of the number of successes in independent trials," Ann. Math. Statist., vol. 27, pp. 713-721, 1956.
11. A. Jean-Marie and Z. Liu, "Stochastic comparisons for queueing models via random sums and intervals," Adv. Appl. Prob., vol. 24, no. 4, pp. 960-985, 1992.
12. O. Johnson, "Log-concavity and the maximum entropy property of the Poisson distribution," Stochastic Process. Their Appl., vol. 117, no. 6, pp. 791-802, Jun. 2007.
13. O. Johnson, I. Kontoyiannis, and M. Madiman, "On the entropy and log-concavity of compound Poisson measures," Preprint 2008 [Online]. Available: http://arxiv.org/abs/0805.4112v1
14. S. Karlin and Y. Rinott, "Entropy inequalities for classes of probability distributions I. The univariate case," Adv. Appl. Prob. vol. 13, pp. 93-112, 1981.
15. S. Karlin and W. J. Studden, Tchebycheff Systems: With Applications in Analysis and Statistics. New York: Interscience, 1966.
16. S. K. Katti, "Infinite divisibility of integer-valued random variables," Ann. Math. Stat., vol. 38, pp. 1306-1308, 1967.
17. I. Kontoyiannis, P. Harremoës, and O. T. Johnson, "Entropy and the law of small numbers," IEEE Trans. Inf. Theory, vol. 51, no. 2, pp. 466-472, Feb. 2005.
18. I. Kontoyiannis and M. Madiman, "Entropy, compound Poisson approximation, log-Sobolev inequalities and measure concentration," in Proc. 2004 IEEE Inf. Theory Workshop, San Antonio, TX, Oct. 2004.
19. M. Madiman and A. Barron, "Generalized entropy power inequalities and monotonicity properties of information," IEEE Trans. Inf. Theory, vol. 53, no. 7, pp. 2317-2329, Jul. 2007.
20. M. Madiman, O. Johnson, and I. Kontoyiannis, "Fisher information, compound Poisson approximation, and the Poisson channel," in Proc. 2007 IEEE Int. Symp. Inf. Theory, Nice, France, Jun. 2007.
21. P. Mateev, "The entropy of the multinomial distribution," Teor. Verojatnost. i Primenen, vol. 23, no. 1, pp. 196-198, 1978.
22. H. H. Panjer, "Recursive evaluation of a family of compound distributions," ASTIN Bullet., vol. 12, no. 1, pp. 22-26, 1981.
23. R. Pemantle, "Towards a theory of negative dependence," J. Math. Phys., vol. 41, no. 3, pp. 1371-1390, 2000.
24. M. Shaked and J. G. Shanthikumar, Stochastic Orders. New York: Springer, 2007.
25. L. A. Shepp and I. Olkin, "Entropy of the sum of independent Bernoulli random variables and of the multinomial distribution," in Contributions to Probability. New York: Academic, 1981, pp. 201-206.
26. F. W. Steutel and K. Van Harn, Infinite Divisibility of Probability Distributions on the Real Line. New York: Marcel-Dekker, 2003.
27. A. M. Tulino and S. Verdú, "Monotonic decrease of the non-Gaussianness of the sum of independent random variables: A simple proof," IEEE Trans. Inf. Theory, vol. 52, no. 9, pp. 4295-4297, Sep. 2006.
28. W. Whitt, "Uniform conditional variability ordering of probability distributions," J. Appl. Prob., vol. 22, pp. 619-633, 1985.
29. Y. Yu, "On the maximum entropy properties of the binomial distribution," IEEE Trans. Inf. Theory, vol. 54, pp. 3351-3353, 2008.
30. Y. Yu, Relative Log-Concavity and a Pair of Triangle Inequalities Dep. Statist., Univ. Calif., Irvine, Tech. Rep., 2008 [Online]. Available: http://www.ics.uci.edu/~yamingy/nb.ps
31. Y. Yu, "On an inequality of Karlin and Rinott concerning weighted sums of i.i.d. random variables," Adv. Appl. Prob., vol. 40, no. 4, pp. 1223-1226, 2008.
32. Y. Yu, "Monotonic convergence in an information-theoretic law of small numbers," Preprint 2009 [Online]. Available: http://arxiv.org/abs/0810.5203