On convergence properties of Shannon entropy

Problems of Information Transmission

Volumn 45, Issue 2, 2009, Pages 75-94

  Piera, F J ^a   Parada, P ^a  

^a UNIVERSITY OF CHILE   (Chile)

Author keywords

[No Author keywords available]

Indexed keywords

BOUNDEDNESS CONDITIONS; CONVERGENCE IN DISTRIBUTION; CONVERGENCE PROPERTIES; DIFFERENTIAL ENTROPY; DIFFERENTIAL SETTING; GENERAL SUPPORTS; PROBABILITY MEASURES; SHANNON ENTROPY; WEAK CONVERGENCE;

PROBABILITY DISTRIBUTIONS;

ENTROPY;

EID: 67749142407     PISSN: 00329460     EISSN: 16083253     Source Type: Journal    
DOI: 10.1134/S003294600902001X     Document Type: Article

References (17)

1. Antos, A. and Kontoyiannis, I., Convergence Properties of Functional Estimates of Discrete Distributions, Random Structures Algorithms, 2001, vol. 19, no. 3-4, pp. 163-193.
2. Hall, P. and Morton, S.C., On the Estimation of Entropy, Ann. Inst. Statist. Math., 1993, vol. 45, no. 1, pp. 69-88.
3. Wieczorkowski, R. and Grzegorzewski, P., Entropy Estimators-Improvements and Comparisons, Comm. Statist. Simulation Comput., 1999, vol. 28, no. 2, pp. 541-567.
4. Wyner, A. and Foster, D., On the Lower Limits of Entropy Estimation, Tech. Rep. Dept. Statistics, Wharton School, Univ. Pennsylvania, 2003. Available at http://www-stat.wharton.upenn.edu/~ajw/lowlimitsentropy.pdf.
5. Blahut, R.E., Principles and Practice of Information Theory, Reading: Addison-Wesley, 1987.
6. Cover, T.M. and Thomas, J.A., Elements of Information Theory, New York: Wiley, 1991.
7. Ho, S.-W. and Yeung, R.W., On the Discontinuity of the Shannon Information Measures, Proc. IEEE Int. Sympos. on Information Theory, Adelaide, Australia, 2005, pp. 159-163.
8. Barron, A.R., Entropy and the Central Limit Theorem, Ann. Probab., 1986, vol. 14, no. 1, pp. 336-342.
9. Barron, A.R., The Strong Ergodic Theorem for Densities: Generalized Shannon-McMillan-Breiman Theorem, Ann. Probab., 1985, vol. 13, no. 4, pp. 1292-1303.
10. Harremoës, P., Information Topologies with Applications, Entropy, Search, Complexity, Csiszár, I., Katona, G.O.H., Tardos, G., Eds., Bolyai Soc. Math. Stud., vol. 16, Berlin: Springer, 2007, pp. 113-150.
11. Dobrushin, R.L., Passage to the Limit under the Information and Entropy Signs, Teor. Veroyatnost. i Primenen., 1960, vol. 5, no. 1, pp. 29-37 [Theory Probab. Appl. (Engl. Transl.), 1960, vol. 5, no. 1, pp. 25-32].
12. Fristedt, B. and Gray, L., A Modern Approach to Probability Theory, Boston: Birkhäuser, 1997.
13. Kallenberg, O., Foundation of Modern Probability, New York: Springer, 2002, 2nd ed.
14. Billingsley, P., Convergence of Probability Measures, New York: Wiley, 1999, 2nd ed.
15. Shiryaev, A.N., Veroyatnost', Moscow: Nauka, 1989, 2nd ed. Translated under the title Probability, New York: Springer, 1996.
16. Royden, H.L., Real Analysis, New York: Macmillan, 1988, 3rd ed.
17. Williams, D., Probability with Martingales, Cambridge: Cambridge Univ. Press, 1991.