Interpretations of Rényi entropies and divergences

Physica A: Statistical Mechanics and its Applications

Volumn 365, Issue 1, 2006, Pages 57-62

  Harremoës, Peter ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Convexity; Kraft's inequality; R nyi divergence; R nyi entropy

Indexed keywords

ENTROPY; MATHEMATICAL TECHNIQUES; NUMERICAL ANALYSIS; OPERATIONAL AMPLIFIERS; PROBLEM SOLVING; STATISTICAL METHODS;

CONVEXITY; KRAFT'S INEQUALITY; RÉNYI DIVERGENCE; RÉNYI ENTROPY;

STATISTICAL MECHANICS;

EID: 33645957631     PISSN: 03784371     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.physa.2006.01.012     Document Type: Article

References (12)

1. C.E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948) 379-423 and 623-656.
2. Arndt C. Information Measures (2001), Springer, Berlin
3. Havrda J., and Charvát F. Quantification method of classification processes. Concept of structural a-entropy. Kybernetika 3 (1967) 30-35 (Review by I. Csiszár in MR, vol. 34, no.8875)
4. Tsallis C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys. 52 (1988) 479 See http://tsallis.cat.cbpf.br/biblio.htm for a complete and updated bibliography
5. A. Rényi, On measures of entropy and information, in: Proceedings of the Fourth Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, University California Press, Berkeley, 1961, pp. 547-561.
6. Linnik Yu.V. An information-theoretic proof of the central limit theorem with Lindeberg condition. Theory Probab. Appl. 4 (1959) 288-299
7. Cambell L.L. A coding theorem and Rényi's entropy. Inf. Control 8 (1965) 423-429
8. Ben-Bessat M., and Raviv J. Rényi's entropy and the probability of error. IEEE Trans. Inf. Theory 24 (1978) 324-331
9. Csiszár I. Generalized cutoff rates and Rényi information measures. IEEE Trans. Inform. Theory 41 1 (1995) 26-34
10. Rényi A. On the foundation of information theory. Rev. Inst. Int. Stat. 33 (1965) 1-14
11. Pfister C.E., and Sullivan W.G. Rényi entropy, guesswork moments, and large deviations. IEEE Trans. Inform. Theory 50 11 (2004) 2794-2800
12. Cover T., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York