Inequalities between entropy and index of coincidence derived from information diagrams

IEEE Transactions on Information Theory

Volumn 47, Issue 7, 2001, Pages 2944-2960

  Harremoës, Peter ^a   Topsøe, Flemming ^b  

^a Ordrup Gymnasium   (Denmark)

^b UNIVERSITY OF COPENHAGEN   (Denmark)

Author keywords

Chi squared divergence; Concave convex optimization; Entropy; Entropy inequalities; Index of coincidence; Information diagrams; Information divergence; Lemma of replacement; Measure of roughness; R nyi entropy

Indexed keywords

OPTIMIZATION; PROBABILITY DISTRIBUTIONS; PROBLEM SOLVING; RANDOM PROCESSES; STATISTICAL METHODS; TOPOLOGY;

INFORMATION DIAGRAMS;

INFORMATION THEORY;

EID: 0035504977     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.959272     Document Type: Article

References (34)

1. F-Entropies, probability of error, and feature selection
2. Information-type measures of difference of probability distributions and indirect observations
3. Generalized information functions
4. Relations between entropy and error probability
5. Quantization
6. Optimal entropy-constrained scalar quantization of a uniform source
7. A general minimax result for relative entropy
8. The concept of structural α-entropy
9. The problem of character recognition from the point of view of mathematical statistics
10. Divergence measures based on the Shannon entropy
11. On measures of entropy and information
12. Bounds on entropy in a guessing game
13. Uncertainty and the probability of error
14. The Jordan-Schönflies theorem and the classification of surfaces
15. Information theoretical optimization techniques
16. Instances of exact prediction and a new type of inequalities obtained by anchoring
17. Some inequalities for information divergence and related measures of discrimination
18. Bounds for entropy and divergence for distributions over a two-element set
19. Bounds on the minimal error probability and checking a finite or countable number of hypothesis
20. Majorization, concave entropies and comparison of experiments
21. Asymptotic minimax regret for data compression, gambling, and prediction