On the interplay between conditional entropy and error probability

IEEE Transactions on Information Theory

Volumn 56, Issue 12, 2010, Pages 5930-5942

  Ho, Siu Wai ^a   Verdú, Sergio ^b  

^a UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

^b Princeton University   (United States)

Author keywords

Entropy; equivocation; Fano's inequality; majorization theory; Schur concavity; Shannon theory

Indexed keywords

EQUIVOCATION; FANO'S INEQUALITY; MAJORIZATION THEORY; SCHUR-CONCAVITY; SHANNON THEORY;

ENTROPY; OPTIMIZATION; PROBABILITY DISTRIBUTIONS;

RANDOM VARIABLES;

EID: 78649349634     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2080891     Document Type: Article

References (13)

1. C. E. Shannon, "A mathematical theory of communication," Bell System Tech. J., vol. 27, pp. 623-656, Oct. 1948.
2. R. M. Fano, Class Notes for Transmission of Information, Course 6.574. Cambridge, MA: MIT, 1952.
3. V. A. Kovalevsky, "The problem of character recognition from the point of view of mathematical statistics," in Character Readers and Pattern Recognition. New York: Spartan, 1968.
4. D. L. Tebbe and S.J. Dwyer, III, "Uncertainty and probability of error," IEEE Trans. Inform. Theory, vol. IT-14, pp. 516-518, May 1968.
5. M. Feder and N. Merhav, "Relations between entropy and error probability," IEEE Trans. Inform. Theory, vol. 40, no. 1, pp. 259-266, Jan. 1994.
6. S.-W. Ho and R. W. Yeung, "On the discontinuity of the Shannon information measures," IEEE Trans. Inform. Theory, vol. 55, no. 12, pp. 5362-5374, Dec. 2009.
7. T. S. Han and S. Verdú, "Generalizing the Fano inequality," IEEE Trans. Inform. Theory, vol. 40, no. 7, pp. 1247-1250, Jul. 1994.
8. R. W. Yeung, Information Theory and Network Coding. Berlin/New York: Springer, 2008.
9. A. W. Marshall and I. Olkin, Inequalities: Theory of Majorization and its Applications. New York: Academic, 1979.
10. V. Erokhin, "e-entropy of a discrete random variable," Theory Probab. ItsApplic, vol. 3, pp. 97-101, 1958.
11. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice-Hall, 1971.
12. F. Topsøe, "Basic concepts, identities and inequalities\the toolkit of information theory," Entropy, vol. 3, pp. 162-190, Sep. 2001.
13. S. Verdú, Class Notes for EE528 Information Theory. Princeton, NJ: Princeton Univ., 2009