Relations Between Entropy and Error Probability

IEEE Transactions on Information Theory

Volumn 40, Issue 1, 1994, Pages 259-266

  Feder, Meir ^a   Merhav, Neri ^b  

^a TEL AVIV UNIVERSITY   (Israel)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

channel coding theorem; Entropy; equivocation; error probability; Fano's inequality; predictability

Indexed keywords

CODES (SYMBOLS); CODING ERRORS; COMMUNICATION CHANNELS (INFORMATION THEORY); ENCODING (SYMBOLS); ERROR ANALYSIS; MATHEMATICAL MODELS; PROBABILITY; SIGNAL FILTERING AND PREDICTION; SIGNAL THEORY;

CHANNEL CODING THEOREM; ENTROPY; EQUIVOCATION; ERROR PROBABILITY; FANO'S INEQUALITY; MAXIMUM A POSTERIORI RULE; PREDICTABILITY; UNCERTAINTY;

INFORMATION THEORY;

EID: 0028273691     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.272494     Document Type: Article

References (11)

1. J. Chu and J Chueh, “Inequalities between information measures and error probability,” J. Franklin Inst., vol. 282, pp. 121-125, Aug. 1966.
2. T. M. Cover, “Behavior of sequential predictors of binary sequences,” in Proc. 4th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, 1965, Publishing House of the Czechoslovak Academy of Sciences, Prague, 1967, 263-272.
3. “Universal gambling schemes and the complexity measures of Kolmogorov and Chaitin,” Dep. of Statistics, Stanford Univ., Tech. Rep. 12, Oct. 1974.
4. T. M. Cover and J. A. Thomas, Elements of Information Theory. Wiley, New York, 1991.
5. M. Feder, N. Merhav, and M. Gutman, “Universal prediction of individual sequences,” IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1270, July 1992.
6. R. G. Gallager, “A simple derivation of the coding theorem and some applications,” IEEE Trans. Inform. Theory, vol. IT-11, pp. 3-18, Jan. 1965.
7. Information Theory and Reliable Communications. Wiley, New York: 1968.
8. M. E. Hellman and J. Raviv, “Probability of error, equivocation, and the Chernoff bound,” IEEE Trans. Inform. Theory, vol. IT-16, pp. 368-372, July 1970.
9. G. Jumarie, Relative Information: Theory and Applications. New York: Springer-Verlag, 1990.
10. J. L. Kelly, Jr., “A new interpretation of information rate,” Bell Syst. Tech. J., vol. 35, pp. 917-926, 1956.
11. H. Yamamoto, “Information theory in cryptology,” IEICE Trans., vol. E-74, no. 9, pp. 2456-2464, 1991.