Minimum entropy of error estimation for discrete random variables

IEEE Transactions on Information Theory

Volumn 42, Issue 4, 1996, Pages 1193-1201

  Janžura, Martin ^b   Koski, Timo ^a,c   Otáhal, Antonín ^b  

^a IEEE

^b INSTITUTE OF INFORMATION THEORY AND AUTOMATION   (Czech Republic)

^c ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Bayesian estimation; Binary hypotheses; Large deviations; Predictive coding

Indexed keywords

ERRORS; ESTIMATION; MATHEMATICAL MODELS; PROBABILITY; RANDOM PROCESSES;

BAYESIAN ESTIMATION; BINARY HYPOTHESES; PREDICTIVE CODING;

INFORMATION THEORY;

EID: 0030193486     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.508841     Document Type: Article

References (24)

1. H. L. Weidemann and E. B. Stear. "Entropy analysis of parameter estimation," Inform. Contr., vol. 14, pp. 493-506, 1969.
2. _, "Entropy analysis of estimation systems," IEEE Trans. Inform, Theory, vol. IT-16, pp. 264-270, May 1970.
3. M. Janžura, T. Koski, and A. Otáhal, "Minimum entropy of principle error in estimation," Inform. Sci., vol. 79, pp. 123-144, 1994.
4. A. Otáhal, "Finiteness and continuity of differential entropy," in Asymptotic Statistics; Proc. 5th Prague Symp., P. Mandl and M. Hušková, Eds. Heidelberg, Germany: Physica Verlag, 1994, pp. 415-419.
5. P. Elias, "Predictive coding I-II," IRE Trans. Inform. Theory, vol. IT-1, pp. 16-24, 24-33, Mar. 1955.
6. R. E. Blahut, Principles and Practice of Information Theory. Reading MA: Addison-Wesley. 1987.
7. O. Pahlm, P.-O. Börjesson, and O. Werner, "Compact digital storage of ECG's," Comput. Progr. Biomed., vol. 9, pp. 293-300, 1979.
8. G. Gabor and Z. Györfi, Recursive Source Coding. New York, Berlin, Heidelberg: Springer-Verlag, 1986.
9. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
10. S. Arimoto, "Information-theoretical considerations on estimation problems," Inform. Com., vol. 19, pp. 181-194, 1971.
11. D. F. Kerridge, "Inaccuracy and inference," J. Roy. Stat. Soc., vol R23, pp. 184-194, 1961.
12. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
13. A. Perez, "Information-theoretic risk estimates in statistical decision," Kybernetika, vol. 3, pp. 1-21, 1967.
14. A. Rényi, "On some basic problems of statistics from the point of view of information theory," in Proc. 5th Berkeley Sytnp. Math. Statist. Probab., L. Lc Cam and J. Neyman, Eds. Berkeley, CA: Univ. Calif. Press, vol. 1, pp. 531-543.
15. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1989.
16. I. Vajda, The Theory of Statistical Inference and Information. Dordrecht/Boston/London: Kluwer, 1989.
17. T. Nemetz, "Information theory and the testing of a hypothesis," in Proc. Colloquium on Information Theory, A. Rényi, Ed. Budapest, Hungary: Janos Bolyai Math. Soc., 1968, vol. II, pp. 283-294.
18. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge. UK: Cambridge Univ. Press, 1986.
19. R. M. Gray, Entropy and Information Theory. New York, Berlin, Heidelberg, London, Paris, Tokyo, Hung Kong: Springer-Verlag, 1990.
20. L. Györfi, Z. Györfi, and L. Vajda, "Bayesian decision with rejection," Probl. Contr. Inform. Theory, vol. 8, pp. 445-452, 1979.
21. K. Jacobs, Discrete Stochastics. Basel, Switzerland: Birkhäuser Verlag, 1992.
22. M. Föllmer and S. Orey, "Large deviations for the empirical field of a Gibbs measure," Annals Probab., vol. 17, pp. 961-977, 1988.
23. d," Annals Probab., vol. 21, pp. 1845-1875, 1993.
24. M. Janžura, "Large deviations for Gibbs random fields," in 5th Pannonian Symp. on Mathematical Statistics, W. Grossman, J. Mogyorodi, I. Vincze, and W. Wertz, Eds. (Visegrad, Hungary, 1985). pp. 97-112.