Characterization of the bayes estimator and the MDL estimator for exponential families

IEEE Transactions on Information Theory

Volumn 43, Issue 4, 1997, Pages 1165-1174

  Takeuchi, Jun Ichi ^a  

^a NEC CORPORATION   (Japan)

Author keywords

Bayes estimator; Exponential family; Higher order asymptotic theory; Information geometry; Minimum Description Length principle; Universal source coding

Indexed keywords

APPROXIMATION THEORY; PARAMETER ESTIMATION; PROBABILITY DENSITY FUNCTION; REDUNDANCY; SIGNAL ENCODING;

BAYES ESTIMATOR; INFORMATION GEOMETRY; MINIMUM DESCRIPTION LENGTH (MDL) ESTIMATOR;

SIGNAL FILTERING AND PREDICTION;

EID: 0031188040     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.605579     Document Type: Article

References (20)

1. S.-I. Amari, Differential-Geometrical Methods in Statistics 2nd ed. (Lecture Notes in Statistics, vol. 28). Berlin, Germany: Springer-Verlag, 1990.
2. A. R. Barron, "Logically smooth density estimation," Ph.D. dissertation, Dept. Elec. Eng., Stanford Univ., Stanford, CA, 1985.
3. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, pp. 1034-1054, 1991.
4. J. M. Bernardo, "Reference posterior distributions for Bayesian inference," J. Roy. Statist. Soc. B., vol. 41. pp. 113-147, 1979.
5. L. Brown, Fundamentals of Statistical Exponential Families. Inst. Math. Statist., 1986.
6. B. Clarke and A. R. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-471, May 1990.
7. _, "Jeffreys prior is asymptotically least favorable under entropy risk," J. Statist. Planning and Inference, vol. 41, pp. 37-60, 1994.
8. L. D. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, Nov. 1973.
9. L. D. Davisson and A. Leon-Garcia, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, pp. 166-174, Mar. 1980.
10. H. Jeffreys, Theory of Probability, 3rd ed. Berkeley, CA: Univ. of California Press, 1961.
11. T. Matsushima, H. Inazumi, and S. Hirasawa, "A class of distortionless codes designed by Bayes decision theory," IEEE Trans. Inform. Theory, vol. 37, pp. 1288-1293, 1991.
12. J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, pp. 465-471, 1978.
13. _, "Stochastic complexity," J. Roy. Statist. Soc. B, vol. 49, pp. 223-239 and 252-265, 1987.
14. J. Takeuchi and T. Kawabata, "Approximation of Bayes code for Markov sources," in Proc. 1995 IEEE Int. Symp. on Information Theory, 1995, p. 391.
15. C. S. Wallace and D. M. Boulton, "An invariant Bayes method for point estimation," Classification Soc. Bull., vol. 3, pp. 11-34, 1975.
16. C. S. Wallace and P. R. Freeman, "Estimating and inference by compact coding," J. Roy. Statist. Soc. B, vol. 49. no. 3 pp. 240-265, 1987.
17. Q. Xie and A. R. Barron, "Minimax redundancy for the class of memoryless sources,"IEEE Trans. Inform. Theory, to be published.
18. K. Yamanishi, "A learning criterion for stochastic rules," Mach. Learn. (Special Issue on COLT '90), vol. 9, no. 2/3, pp. 165-203, 1992.
19. J. Takeuchi, "On the convergence rate of the MDL estimator with respect to the KL-divergence" (in Japanese), in Proc. 16th Symp. on Information Theory and its Application, 1993, pp. 161-164.
20. J. Takeuchi and N. Abe, "Evaluation of Laplace-like estimators in the probabilistic PAC learning model" (in Japanese), IEICE Tech. Rep. IT92-128, 1993-03. pp. 1-6, 1993.