Information-Theoretic Asymptotics of Bayes Methods

IEEE Transactions on Information Theory

Volumn 36, Issue 3, 1990, Pages 453-471

  Clarke, Bertrand S ^a   Barron, Andrew R ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DECISION THEORY AND ANALYSIS; INFORMATION THEORY--DATA COMPRESSION; MATHEMATICAL TECHNIQUES--ESTIMATION;

BAYES METHODS; DENSITY ESTIMATION; HYPOTHESIS TESTING; PORTFOLIO SELECTION; RELATIVE ENTROPY;

PROBABILITY;

EID: 0025430804     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.54897     Document Type: Article

References (60)

1. J. Aitchison, “Goodness of prediction fit,” Biometrika, vol. 62, no. 3, pp. 547–554, Dec. 1975.
2. R. R. Bahadur, “Some limit theorems in statistics,” in. Proc. Regional Conf. Series Appl. Mathematics, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1971.
3. A. R. Barron, “Logically smooth density estimation,” Ph.D. dissertation, Elec. Eng., Stanford Univ., Stanford, CA, Aug. 1985.
4. A. R. Barron, “Are Bayes rules consistent in information?” in Problems in Communications and Computation (T. M. Cover and B. Gopinath, Eds.). New York: Springer-Verlag, 1987, 85–91.
5. A. R. Barron, “The exponential convergence of posterior probabilities with implications for Bayes estimators of density functions,” Tech. Report 7, Univ. of Illinois, Dept, of Statist., Apr. 1988.
6. A. R. Barron, “Uniformly powerful goodness of fit tests,” Ann. Statist., vol. 17, no. 1, pp. 107–124, Mar. 1989.
7. A. R. Barron and T. M. Cover, “A bound on the financial value of information,” IEEE Trans. Inform. Theory, vol. 34, pp. 1097–1100, Sept. 1988.
8. A. R. Barron and T. M. Cover, “Minimum complexity density estimation,” to appear in IEEE Trans. Inform. Theory.
9. R. H. Berk, “Limiting behavior of posterior distributions when the model is incorrect,” Ann. Statist., vol. 37, no. 1, pp. 51–58, Feb. 1966.
10. R. H. Berk, “Consistency a posterion, ” Ann. Statist., vol. 41, no. 3, pp. 894–906, June 1970.
11. P. Bickel and J. Yahav, “Some contributions to the asymptotic theory of Bayes’ solutions,” Z. Wahrsch. verw. Gebiete, vol. 11, pp. 257–276, 1969.
12. R. E. Blahut, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 1987.
13. L. Brown, Fundamentals of Statistical Exponential Families. Hay-ward, CA: Institute of Mathematical Statistics, 1986.
14. N. N. Cencov, Statistical Decision Rules and Optimal Inference. Providence: American Mathematical Society, 1981.
15. H. Chernoff, “On the distribution of the likelihood ratio,” Ann. Math. Statist., vol. 25, no. 3, pp. 573–578, Sept. 1954.
16. H. Chernoff, “Large sample theory: parametric case,” Ann. Math. Statist., vol. 27, no. 1, pp. 1–22, Mar. 1956.
17. K. L. Chung, A Course in Probability Theory. New York: Academic, 1974.
18. B. Clarke, “Asymptotic cumulative risk and Bayes’ risk under entropy loas, with applications,” Ph.D. dissertation, Dept. Statistics, Univ. of Illinois, Urbana-Cbampaign, 1989.
19. B. Clarke and A. R. Barron, “Information theoretic asymptotics of Bayes’ methods.” Univ. of Illinois, Department of Statistics Tech. Rep. 26, July 1989.
20. I. Csiszar, “Information-type measures of difference of probability distributions and individual observations,” Studia Sci. Math. Hungar., vol. 2, pp. 299–318, 1967.
21. L. D. Davisson, “Universal noiseless coding,” IEEE Trans. Inform. Theory, vol. IT-19, pp. 783–795, Nov. 1973.
22. N. G. De Bruijn, Asymptotic Methods in Analysis. New York: Dover, 1958.
23. M. De Groot, Optimal Statistical Decisions. New York: McGraw-Hill, 1970.
24. R. Gray, Probability, Random Processes, and Ergodic Properties. New York: Springer-Verlag, 1988.
25. J. Hartigan, Bayes ’ Theory. New York: Springer-Verlag, 1983.
26. D. Haughton, “On the choice of a model to fit data from an exponential family,” Ann. Statist., vol. 16, no. 1, pp. 342–355, Mar. 1988.
27. W. Hoeffding, “Probability inequalities for sums of bounded random variables,” J. Amer. Statist. Assoc., vol. 58, pp. 13–30, Mar. 1963.
28. W. Hoeffding and J. Wolfowitz, “Distinguishability of sets of distributions,” Ann. Math. Statist., vol. 29, no. 3, pp. 700–718, 1958.
29. I. bragimov and R. Hasminskii, Statistical Estimation: Asymptotic Theory. New York: Springer-Verlag, 1980.
30. R. A. Johnson, “Asymptotic expansions associated with the nth power of a density,” Ann. Statist., vol. 38, no. 4, pp. 1266–1272, Aug. 1967.
31. R. A. Johnson, “Asymptotic expansions associated with posterior distributions,” Ann. Statist., vol. 41, no. 3, 851–864, June 1970.
32. J. Kelly, “New interpretation of information rate,” Bell Syst. Tech. J., vol. 35, 917–926, July 1956.
33. J. C. Kieffer and J. Wolfowitz, “On the deviations of the empiric distribution function of vector chance variables,” Trans. Amer. Math. Soc., vol. 87, pp. 173–186, 1958.
34. J. C. Kieffer and J. Wolfowitz, “A counterexample to Perez’s generalization of the Shannon-McMillan theorem, Ann. Probab., vol. 1, no. 2, pp. 362–364, Apr. 1973; “Correction,” vol. 4, no. 1, pp. 153–154, Feb. 1976.
35. J. C. Kieffer and J. Wolfowitz, “A simple proof of the Moy-Perez generalization of the Shannon-McMillan theorem,” Pacific J. Math., vol. 51, pp. 203–206, 1974.
36. R. E. Krichevsky and V. K. Trofimov. The performance of universal encoding IEEE Trans. Inform. Throty, vol. IT-27, pp. 199-207. Mar. 1981.
37. S. Kullback. InfOrmation Theory and Statistics New York: Wiley 1959
38. S. Kullback, J. C. Keegel, and J. H. Kullback. Topics in Statistical Information Theory. Berlin: Springer-Verlag. 1980.
39. P. S. Laplace, Oevres. Paris: Imprimerie Royale, 1847, vol. 7.
40. L. Le Cam, On some asymptotic properties of maximum likelihood and related Bayes estimates, in University of California Publications in Statistics (J. Neyman, M. Loeve, and 0. Struve, Eds.). London: Cambridge University Press, 1953, pp. 277-329, vol. 1.
41. Les proprietes asymptotiques des solutions de Bayes Pirhl. Inst. Statist. Univ. Paris, vol. 7, pp. 18-35, 1958.
42. E. L. Lehmann, Theory of Point Estimation. New York: Wiley, 1983.
43. T. Leonard, Comment on A simple predictive density function J. Amer. Statist. Assoc., vol. 77. pp. 657-658, 1982.
44. D. Pollard, New ways to prove central limit theorems Econometric Theoq, vol. 1, pp. 295-314, 1985.
45. J. Rissanen, Universal coding, information, prediction, and estimation, IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
46. Stochastic complexity and modeling, Ann. Statist., vol. 14, pp. 1080-1100, Sept. 1986.
47. Stochastic complexity, J. Royal Statist. Soc., Ser. B, vol. 49, no. 3, pp. 223-239, 1987.
48. L. Schwartz, On Bayes consistency, Z. Wahrsch. verw. Gebiete vol. 4, pp. 10-26, 1965
49. G. Schwarz, Estimating the dimension of a model, Ann. Statist., vol. 6, no. 2, pp. 461-464, Mar. 1978.
50. S. M. Stigler, Laplaces 1774 memoir on inverse probability, Statistical Sci., vol. 1, no. 3, pp. 359-378, Aug. 1986.
51. H. Strasser, Consistency of maximum likelihood and Bayes estimates, Ann. Statist., vol. 9, no. 5, pp. 1107-1113, Sept. 1981.
52. L. Tierney and J. Kadane, Accurate approximations for posterior moments and marginal densities, Univ. of Minnesota Tech. Rep. 431, 1984.
53. L. Tierney and J. Kadane, Accurate approximations for posterior moments and marginal densities, J. Amer. Statist. Assoc., vol. 81, pp. 82-86, Mar. 1986.
54. V. Vapnik and A. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities, Theory Probah. Appl., vol. 16, pp. 264-280, 1971.
55. R. Von Mises, Mathematical Theory of Probability and Statistics. (H. Geiringer, Ed.). New York: Academic, 1964.
56. A. Wald, Tests of statistical hypotheses concerning several parameters when the number of observations is large, Trans. Amer. Math. Soc., vol. 54, pp. 426-482, Nov. 1943.
57. Note on the consistency of the maximum likelihood estimate, Ann. Math. Statist., vol. 20, no. 4, pp. 595-601, Dec. 1949.
58. A. M. Walker. On the asymptotic behaviour of posterior distributions, J. Roy. Statist. Soc., Ser. B, vol. 31, pp. 80-88, 1967.
59. S. S. Wilks, Mathematical Statistics New York: Wiley, 1962.
60. J. Wolfowitz, On Walds proof of the consistency of the maximum likelihood estimate, Ann. Math. Statist., vol. 20, no. 4, pp. 601-602. Dec. 1949.