Information theory and superefficiency

Annals of Statistics

Volumn 26, Issue 5, 1998, Pages 1800-1825

  Barron, Andrew ^a   Hengartner, Nicolas ^a  

^a YALE UNIVERSITY   (United States)

Author keywords

Data compression; Information theory; Kullback Leibler loss; Superefficiency

Indexed keywords

EID: 0032259025     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1024691358     Document Type: Article

References (28)

1. ALI, S. and SILVEY, S. (1966). A general class of coefficient of divergence of one distribution from another. J. Roy. Statist. Soc. Ser. B 28 131-142.
2. BAHADUR, R. R. (1964). On Fisher's bound for asymptotic variances. Ann. Math. Statist. 35 1545-1552.
3. BAHADUR, R. R. (1967). Rates of convergence of estimates and test statistics. Ann. Math. Statist. 38 303-324.
4. BAHADUR, R. R. (1971). Some Limit Theorems in Statistics. SIAM, Philadelphia.
5. BARRON, A. (1987). Are Bayes rules consistent in information? In Open Problems in Communica-tion and Computation (T. Cover and B. Gopinath. eds.) 85-91. Springer, New York.
6. BARRON, A. and COVER, T. (1991). Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034-1054.
7. BIRGE, L. (1986). On estimating a density using Hellinger distance and some other strange facts. Probab. Theory Related Fields 71 271-291.
8. BROWN, L. D. (1993). An information inequality for the Bayes risk under truncated squared error loss. In Multivariate Analysis: Future Directions (C. R. Rao, ed.). North-Holland, Amsterdam.
9. BROWN, L. D., LOW. M. G. and ZHAO, L. H. (1997). Superefficiency in nonparametric function estimation. Ann. Statist. 25 2607-2625.
10. CENCOV, N. N. (1982). Statistical decision rules and optimal inference. Amer. Math. Soc. Transl. Ser. 2 53.
11. CLARKE, B. and BARRON, A. (1990). Information-theoretic asymptotics of Bayes methods. IEEE Trans. Inform. Theory 36 453-471.
12. COVER, T. and THOMAS J. (1991). Information Theory. Wiley, New York.
13. CSISZÁR, I. (1967). Information-type measures of difference of probability distributions and indirect observations. Stadia Sci. Math. Hungar. 2 299-318.
14. HALMOS, P. R. (1988). Measure Theory, 4th, ed. Springer, New York.
15. HARTIGAN, J. A. (1983). Bayes Theory. Springer, New York.
16. HARTIGAN, J. A. (1998). The maximum likelihood prior. Ann. Statist. 26 2083-2103.
17. JEFFREYS, H. (1946). An invariant form for the prior probability in estimation problems. Proc. Roy. Soc. London Ser. A 186 453-461.
18. KOMAKI, F. (1994). On asymptotic properties of predictive distributions. Technical Report, Dept. of Mathematical Engineering and Information Physics, Faculty of Engineering, Univ. Tokyo.
19. KULLBACK, S. and LEIBLER, R. (1951). Information and sufficiency. Ann. Math. Statist. 22 79-86.
20. IBRAGIMOV, I. A. and HASMINSKII, R. Z. (1982). Statistical Estimation: Asymptotical Theory. Applications of Mathematics. Springer, New York.
21. LE CAM, L. (1953). On some asymptotic properties of maximum likelihood estimates and related Bayes estimates. Univ. California Publ. Statist. 277-330.
22. LE CAM, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York.
23. LORENTZ (1966). Metric entropy and approximation. Bull. Amer. Math. Soc. 72 903-937.
24. MERHAV, N. and FEDER, M. (1995). A strong version of the redundancy-capacity theorem of universal coding. IEEE Trans. Inform. Theory 41 714-722.
25. RISSANEN, J. (1984). Universal coding, information, prediction, and estimation. IEEE Trans. Inform. Theory 30 629-636.
26. RISSANEN, J. (1986). Stochastic complexity and modeling. Ann. Statist. 14 1080-1100.
27. VOVK (1991). Asymptotic efficiency of estimators: algorithmic approach. Theory Probab. Appl. 36 329-343.
28. YANG, Y. and BARRON, A. (1995) Information theoretic determination of minimax rates of convergence Unpublished manuscript.