Information-theoretic upper and lower bounds for statistical estimation

IEEE Transactions on Information Theory

Volumn 52, Issue 4, 2006, Pages 1307-1321

  Zhang, Tong ^a  

^a YAHOO INC   (United States)

Author keywords

Gibbs algorithm; Lower bound; Minimax; PAC Bayes; Randomized estimatin; Statistical estimation

Indexed keywords

ALGORITHMS; BOUNDARY VALUE PROBLEMS; ESTIMATION; RANDOM PROCESSES; STATISTICAL METHODS;

GIBBS ALGORITHM; LOWER BOUND; RANDOMIZED ESTIMATION; STATISTICAL ESTIMATION;

INFORMATION THEORY;

EID: 33645722194     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.864439     Document Type: Article

References (16)

1. T. Zhang, "Learning bounds for a generalized family of Bayesian posterior distributions," Proc. NIPS 03, 2004.
2. D. McAllester, "PAC-Bayesian model averaging," Proc. COLT'99, pp. 164-170, 1999.
3. O. Catoni, Statistical Learning Theory and Stochastic Optimization, vol. 1851, 2001. ser. Lecture Notes in Mathematics. Ecole d'Ete de Probabilites de Saint-Flour XXXI.
4. O. Catoni, "A PAC-Bayesian Approach to Adaptive Classification,", http://www.proba.jussieu.fr/users/catoni/ homepage/classif.pdf, Laboratoire de Probabilites et Modeles Aleatoires, Tech. Rep., 2003.
5. J.-Y. Audibert, "PAC-Bayesian Statistical Learning Theory," Ph.D. dissertation, Universit Paris VI, France, 2004.
6. T. Zhang, "From ε-entropy to KL-entropy: Analysis of minimum information complexity density estimation," Ann. Statist., to be published.
7. S. van de Geer, Empirical Processes in M-Estimation. Cambridge, U.K.: Cambridge Univ. Press, 2000.
8. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. New York: Springer-Verlag, 1996. Springer Series in Statistics.
9. Y. Yang and A. Barron, "Information-theoretic determination of minimax rates of convergence," Ann. Statist., vol. 27, pp. 1564-1599, 1999.
10. A. Barron and T. Cover, "Minimum complexity density estimation," IEEE Trans. Inf. Theory, vol. 37, no. 4, pp. 1034-1054, Jul. 1991.
11. R. Meir and T. Zhang, "Generalization error bounds for Bayesian mixture algorithms," J. Mach. Learn. Res., vol. 4, pp. 839-860, 2003.
12. M. Seeger, "PAC-Bayesian generalization error bounds for Gaussian process classification," J. Mach. Learn. Res., vol. 3, pp. 233-269, 2002.
13. T. S. Han and S. Verdú, "Generalizing the Fano inequality," IEEE Trans. Inf. Theory, vol. 40, no. 4, pp. 1247-1251, Jul. 1994.
14. R. Blahut, "Information bounds of the Fano-Kullback type," IEEE Trans. Inf. Theory, vol. IT-22, no. 4, pp. 410-421, Jul. 1976.
15. L. Birgé and P. Massart, "Rates of convergence for minimum contrast estimators," Probab. Theory Related Fields, vol. 97, no. 1-2, pp. 113-150, 1993.
16. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statist. Assoc., vol. 58, no. 301, pp. 13-30, Mar. 1963.