Information consistency of nonparametric Gaussian process methods

IEEE Transactions on Information Theory

Volumn 54, Issue 5, 2008, Pages 2376-2382

  Seeger, Matthias W ^a   Kakade, Sham M ^b   Foster, Dean P ^c  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Bayesian prediction; Eigenvalue asymptotics; Gaussian process; Information consistency; Nonparametric statistics; Online learning; Posterior consistency; Regret bound

Indexed keywords

ASYMPTOTIC ANALYSIS; CONVERGENCE OF NUMERICAL METHODS; EIGENVALUES AND EIGENFUNCTIONS; HILBERT SPACES; MATHEMATICAL MODELS;

BAYESIAN PREDICTION; EIGENVALUE ASYMPTOTICS; INFORMATION CONSISTENCY; NONPARAMETRIC GAUSSIAN PROCESS METHODS; NONPARAMETRIC STATISTICS; ONLINE LEARNING;

INFORMATION THEORY;

EID: 43749083506     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2007.915707     Document Type: Article

References (25)

1. P. Diaconis and D. Freedman, "On the consistency of Bayes estimates," Ann. Statist., vol. 14, pp. 1-26, 1986.
2. B. Clarke and A. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inf. Theory, vol. 36, no. 3, pp. 453-471, May 1990.
3. A. Barron, "Information-theoretic characterization of Bayes performance and the choice of priors in parametric and nonparametric problems," in Bayesian Statistics 6, J. Bernardo, J. Berger, A. Dawid, and A. Smith, Eds., 1998, pp. 27-52.
4. A. Barron, M. Schervish, and L. Wasserman, "The consistency of posterior distributions in nonparametric problems," Ann. Statist. vol. 2, no. 27, pp. 536-561, 1999.
5. X. Shen and L. Wasserman, "Rates of convergence of posterior distributions," Ann. Statist., vol. 29, no. 3, pp. 687-714, 2001.
6. H. Widom, "Asymptotic behavior of the eigenvalues of certain integral equations," Trans. Amer. Math. Soc., vol. 109, no. 2, pp. 278-295, 1963.
7. D. Haussler and M. Opper, "Mutual information, metric entropy and cumulative relative entropy risk," Ann. Statist., vol. 25, no. 6, pp. 2451-2492, 1997.
8. N. Cesa-Bianchi and G. Lugosi, "Worst case prediction over sequences under log loss," Mach. Learn., vol. 43, 2001.
9. M. Seeger, "Gaussian processes for machine learning," Int. J. Neural Syst., vol. 14, no. 2, pp. 69-106, 2004.
10. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press, 2006.
11. M. Stein, Interpolation of Spatial Data: Some Theory for Kriging. New York: Springer, 1999.
12. A. O'Hagan, "Curve fitting and optimal design," J. Roy. Stat. Soc. B, vol. 40, no. 1, pp. 1-42, 1978.
13. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, 1950.
14. G. Wahba, Spline Models for Observational Data, ser. CBMS-NSF Regional Conference Series. Philadelphia, PA: SIAM, 1990.
15. S. Kakade, M. Seeger, and D. Foster, "Worst-case bounds for Gaussian process models," in Advances in Neural Information Processing Systems 18, Y. Weiss, B. Schölkopf, and J. Platt, Eds. Cambridge, MA: MIT Press, 2006.
16. S. Kakade and A Ng, "Online bounds for Bayesian algorithms," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004.
17. M. Opper and F. Vivarelli, "General bounds on Bayes errors for regression with Gaussian processes," in Advances in Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn, Eds. Cambridge, MA: MIT Press, 1999.
18. H. Zhu, C. K. I. Williams, R. Rohwer, and M. Morciniec, "Gaussian regression and optimal finite dimensional linear models," in Neural Networks and Machine Learning, ser. NATO ASI, C. Bishop, Ed. New York: Springer, 1998, vol. 168.
19. T. Zhang, "Learning bounds for a generalized family of Bayesian posterior distributions," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004.
20. T. Zhang, "From ε-entropy to KL-entropy: Analysis of minimum information complexity density estimation," Ann. Statist., vol. 34, pp. 2180-2210, 2006.
21. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2002.
22. R. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
23. M. Shervish, Theory of Statistics. New York: Springer, 1995.
24. I. Gradshteyn and I. Ryzhik, Table of Integral, Series, and Products 6th ed. San Diego, CA: Academic, 2000.
25. M. Seeger, Addendum to: Information Consistency of Nonparametric Gaussian Process Methods Max Planck Institute for Biological Cybernetics, Tübingen, Germany, 2007, Tech. Rep.