Upper and lower bounds on the learning curve for Gaussian processes

Machine Learning

Volumn 40, Issue 1, 2000, Pages 77-102

  Williams, Christopher K I ^a   Vivarelli, Francesco ^b,c  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

^b ASTON UNIVERSITY   (United Kingdom)

^c Edison Spa   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; ERRORS; GRAPH THEORY; MARKOV PROCESSES; MATRIX ALGEBRA; PROBABILITY DISTRIBUTIONS; RANDOM PROCESSES; VECTORS;

BAYESIAN INFERENCE; GAUSSIAN PROCESSES; LEARNING CURVE; LOWER BOUNDS; UPPER BOUNDS;

LEARNING SYSTEMS;

EID: 0033686947     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007601601278     Document Type: Article

References (29)

1. Adler, R. J. (1981). The Geometry of Random Fields. New York: John Wiley and Sons.
2. Akaike, H. (1974). A new look at statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.
3. Barber, D. & Williams, C. K. I. (1997). Gaussian processes for Bayesian classification via hybrid Monte Carlo. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in Neural Information Processing Systems, Vol. 9. MIT Press.
4. David, H. A. (1970). Order Statistics. New York: John Wiley and Sons.
5. Gradshteyn, E. S. & Ryzhik, I. M. (1993). Table of Integrals, Series and Products, 5th ed. New York: Academic Press.
6. Hansen, L. K. (1993). Stochastic linear learning: Exact test and training error averages. Neural Networks, 6, 393-396.
7. Hastie, T. J. & Tibshirani, R. J. (1990). Generalized Additive Models. London: Chapman and Hall.
8. Haussler, D. & Opper, M. (1997). Mutual information, metric entropy and cumulative relative entropy risk. The Annals of Statistics, 25(6), 2451.
9. Ihara, S. (1993). Information Theory. Singapore: World Scientific Publishing.
10. Matérn, B. (1986). Spatial Variation, 2nd ed. Berlin: Springer-Verlag. Lecture Notes in Statistics, Vol. 36.
11. Michelli, C. A. & Wahba, G. (1981). Design problems for optimal surface interpolation. In Z. Ziegler (Ed.), Approximation Theory and Applications (pp. 329-348). Academic Press.
12. Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion-determining the number of hidden units for artificial neural network models. IEEE Transactions on Neural Networks, 5, 865-872.
13. Neal, R. M. (1996). Bayesian Learning for Neural Networks. Springer. Lecture Notes in Statistics, Vol. 118.
14. Opper, M. (1997). Regression with gaussian processes: Average case performance. In M. W. Kwok-Yee, K. Irwin, & Y. Dit-Yan (Eds.), Theoretical Aspects of Neural Computation: A Multidisciplinary Perspective. Springer-Verlag.
15. Opper, M. & Vivarelli, F. (1999). General bounds on Bayes errors for regression with Gaussian Processes. In M. J. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in Neural Information Processing Systems, Vol. 11 (pp. 302-308). MIT Press.
16. Papoulis, A. (1991). Probability, Random Variables, and Stochastic Processes, 3rd ed. New York: McGraw-Hill.
17. Plaskota, L. (1996). Noisy Information and Computational Complexity. Cambridge: Cambridge University Press.
18. Qazaz, C. S., Williams, C. K. I., & Bishop, C. M. (1997). An upper bound on the Bayesian error bars for generalized linear regression. In S. W. Ellacott, J. C. Mason, & I. J. Anderson (Eds.), Mathematics of Neural Networks (pp. 295-299). Kluwer.
19. Rasmussen, C. E. (1996). Evaluation of Gaussian processes and other methods for non-linear regression. PhD Thesis, Department of Computer Science, University of Toronto, Toronto, Canada.
20. Ritter, K. (1996). Almost optimal differentiation using noisy data. Journal of Approximation Theory, 86(3), 293-309.
21. Ritter, K., Wasilkowski, G. W., & Wozniakowski, H. (1995). Multivariate integration and approximation for random fields satisfying Sacks-Ylvisaker conditions. Ann. Appl. Prob., 5, 518-540.
22. Silverman, B. W. (1985). Some aspects of the spline smoothing approach to non-parametric regression curve filtering. Journal of the Royal Statistical Society B, 47(1), 1-52.
23. Sollich, P. (1999). Learning curves for Gaussian processes. In M. J. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in Neural Information Processing Systems, Vol. 11 (pp. 344-350). MIT Press.
24. Stein, M. L. (1989). Comment on the paper by Sacks, J. et al. Design and Analysis of Computer Experiments. Statistical Science, 4(4):432-433.
25. Valiant, L. G. (1984). A theory of the learnable. Communication of the Association for Computing Machinery, 27, 1134-1142.
26. Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. New York: Springer-Verlag.
27. Vivarelli, F. (1998). Studies on generalisation in Gaussian processes and Bayesian neural networks. PhD Thesis, Neural Computing Research Group, Aston University, Birmingham, United Kingdom.
28. Whittle, P. (1963). Prediction and Regulation by Linear Least Square Methods. English Universities Press.
29. Williams, C. K. I. (1997). Computing with infinite networks. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in Neural Information Processing Systems Vol. 9. MIT Press.