Generalized Cross-Validation for Large-Scale Problems

Journal of Computational and Graphical Statistics

Volumn 6, Issue 1, 1997, Pages 1-34

  Golub, Gene H ^a   Matt, Urs Von ^b  

^a Stanford University ^*  (United States)

^b Centro Svizzero Di Calcolo Scientifico   (Switzerland)

Author keywords

Gauss quadrature; Generalized cross validation; Ill posed problems; Lanczos algorithm; Stochastic trace estimator; Tikhonov regularization

Indexed keywords

EID: 0031498439     PISSN: 10618600     EISSN: 15372715     Source Type: Journal    
DOI: 10.1080/10618600.1997.10474725     Document Type: Article

References (45)

1. Bai, Z., Fahey, M., and Golub, G. H. (1996), “Some Large Scale Matrix Computation Problems,” Journal of Computational and Applied Mathematics, 74, 71-89.
2. Berman, M. (1994), “Automated Smoothing of Image and Other Regularly Spaced Data,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 16, 460-468.
3. Bemardson, S., McCarty, P., and Thron, C. (1994), “Monte Carlo Methods for Estimating Linear Combinations of Inverse Matrix Entries in Lattice QCD,” Computer Physics Communications, 78, 256-264.
4. Burrage, K., Williams, A., Erhel, J., and Pohl, B. (1994), “The Implementation of a Generalized Cross Validation Algorithm Using Deflation Techniques for Linear Systems,” Technical Report No. 94-05, Seminar für Angewandte Mathematik, Eidgenössische Technische Hochschule, Zürich, Switzerland.
5. Craven, P., and Wahba, G. (1979), “Smoothing Noisy Data with Spline Functions,” Numerische Mathematik, 31, 377-403.
6. Davis, P. J., and Rabinowitz, P. (1984), Methods of Numerical Integration, Orlando, FL: Academic Press.
7. Elddn, L. (1977), “Algorithms for the Regularization of Ill-Conditioned Least Squares Problems,” BIT, 17, 134-145.
8. Elddn, L. (1984), “A Note on the Computation of the Generalized Cross-Validation Function for Ill-Conditioned Least Squares Problems,” BIT, 24, 467-472.
9. Gander, W. (1978), “On the Linear Least Squares Problem with a Quadratic Constraint,” Habilitationsschrift ETH Zürich, STAN-CS-78-697, Stanford University.
10. Girard, D. A. (1989), “A Fast ‘Monte-Carlo Cross-Validation’ Procedure for Large Least Squares Problems with Noisy Data,” Numerische Mathematik, 56, 1-23.
11. Girard, D. A. (1991), “Asymptotic Optimality of the Fast Randomized Versions of GCV and Cl in Ridge Regression and Regularization,” The Annals of Statistics, 19, 1950-1963.
12. Girard, D. A. (1995), “The Fast Monte-Carlo Cross-Validation and Cl Procedures: Comments, New Results and Application to Image Recovery Problems,” Computational Statistics, 10, 205-231.
13. Golub, G. H. (1973), “Some Modified Matrix Eigenvalue Problems,” SIAM Review, 15, 318-334.
14. Golub, G. H., Heath, M., and Wahba, G. (1979), “Generalized Cross-Validation as a Method for Choosing a Good Ridge Parameter,” Technometrics, 21, 215-223.
15. Golub, G. H., and Kahan, W. (1965), “Calculating the Singular Values and Pseudo-Inverse of a Matrix,” SIAM Journal on Numerical Analysis, 2, 205-224.
16. Golub, G. H., and Van Loan, C. F. (1989), Matrix Computations (2nd ed.), Baltimore, MD: The Johns Hopkins University Press.
17. Golub, G. H., and Welsch, J. H. (1969), “Calculation of Gauss Quadrature Rules,” Mathematics of Computation, 23, 221-230.
18. Gu, C., Bates, D. M., Chen, Z., and Wahba, G. (1989), “The Computation of Generalized Cross-Validation Functions Through Householder Tridiagonalization With Applications to the Fitting of Interaction Spline Models,” SIAM Journal on Matrix Analysis and Applications, 10, 457-480.
19. Hansen, P. C. (1994), “Regularization Tools: A Matlab Package for Analysis and Solution of Discrete Ill-Posed Problems,” Numerical Algorithms, 6, 1-35.
20. Hutchinson, M. F. (1986), “Algorithm 642: A Fast Procedure for Calculating Minimum Cross-Validation Cubic Smoothing Splines,” ACM Transactions on Mathematical Software, 12, 150-153.
21. Hutchinson, M. F. (1990), “A Stochastic Estimator of the Trace of the Influence Matrix for Laplacian Smoothing Splines,” Communications in Statistics, Simulation and Computation, 19, 433-450.
22. Hutchinson, M. F., and de Hoog, F. R. (1985), “Smoothing Noisy Data with Spline Functions,” Numerische Mathematik, 47, 99-106.
23. Jagiello, J. (1994), “Stable Numerical Solution of the Adsorption Integral Equation Using Splines,” Langmuir, 10, 2778-2785.
24. Krylov, V. I. (1962), Approximate Calculation of Integrals, translated by A. H. Stroud, New York: Macmillan.
25. Lu, H., and Mathis, F. H. (1992), “Surface Approximation by Spline Smoothing and Generalized CrossValidation,” Mathematics and Computers in Simulation, 34, 541-549.
26. Morozov, V. A. (1984), Methods for Solving Incorrectly Posed Problems, New York: Springer.
27. O’Sullivan, F. (1991), “Sensitivity Analysis for Regularized Estimation in Some System Identification Problems,” SIAM Journal on Scientific and Statistical Computing, 12, 1266-1283.
28. Schumaker, L. L., and Utreras, F. I. (1990), “On Generalized Cross Validation for Tensor Smoothing Splines,” SIAM Journal on Scientific and Statistical Computing, 11, 713-731.
29. Shahraray, B., and Anderson, D. J. (1989), “Optimal Estimation of Contour Properties by Cross-Validated Regularization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 600-610.
30. Sidje, R., and Williams, A. (in press), “Fast Generalised Cross Validation,” SIAM Journal of Numerical Analysis.
31. Stewart, G. W. (1993), “Determining Rank in the Presence of Error,” in Linear Algebra for Large Scale and Real-Time Applications, eds. M. S. Moonen, G. H. Golub, and B. L. R. De Moor, NATO ASI Series, Series E: Applied Sciences, Vol. 232, Kluwer, pp. 275-291.
32. Stieltjes, T. J. (1914), “Quelques recherches sur la théorie des quadratures dites mécaniques,” in Oeuvres Complètes de Thomas Jan Stieltjes, (Vol. 1), Société Mathématique d’Amsterdam, pp. 377-394.
33. Stroud, A. H., and Secrest, D. (1966), Gaussian Quadrature Formulas, Englewood Cliffs, NJ: Prentice-Hall.
34. The MathWorks Inc. (1992), MATLAB, High-Performance Numeric Computation and Visualization Software, Natick, MA: author.
35. Thompson, A. M. (1992), “Estimating the Sun’s Rotation From Solar Oscillations by Régularisation,” Astronomy and Astrophysics, 265, 289-295.
36. Thompson, A. M., Brown, J. C., Kay, J. W., and Titterington, D. M. (1991), “A Study of Methods of Choosing the Smoothing Parameter in Image Restoration by Regularization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 13, 326-339.
37. Utreras, F. I. (1980), “Sur le choix du paramètre d’ajustement dans le lissage par fonctions spline,” Numerische Mathematik, 34, 15-28.
38. Utreras, F. I. (1981), “Optimal Smoothing of Noisy Data Using Spline Functions,” SIAM Journal on Scientific and Statistical Computing, 2, 349-362.
39. Varah, J. M. (1983), “Pitfalls in the Numerical Solution of Linear Ill-Posed Problems,” SIAM Journal on Scientific and Statistical Computing, 4, 164-176.
40. von Matt, U. (1993), Large Constrained Quadratic Problems, Zürich: Verlag der Fachvereine.
41. Wahba, G. (1978), “Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression,” Journal of the Royal Statistical Society, Ser. B, 40, 364-372.
42. Wahba, G. (1980), “Spline Bases, Regularization, and Generalized Cross Validation for Solving Approximation Problems With Large Quantities of Noisy Data,” in Approximation Theory III, ed. E. W. Cheney, New York: Academic Press, pp. 905-912.
43. Wahba, G. (1990), “Spline Models for Observational Data,” CBMS-NSF, Regional Conference Series in Applied Mathematics (Vol. 59), Philadelphia, PA: SIAM.
44. Wahba, G., Johnson, D. R., Gao, F., and Gong, J. (1995), “Adaptive Tuning of Numerical Weather Prediction Models: Randomized GCV in Three- and Four-Dimensional Data Assimilation,” Monthly Weather Review, 123, 3358-3369 (also available by anonymous ftp from ftp.stat.wisc.edu in pub/wahba).
45. Wahba, G., and Wendelberger, J. (1980), “Some New Mathematical Methods for Variational Objective Analysis Using Splines and Cross Validation,” Monthly Weather Review, 108, 1122-1143.