A matrix-free approach for solving the parametric gaussian process maximum likelihood problem

SIAM Journal on Scientific Computing

Volumn 34, Issue 1, 2012, Pages

  Anitescu, Mihai ^a   Chen, Jie ^a   Wang, Lei ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

Circulant preconditioner; Fast multipole method; Gaussian process; Maximum likelihood estimation; Preconditioned conjugate gradient; Ssample average approximation; Toeplitz system

Indexed keywords

COVARIANCE MATRIX; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); STOCHASTIC PROGRAMMING; STOCHASTIC SYSTEMS;

CIRCULANT PRECONDITIONER; FAST-MULTIPOLE METHODS; GAUSSIAN PROCESSES; MA XIMUM LIKELIHOODS; MATRIX FREE; MAXIMUM-LIKELIHOOD; MAXIMUM-LIKELIHOOD ESTIMATION; PRECONDITIONED CONJUGATE GRADIENT; SSAMPLE AVERAGE APPROXIMATIONS; TOEPLITZ SYSTEM;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 84861377864     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/110831143     Document Type: Article

References (34)

1. J. Barnes and P. Hut, A hierarchical O(N logN) force-calculation algorighm, Nature, 324 (1986), pp. 446-449.
2. R. H.-F. Chan and X.-Q. Jin, An Introduction to Iterative Toeplitz Solvers, SIAM, Philadelphia, 2007.
3. R. H. Chan, X.-Q. Jin, and M.-C. Yeung, The circulant operator in the Banach algebra of matrices, Linear Algebra Appl., 149 (1991), pp. 41-53.
4. T. F. Chan, An optimal circulant preconditioner for Toeplitz systems, SIAM J. Sci. Statist. Comput., 9 (1988), pp. 766-771.
5. J. Chen, M. Anitescu, and Y. Saad, Computing f(A)b via least squares polynomial approximations, SIAM J. Sci. Comput., 33 (2011), pp. 195-222.
6. J. Chiles and P. Delfiner, Geostatistics: Modeling Spatial Uncertainty, Wiley, New York, 1999.
7. N. Cressie, Statistics for Spatial Data, Wiley-Interscience, New York, 1993.
8. P. J. Diggle, J. A. Tawn, and R. A. Moyeed, Model-based geostatistics, Appl. Statist., 47 (1998), pp. 299-350.
9. Z. Duan and R. Krasny, An adaptive treecode for computing nonbounded potential energy in classical molecular systems, J. Comput. Chem., 23 (2001), pp. 1549-1571.
10. A. C. Faul, G. Goodsell, and M. J. D. Powell, A Krylov subspace algorithm for multiquadric interpolation in many dimensions, IMA J. Numer. Anal., 25 (2005), pp. 1-24. (Pubitemid 40083525)
11. R. Furrer, M. Genton, and D. Nychka, Covariance tapering for interpolation of large spatial datasets, J. Comput. Graph. Statist., 15 (2006), pp. 502-523. (Pubitemid 44566290)
12. M. Gibbs and D. MacKay, Efficient Implementation of Gaussian Processes, Technical report, Cavendish Laboratories, Cambridge, UK, 1997.
13. A. Gray and A. Moore, 'N-Body' problems in statistical learning, Adv. Neural Inf. Process. Syst., 2001, pp. 521-527.
14. N. A. Gumerov and R. Duraiswami, Fast radial basis function interpolation via preconditioned Krylov iteration, SIAM J. Sci. Comput., 29 (2007), pp. 1876-1899.
15. E. Haber, M. Chung, and F. J. Herrmann, An Effective Method for Parameter Estimation with PDE Constraints with Multiple Right Hand Sides, Technical report TR-2010-4, Earth and Ocean Sciences Department, University of British Columbia, Vancouver, Canada, 2010.
16. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, Springer, Berlin, 2003.
17. K. Heitmann, D. Higdon, C. Nakhleh, and S. Habib, Cosmic calibration, Astrophys. J., 646 (2006), pp. L1-L4. (Pubitemid 44258249)
18. D. Higdon, M. Kennedy, J. Cavendish, J. Cafeo, and R. Ryne, Combining field data and computer simulations for calibration and prediction, SIAM J. Sci. Comput., 26 (2004), pp. 448-466.
19. M. Hutchinson, A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines., Comm. Statist. Simulations Comput., 19 (1990), pp. 433-450.
20. M. Kennedy and A. O'Hagan, Bayesian calibration of complex computer models, J. Royal Statist. Soc. Ser. B, 63 (2001), pp. 425-464. (Pubitemid 33305774)
21. R. Krasny and L. Wang, Fast evaluation of multiquatric RBF sums by a Cartesian treecode, SIAM Sci. Comput., to appear.
22. K. Lindsay and R. Krasny, A particle method and adaptive treecode for vortex sheet motion in three-dimensional flow, J. Comput. Phys., 172 (2001), pp. 879-907. (Pubitemid 33082566)
23. P. Li, H. Johnston, and R. Krasny, A Cartesian treecode for screened Coulomb interactions, J. Comput. Phys., 228 (2009), pp. 3858-3868.
24. C. A. Micchelli, Interpolation of scattered data: Distance matrices and conditionally positive definite functions, Constr. Approx., 2 (1986), pp. 11-22.
25. A. A. Nikishin and A. Y. Yeremin, Variable block CG algorithms for solving large sparse symmetric positive definite linear systems on parallel computers, I: General iterative scheme, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 1135-1153.
26. J. Nocedal and S. Wright, Numerical Optimization, Springer, Berlin, 2006.
27. D. P. O'Leary, The block conjugate gradient algorithm and related methods, Linear Algebra Appl., 29 (1980), pp. 293-322.
28. C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning, MIT Press, Cambridge, MA, 2006.
29. K. P. Schmitt, M. Anitescu, and D. Negrut, Efficient sampling for spatial uncertainty quantification in multibody system dynamics applications, Internat. J. Numer. Methods Engrg., 80 (2009), pp. 537-564.
30. A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, MPS/SIAM Ser. Optim. 9, SIAM, Philadelphia, 2009.
31. J. Skilling, Bayesian numerical analysis, in Physics and Probability: Essays in Honor of Edwin T. Jaynes, Cambridge University, London, 1993, pp. 207-221.
32. M. Stein, Interpolation of Spatial Data: Some Theory for Kriging, Springer, Berlin, 1999.
33. J. Tsitsiklis, D. Bertsekas, and M. Athans, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automat. Control, 31 (1986), pp. 803-812. (Pubitemid 16629805)
34. E. E. Tyrtyshnikov, Optimal and superoptimal circulant preconditioners, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 459-173.