Efficient Gaussian process regression for large datasets

Biometrika

Volumn 100, Issue 1, 2013, Pages 75-89

  Banerjee, Anjishnu ^a   Dunson, David B ^a   Tokdar, Surya T ^a  

^a Duke University   (United States)

Author keywords

Bayesian regression; Compressive sensing; Dimensionality reduction; Gaussian process; Random projection

Indexed keywords

EID: 84874776816     PISSN: 00063444     EISSN: 14643510     Source Type: Journal    
DOI: 10.1093/biomet/ass068     Document Type: Article

References (37)

1. ADLER, R. J. (1990). An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes, vol. 12 of IMS Lecture Notes-Monograph Series. Hayward: Institute of Mathematical Statistics, pp. 75-6.
2. BANERJEE, S.,GELFAND, A. E., FINLEY, A. O. & SANG, H. (2008). Gaussian predictive process models for large spatial data sets. J. R. Statist. Soc. B 70, 825-48.
3. BHATIA, R. (1997). Matrix Analysis. New York: Springer.
4. CANDES, E. J., ROMBERG, J. & TAO, T. (2006). Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Info. Theory 52, 489-509.
5. CHOI, T. & SCHERVISH, M. J. (2007). On posterior consistency in nonparametric regression problems. J. Mult. Anal. 98, 1969-87.
6. CRESSIE, N. & JOHANNESSON, G. (2008). Fixed rank kriging for very large spatial data sets. J. R. Statist. Soc. B 70, 209-26.
7. CSAT ́O, L. & OPPER, M. (2002). Sparse on-line Gaussian processes. Neural Comp. 14, 641-68.
8. DASGUPTA, S. & GUPTA, A. (2003). An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algor. 22, 60-5.
9. DIXON, J. D. (1983). Estimating extremal eigenvalues and condition numbers of matrices. SIAM J. Numer. Anal. 20, 812-4.
10. DOKOVI'C, D. (1991). Unitary similarity of projectors. Aequationes Math. 42, 220-4.
11. DONOHO, D. L. (2006). Compressed sensing. IEEE Trans. Info. Theory 52, 1289-306.
12. DRINEAS, P. & MAHONEY, M. W. (2005). On the Nystr?om method for approximating a Gram matrix for improved kernel-based learning. J. Mach. Learn. Res. 6, 2153-75.
13. FINLEY, A. O., SANG, H., BANERJEE, S. & GELFAND, A. E. (2009). Improving the performance of predictive process modeling for large datasets. Comp. Statist. Data Anal. 53, 2873-84.
14. FOSTER, L., WAAGEN, A., AIJAZ, N., HURLEY, M., LUIS, A., RINSKY, J., SATYAVOLU, C., WAY, M. J., GAZIS, P. & SRIVASTAVA, A. (2009). Stable and efficient Gaussian process calculations. J. Mach. Learn. Res. 10, 857-82.
15. FRANK, A. & ASUNCION, A. (2010). UCI Machine Learning Repository. School of Information and Computer Sciences, University of California, Irvine. URL: http://archive.ics.uci.edu/ml.
16. FRAUENFELDER, P., SCHWAB, C. & TODOR, R. A. (2005). Finite elements for elliptic problems with stochastic coefficients. Comp. Meth. Appl. Mech. Eng. 194, 205-28.
17. GHANEM, R. & SPANOS, P. D. (2003). Stochastic Finite Elements: A Spectral Approach. New York: Dover Publications, revised ed.
18. GOLIGHTLY, A.&WILKINSON, D. J. (2006). Bayesian sequential inference for nonlinear multivariate diffusions. Statist. Comp. 16, 323-38.
19. GRIGORIU, M. (2002). Stochastic Calculus: Applications in Science and Engineering. Boston: Birkh?auser.
20. GUHANIYOGI, R., FINLEY, A. O., BANERJEE, S. & GELFAND, A. E. (2011). Adaptive Gaussian predictive process models for large spatial datasets. Environmetrics 22, 997-1007.
21. HALKO, N.,MARTINSSON, P. G. & TROPP, J. A. (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev. 53, 217-88.
22. HARVILLE, D. A. (2008). Matrix Algebra from a Statistician's Perspective. New York: Springer, reprinted ed. JOHNSON, W. B.
23. LINDENSTRAUSS, J. & SCHECHTMAN, G. (1986). Extensions of Lipschitz maps into Banach spaces. Israel J. Math. 54, 129-38.
24. KEERTHI, S. & CHU, W. (2006). A matching pursuit approach to sparse Gaussian process regression. Adv. Neural Info. Proces. Syst. 18, 643-50.
25. PLUMMER, M., BEST, N., COWLES, K. & VINES, K. (2006). CODA: Convergence diagnosis and output analysis for MCMC. R News 6, 7-11.
26. QUI ?NONERO CANDELA, J. & RASMUSSEN, C. E. (2005). A unifying view of sparse approximate Gaussian process regression. J. Mach. Learn. Res. 6, 1939-59.
27. R DEVELOPMENT CORE TEAM (2012). R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing, ISBN 3-900051-07-0, http://www.R-project.org.
28. SARLOS, T. (2006). Improved approximation algorithms for large matrices via random projections. In Proc. 47th Ann. IEEE Symp. Found. Comp. Sci. pp. 143-52.
29. SCHWAIGHOFER, A. & TRESP, V. (2002). Transductive and inductive methods for approximate Gaussian process regression. Adv. Neural Info. Proces. Syst. 15, 953-60.
30. SEEGER, M.,WILLIAMS, C. K. I. & LAWRENCE, N. D. (2003). Fast forward selection to speed up sparse Gaussian process regression. In Proc. Ninth Int. Workshop Artif. Intel. Statist. Society for Artificial Intelligence and Statistics, pp. 2003-10.
31. SMOLA, A. J. & BARTLETT, P. (2001). Sparse greedy Gaussian process regression. Adv. Neural Info. Proces. Syst. 13, 619-25.
32. SNELSON, E. & GHAHRAMANI, Z. (2006). Sparse Gaussian processes using pseudo-inputs. Adv. Neural Info. Proces. Syst. 18, 1257-64.
33. STEIN, M. (1999). Interpolation of Spatial Data: Some Theory for Kriging. New York: Springer.
34. STEWART, G. W. (1993). On the early history of the singular value decomposition. SIAM Rev. 35, 551-66.
35. TOKDAR, S. T. (2007). Towards a faster implementation of density estimation with logistic Gaussian process priors. J. Comp. Graph. Statist. 16, 633-55.
36. VAN DER VAART, A. W. & VAN ZANTEN, J. H. (2008). Rates of contraction of posterior distributions based on Gaussian process priors. Ann. Statist. 36, 1435-63.
37. VIJAYAKUMAR, S. & SCHAAL, S. (2000). Locally weighted projection regression: An O(n) algorithm for incremental real time learning in high dimensional space. In Proc. 17th Int. Conf. Mach. Learn. San Francisco: Morgan Kaufmann Publishers, pp. 1079-86.