Divide and conquer kernel ridge regression

Journal of Machine Learning Research

Volumn 30, Issue , 2013, Pages 592-617

  Zhang, Yuchen ^a   Duchi, John ^a   Wainwright, Martin ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

OPTIMIZATION;

COMPUTATION TIME; DIVIDE AND CONQUER; GAUSSIAN KERNELS; KERNEL RIDGE REGRESSIONS; OPTIMAL RATE; SAMPLE SIZES; SCALABLE APPROACH; SUBSTANTIAL REDUCTION;

REGRESSION ANALYSIS;

EID: 84898041590     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (33)

1. P. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher complexities. Annals of Statistics, 33(4): 1497-1537, 2005.
2. A. Berlinet and C. Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer Academic, 2004.
3. R. Bhatia. Matrix Analysis. Springer, 1997.
4. α. Sbornik: Mathematics, 2(3): 295-317, 1967.
5. G. Blanchard and N. Krämer. Optimal learning rates for kernel conjugate gradient regression. In Advances in Neural Information Processing Systems 24, 2010.
6. A. Caponnetto and E. De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3): 331-368, 2007.
7. R. Chen, A. Gittens, and J. A. Tropp. The masked sample covariance estimator: an analysis using matrix concentration inequalities. Information and Inference, to appear, 2012.
8. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2: 243-264, 2002.
9. C. Gu. Smoothing spline ANOVA models. Springer, 2002.
10. L. Gyorfi, M. Kohler, A. Krzyzak, and H. Walk. A Distribution-Free Theory of Nonparametric Regression. Springer Series in Statistics. Springer, 2002.
11. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001.
12. A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12: 55-67, 1970.
13. A. Kleiner, A. Talwalkar, P. Sarkar, and M. Jordan. Bootstrapping big data. In Proceedings of the 29th International Conference on Machine Learning, 2012.
14. V. Koltchinskii. Local Rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics, 34(6): 2593-2656, 2006.
15. M. Ledoux and M. Talagrand. Probability in Banach Spaces. Springer, 1991.
16. D. Luenberger. Optimization by Vector Space Methods. Wiley, 1969.
17. R. McDonald, K. Hall, and G. Mann. Distributed training strategies for the structured perceptron. In North American Chapter of the Association for Computational Linguistics (NAACL), 2010.
18. S. Mendelson. Geometric parameters of kernel machines. In Proceedings of COLT, pages 29-43, 2002.
19. G. Raskutti, M. Wainwright, and B. Yu. Early stopping for non-parametric regression: An optimal data-dependent stopping rule. In 49th Annual Allerton Conference on Communication, Control, and Computing, pages 1318-1325, 2011.
20. G. Raskutti, M. J. Wainwright, and B. Yu. Minimax-optimal rates for sparse additive models over kernel classes via convex programming. Journal of Machine Learning Research, 12: 389-427, March 2012.
21. C. Saunders, A. Gammerman, and V. Vovk. Ridge regression learning algorithm in dual variables. In Proceedings of the 15th International Conference on Machine Learning, pages 515-521. Morgan Kaufmann, 1998.
22. B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. IEEE Transactions on Information Theory, 10(5): 1299-1319, 1998.
23. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
24. I. Steinwart, D. Hush, and C. Scovel. Optimal rates for regularized least squares regression. In Proceedings of the 22nd Annual Conference on Learning Theory, pages 79-93, 2009.
25. C. J. Stone. Optimal global rates of convergence for non-parametric regression. Annals of Statistics, 10(4): 1040-1053, 1982.
26. A. B. Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009.
27. S. van de Geer. Empirical Processes in M-Estimation. Cambridge University Press, 2000.
28. G. Wahba. Spline models for observational data. CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, PN, 1990.
29. L. Wasserman. All of Nonparametric Statistics. Springer, 2006.
30. C. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems 14, pages 682-688, 2001.
31. Y. Yao, L. Rosasco, and A. Caponnetto. On early stopping in gradient descent learning. Constructive Approximation, 26(2): 289-315, 2007.
32. T. Zhang. Learning bounds for kernel regression using effective data dimensionality. Neural Computation, 17(9): 2077-2098, 2005.
33. Y. Zhang, J. C. Duchi, and M. J. Wainwright. Communication-efficient algorithms for statistical optimization. In Advances in Neural Information Processing Systems 26, 2012.