Less is more: Nyström computational regularization

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 1657-1665

  Rudi, Alessandro ^a   Camoriano, Raffaello ^a,b   Rosasco, Lorenzo ^a,c  

^a UNIVERSITY OF GENOA   (Italy)

^b ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INFORMATION SCIENCE;

EXPERIMENTAL ANALYSIS; HIGH PROBABILITY; KERNEL METHODS; LARGE-SCALE DATASETS; RANDOM SAMPLING; REGULARIZED LEAST SQUARES; STATE-OF-THE-ART PERFORMANCE; STATISTICAL LEARNING;

SAMPLING;

EID: 84965167586     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning). MIT Press, 2002.
2. Alex J. Smola and Bernhard Schölkopf. Sparse Greedy Matrix Approximation for Machine Learning. In ICML, pages 911-918. Morgan Kaufmann, 2000.
3. C. Williams and M. Seeger. Using the Nyström Method to Speed Up Kernel Machines. In NIPS, pages 682-688. MIT Press, 2000.
4. Ali Rahimi and Benjamin Recht. Random Features for Large-Scale Kernel Machines. In NIPS, pages 1177-1184. Curran Associates, Inc., 2007.
5. J. Yang, V. Sindhwani, H. Avron, and M. W. Mahoney. Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels. In ICML, volume 32 of JMLR Proceedings, pages 485-493. JMLR.org, 2014.
6. Quoc V. Le, Tamás Sarlós, and Alexander J. Smola. Fastfood - Computing Hilbert Space Expansions in loglinear time. In ICML, volume 28 of JMLR Proceedings, pages 244-252. JMLR.org, 2013.
7. Si Si, Cho-Jui Hsieh, and Inderjit S. Dhillon. Memory Efficient Kernel Approximation. In ICML, volume 32 of JMLR Proceedings, pages 701-709. JMLR.org, 2014.
8. Yuchen Zhang, John C. Duchi, and Martin J. Wainwright. Divide and Conquer Kernel Ridge Regression. In COLT, volume 30 of JMLR Proceedings, pages 592-617. JMLR.org, 2013.
9. S. Kumar, M. Mohri, and A. Talwalkar. Ensemble Nystrom Method. In NIPS, pages 1060-1068, 2009.
10. Mu Li, James T. Kwok, and Bao-Liang Lu. Making Large-Scale Nyström Approximation Possible. In ICML, pages 631-638. Omnipress, 2010.
11. Kai Zhang, Ivor W. Tsang, and James T. Kwok. Improved Nyström Low-rank Approximation and Error Analysis. ICML, pages 1232-1239. ACM, 2008.
12. Bo Dai, Bo Xie 0002. Niao He, Yingyu Liang, Anant Raj, Maria-Florina Balcan, and Le Song. Scalable Kernel Methods via Doubly Stochastic Gradients. In NIPS, pages 3041-3049, 2014.
13. Petros Drineas and Michael W. Mahoney. On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning. JMLR, 6:2153-2175, December 2005.
14. A. Gittens and M. W. Mahoney. Revisiting the Nystrom method for improved large-scale machine learning. 28:567-575, 2013.
15. Shusen Wang and Zhihua Zhang. Improving CUR Matrix Decomposition and the Nyström Approximation via Adaptive Sampling. JMLR, 14(1):2729-2769, 2013.
16. Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, and David P. Woodruff. Fast approximation of matrix coherence and statistical leverage. JMLR, 13:3475-3506, 2012.
17. Michael B. Cohen, Yin Tat Lee, Cameron Musco, Christopher Musco, Richard Peng, and Aaron Sidford. Uniform Sampling for Matrix Approximation. In ITCS, pages 181-190. ACM, 2015.
18. Shusen Wang and Zhihua Zhang. Efficient Algorithms and Error Analysis for the Modified Nystrom Method. In AISTATS, volume 33 of JMLR Proceedings, pages 996-1004. JMLR.org, 2014.
19. S. Kumar, M. Mohri, and A. Talwalkar. Sampling methods for the Nyström method. JMLR, 13(1):981-1006, 2012.
20. Corinna Cortes, Mehryar Mohri, and Ameet Talwalkar. On the Impact of Kernel Approximation on Learning Accuracy. In AISTATS, volume 9 of JMLR Proceedings, pages 113-120. JMLR.org, 2010.
21. R. Jin, T. Yang, M. Mahdavi, Y. Li, and Z. Zhou. Improved Bounds for the Nyström Method With Application to Kernel Classification. Information Theory, IEEE Transactions on, 59(10), Oct. 2013.
22. Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, and Zhi-Hua Zhou. Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison. In NIPS, pages 485-493, 2012.
23. Francis Bach. Sharp analysis of low-rank kernel matrix approximations. In COLT, volume 30, 2013.
24. A. Alaoui and M. W. Mahoney. Fast randomized kernel methods with statistical guarantees. arXiv, 2014.
25. I. Steinwart and A. Christmann. Support Vector Machines. Springer New York, 2008.
26. Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3):331-368, 2007.
27. L. Lo Gerfo, Lorenzo Rosasco, Francesca Odone, Ernesto De Vito, and Alessandro Verri. Spectral Algorithms for Supervised Learning. Neural Computation, 20(7):1873-1897, 2008.
28. I. Steinwart, D. Hush, and C. Scovel. Optimal rates for regularized least squares regression. In COLT, 2009.
29. S. Mendelson and J. Neeman. Regularization in kernel learning. The Annals of Statistics, 38(1), 2010.
30. F. Bauer, S. Pereverzev, and L. Rosasco. On regularization algorithms in learning theory. Journal of complexity, 23(1):52-72, 2007.
31. A. Caponnetto and Yuan Yao. Adaptive rates for regularization operators in learning theory. Analysis and Applications, 08, 2010.
32. Y. Ying and M. Pontil. Online gradient descent learning algorithms. Foundations of Computational Mathematics, 8(5):561-596, 2008.
33. Alessandro Rudi, Guillermo D. Canas, and Lorenzo Rosasco. On the Sample Complexity of Subspace Learning. In NIPS, pages 2067-2075, 2013.
34. Gene H. Golub and Charles F. Van Loan. Matrix computations, volume 3. JHU Press, 2012.