Efficient algorithms and error analysis for the modified Nyström Method

Journal of Machine Learning Research

Volumn 33, Issue , 2014, Pages 996-1004

  Wang, Shusen ^a   Zhang, Zhihua ^b  

^a ZHEJIANG UNIVERSITY   (China)

^b SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIG DATA; ERRORS; MATRIX ALGEBRA;

APPROXIMATION ACCURACY; BIG DATA APPLICATIONS; COLUMN SELECTIONS; COMPUTATIONAL CHALLENGES; KERNEL MATRICES; STATE-OF-THE-ART ALGORITHMS; TIME AND SPACE COMPLEXITY; TIME COMPLEXITY;

ALGORITHMS;

EID: 84955443267     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (37)

1. Affandi, R. H., A. Kulesza, E. B. Fox, and B. Taskar (2013). Nyström approximation for large-scale determinantal processes. In International Conference on Artificial Intelligence and Statistics (AISTATS).
2. Ben-Israel, A. and T. N. Greville (2003). Generalized Inverses: Theory and Applications. Second Edition. Springer.
3. Boutsidis, C., P. Drineas, and M. Magdon-Ismail (2011). Near optimal column-based matrix reconstruction. In Annual Symposium on Foundations of Computer Science (FOCS).
4. Cortes, C., M. Mohri, and A. Talwalkar (2010). On the impact of kernel approximation on learning accuracy. In Conference on Artificial Intelligence and Statistics (AISTATS).
5. Cortez, P., A. Cerdeira, F. Almeida, T. Matos, and J. Reis (2009). Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems 47(4), 547-553.
6. De Brabanter, K., J. De Brabanter, J. A. Suykens, and B. De Moor (2010). Optimized fixed-size kernel models for large data sets. Computational Statistics & Data Analysis 54(6), 1484-1504.
7. Deshpande, A., L. Rademacher, S. Vempala, and G. Wang (2006). Matrix approximation and projective clustering via volume sampling. Theory of Computing 2(2006), 225-247.
8. Drineas, P., M. Magdon-Ismail, M. W. Mahoney, and D. P. Woodruff (2012). Fast approximation of matrix coherence and statistical leverage. Journal of Machine Learning Research 13, 3441-3472.
9. Drineas, P. and M. W. Mahoney (2005). On the Nyström method for approximating a gram matrix for improved kernel-based learning. Journal of Machine Learning Research 6, 2153-2175.
10. Drineas, P., M. W. Mahoney, and S. Muthukrishnan (2008, September). Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications 30(2), 844-881.
11. Fowlkes, C., S. Belongie, F. Chung, and J. Malik (2004). Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(2), 214-225.
12. Frank, A. and A. Asuncion (2010). UCI machine learning repository.
13. Genton, M. G. (2001). Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research 2, 299-312.
14. Gittens, A. (2011). The spectral norm error of the naive Nyström extension. arXiv preprint arXiv:1110.5305.
15. Gittens, A. and M. W. Mahoney (2013). Revisiting the nyström method for improved large-scale machine learning. In International Conference on Machine Learning (ICML).
16. Gu, M. and S. C. Eisenstat (1996). Efficient algorithms for computing a strong rank-revealing QR factorization. SIAM Journal on Scientific Computing 17(4), 848-869.
17. Guruswami, V. and A. K. Sinop (2012). Optimal column-based low-rank matrix reconstruction. In Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA).
18. Halko, N., P.-G. Martinsson, and J. A. Tropp (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review 53(2), 217-288.
19. Jin, R., T. Yang, M. Mahdavi, Y.-F. Li, and Z.-H. Zhou (2012). Improved bounds for the Nyström method with application to kernel classification. CoRR abs/1111.2262.
20. Kumar, S., M. Mohri, and A. Talwalkar (2009). On sampling-based approximate spectral decomposition. In International Conference on Machine Learning (ICML).
21. Kumar, S., M. Mohri, and A. Talwalkar (2012). Sampling methods for the Nyström method. Journal of Machine Learning Research 13, 981-1006.
22. Li, M., X.-C. Lian, J. T. Kwok, and B.-L. Lu (2011). Time and space efficient spectral clustering via column sampling. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
23. Mahoney, M. W. (2011). Randomized algorithms for matrices and data. Foundations and Trends in Machine Learning 3(2), 123-224.
24. Michie, D., D. J. Spiegelhalter, and C. C. Taylor (1994). Machine Learning, Neural and Statistical Classification. Prentice Hall.
25. Nyström, E. J. (1930). Über die praktische auflösung von integralgleichungen mit anwendungen auf randwertaufgaben. Acta Mathematica 54(1), 185-204.
26. Schölkopf, B. and A. J. Smola (2002). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press.
27. Shawe-Taylor, J. and N. Cristianini (2004). Kernel Methods for Pattern Analysis. Cambridge University Press.
28. Shawe-taylor, J., C. K. I. Williams, N. Cristianini, and J. Kandola (2005). On the eigenspectrum of the gram matrix and the generalisation error of kernel pca. IEEE Transactions on Information Theory 51, 2510-2522.
29. Stewart, G. W. (1999). Four algorithms for the the efficient computation of truncated pivoted QR approximations to a sparse matrix. Numerische Mathematik 83(2), 313-323.
30. Talwalkar, A., S. Kumar, M. Mohri, and H. Rowley (2013). Large-scale svd and manifold learning. Journal of Machine Learning Research 14, 3129-3152.
31. Talwalkar, A. and A. Rostamizadeh (2010). Matrix coherence and the Nyström method. Conference on Uncertainty in Artificial Intelligence (UAI).
32. Tropp, J. A. (2012). User-friendly tools for random matrices: An introduction.
33. Wang, S. and Z. Zhang (2013). Improving CUR matrix decomposition and the Nyström approximation via adaptive sampling. Journal of Machine Learning Research 14, 2729-2769.
34. Williams, C. and M. Seeger (2001). Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems (NIPS).
35. Yang, T., Y.-F. Li, M. Mahdavi, R. Jin, and Z.-H. Zhou (2012). Nyström method vs random fourier features: A theoretical and empirical comparison. In Advances in Neural Information Processing Systems (NIPS).
36. Zhang, K. and J. T. Kwok (2010). Clustered Nyström method for large scale manifold learning and dimension reduction. IEEE Transactions on Neural Networks 21(10), 1576-1587.
37. Zhang, K., I. W. Tsang, and J. T. Kwok (2008). Improved Nyström low-rank approximation and error analysis. In International Conference on Machine Learning (ICML).