On the Nyström method for approximating a Gram matrix for improved kernel-based learning

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages 2153-2175

  Drineas, Petros ^a   Mahoney, Michael W ^b  

^a Rensselaer Polytechnic Institute   (United States)

^b YALE UNIVERSITY   (United States)

Author keywords

Gram matrix; Kernel methods; Nystr m method; Randomized algorithms

Indexed keywords

ALGORITHMS; COMPUTATION THEORY; INTEGRAL EQUATIONS; LEARNING SYSTEMS; MATRIX ALGEBRA; PROBABILITY DISTRIBUTIONS;

GRAM MATRIX; KERNEL METHODS; NYSTRÖM METHOD; RANDOMIZED ALGORITHMS;

APPROXIMATION THEORY;

EID: 29244453931     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (42)

1. D. Achlioptas and F. McSherry. Fast computation of low rank matrix approximations. In Proceedings of the 33rd Annual ACM Symposium on Theory of Computing, pages 611-618, 2001.
2. D. Achlioptas, F. McSherry, and B. Schölkopf. Sampling techniques for kernel methods. In Annual Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference, pages 335-342, 2002.
3. Y. Azar, A. Fiat, A. R. Karlin, F. McSherry, and J. Saia. Spectral analysis of data. In Proceedings of the 33rd Annual ACM Symposium on Theory of Computing, pages 619-626, 2001.
4. M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6): 1373-1396, 2003.
5. A. Ben-Israel and T. N. E. Greville. Generalized Inverses: Theory and Applications. Springer-Verlag, New York, 2003.
6. Y. Bengio, J. F. Paiement, P. Vincent, O. Delalleau, N. Le Roux, and M. Ouimet. Out-of-sample extensions for LLE, Isomap, MDS, eigenmaps, and spectral clustering. In Annual Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference, pages 177-184, 2004.
7. R. Bhatia. Matrix Analysis. Springer-Verlag, New York, 1997.
8. J. C. Burges. Simplified support vector decision rules. In Proceedings of the 13th International Conference on Machine Learning, pages 71-77, 1996.
9. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and Other Kernelbased Learning Methods. Cambridge University Press, Cambridge, 2000.
10. L. M. Delves and J. L. Mohamed. Computational Methods for Integral Equations. Cambridge University Press, Cambridge, 1985.
11. D. L. Donoho and C. Grimes. Hessian eigenmaps: Locally linear embedding techniques for highdimensional data. Proc. Natl. Acad. Sci. USA, 100(10):5591-5596, 2003.
12. P. Drineas, A. Frieze, R. Kannan, S. Vempala, and V. Vinay. Clustering in large graphs and matrices. In Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 291-299, 1999.
13. P. Drineas and R. Kannan. Fast Monte-Carlo algorithms for approximate matrix multiplication. In Proceedings of the 42nd Annual IEEE Symposium on Foundations of Computer Science, pages 452-459, 2001.
14. P. Drineas and R. Kannan. Pass efficient algorithms for approximating large matrices. In Proceedings of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 223-232, 2003.
15. P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. Technical Report YALEU/DCS/TR-1269, Yale University Department of Computer Science, New Haven, CT, February 2004a. Accepted for publication in the SIAM Journal on Computing.
16. P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix. Technical Report YALEU/DCS/TR-1270, Yale University Department of Computer Science, New Haven, CT, February 2004b. Accepted for publication in the SIAM Journal on Computing.
17. P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition. Technical Report YALEU/DCS/TR-1271, Yale University Department of Computer Science, New Haven, CT, February 2004c. Accepted for publication in the SIAM Journal on Computing.
18. P. Drineas, R. Kannan, and M. W. Mahoney. Sampling sub-problems of heterogeneous Max-Cut problems and approximation algorithms. Technical Report YALEU/DCS/TR-1283, Yale University Department of Computer Science, New Haven, CT, April 2004d.
19. P. Drineas, R. Kannan, and M. W. Mahoney. Sampling sub-problems of heterogeneous Max-Cut problems and approximation algorithms. In Proceedings of the 22nd Annual International Symposium on Theoretical Aspects of Computer Science, pages 57-68, 2005.
20. P. Drineas and M. W. Mahoney. Approximating a Gram matrix for improved kernel-based learning. In Proceedings of the 18th Annual Conference on Learning Theory, pages 323-337, 2005a.
21. P. Drineas and M. W. Mahoney. On the Nyström method for approximating a Gram matrix for improved kernel-based learning. Technical Report YALEU/DCS/TR-1319, Yale University Department of Computer Science, New Haven, CT, April 2005b.
22. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2001.
23. C. Fowlkes, S. Belongie, F. Chung, and J. Malik. Spectral grouping using the Nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):214-225, 2004.
24. A. Frieze, R. Kannan, and S. Vempala. Fast Monte-Carlo algorithms for finding low-rank approximations. In Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science, pages 370-378, 1998.
25. G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, 1989.
26. S. A. Goreinov and E. E. Tyrtyshnikov. The maximum-volume concept in approximation by lowrank matrices. Contemporary Mathematics, 280:47-51, 2001.
27. S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. A theory of pseudoskeleton approximations. Linear Algebra and Its Applications, 261:1-21, 1997.
28. J. Ham, D. D. Lee, S. Mika, and B. Schölkopf. A kernel view of the dimensionality reduction of manifolds. Technical Report TR-110, Max Planck Institute for Biological Cybernetics, July 2003.
29. R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, New York, 1985.
30. S. Lafon. Diffusion Maps and Geometric Harmonics. PhD thesis, Yale University, 2004.
31. M. Z. Nashed, editor. Generalized Inverses and Applications. Academic Press, New York, 1976.
32. E. Osuna, R. Freund, and F. Girosi. An improved training algorithm for support vector machines. In Proceedings of the 1997 IEEE Workshop on Neural Networks for Signal Processing VII, pages 276-285, 1997.
33. L. Rademacher, S. Vempala, and G. Wang. Matrix approximation and projective clustering via iterative sampling. Technical Report MIT-LCS-TR-983, Massachusetts Institute of Technology, Cambridge, MA, March 2005.
34. S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by local linear embedding. Science, 290:2323-2326, 2000.
35. B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998.
36. A. J. Smola and B. Schölkopf. Sparse greedy matrix approximation for machine learning. In Proceedings of the 17th International Conference on Machine Learning, pages 911-918, 2000.
37. G. W. Stewart and J. G. Sun. Matrix Perturbation Theory. Academic Press, New York, 1990.
38. J. B. Tenenbaum, V. de Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319-2323, 2000.
39. K. Q. Weinberger, F. Sha, and L. K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction. In Proceedings of the 21st International Conference on Machine Learning, pages 839-846, 2004.
40. C. K. I. Williams, C. E. Rasmussen, A. Schwaighofer, and V. Tresp. Observations on the Nyström method for Gaussian process prediction. Technical report, University of Edinburgh, 2002.
41. C. K. I. Williams and M. Seeger. The effect of the input density distribution on kernel-based classifiers. In Proceedings of the 17th International Conference on Machine Learning, pages 1159-1166, 2000.
42. C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In. Annual Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference, pages 682-688, 2001.