Large-scale SVD and manifold learning

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 3129-3152

  Talwalkar, Ameet ^a   Kumar, Sanjiv ^b   Mohri, Mehryar ^b   Rowley, Henry ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b GOOGLE INC   (United States)

Author keywords

Large scale matrix factorization; Low rank approximation; Manifold learning

Indexed keywords

COMPUTATIONAL CHALLENGES; LAPLACIAN EIGENMAPS; LOW RANK APPROXIMATIONS; LOW-DIMENSIONAL MANIFOLDS; MANIFOLD LEARNING; MATRIX FACTORIZATIONS; NONLINEAR DIMENSIONALITY REDUCTION; SINGULAR VALUE DECOMPOSITION TECHNIQUE;

APPROXIMATION THEORY; FACE RECOGNITION; SINGULAR VALUE DECOMPOSITION;

CLASSIFICATION (OF INFORMATION);

EID: 84887492519     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. C. Allauzen, M. Riley, J. Schalkwyk, W. Skut, and M. Mohri. OpenFST: A general and efficient weighted finite-state transducer library. In Conference on Implementation and Application of Automata, 2007.
2. M. Balasubramanian and E. L. Schwartz. The Isomap algorithm and topological stability. Science, 295, 2002.
3. M. Belkin and P. Niyogi. Laplacian Eigenmaps and spectral techniques for embedding and clustering. In Neural Information Processing Systems, 2001.
4. B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Conference on Learning Theory, 1992.
5. E. Chang, K. Zhu, H.Wang, H. Bai, J. Li, Z. Qiu, and H. Cui. Parallelizing support vector machines on distributed computers. In Neural Information Processing Systems, 2008.
6. C. Cortes,M.Mohri, and A. Talwalkar. On the impact of kernel approximation on learning accuracy. In Conference on Artificial Intelligence and Statistics, 2010.
7. V. de Silva and J. Tenenbaum. Global versus local methods in nonlinear dimensionality reduction. In Neural Information Processing Systems, 2003.
8. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang. Matrix approximation and projective clustering via volume sampling. In Symposium on Discrete Algorithms, 2006.
9. I. Dhillon and B. Parlett. Multiple representations to compute orthogonal eigenvectors of symmetric tridiagonal matrices. Linear Algebra and its Applications, 387:1-28, 2004.
10. D. Donoho and C. Grimes. Hessian Eigenmaps: locally linear embedding techniques for high dimensional data. Proceedings of the National Academy of Sciences, 100(10):5591-5596, 2003. (Pubitemid 36576852)
11. P. Drineas and M. W. Mahoney. On the Nystr om method for approximating a gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6:2153-2175, 2005. (Pubitemid 41832630)
12. P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices ii: Computing a low-rank approximation to a matrix. SIAM Journal on Computing, 36(1), 2006.
13. S. Fine and K. Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2002.
14. C. Fowlkes, S. Belongie, F. Chung, and J. Malik. Spectral grouping using the Nystr om method. Transactions on Pattern Analysis and Machine Intelligence, 26(2):214-225, 2004.
15. A. Frieze, R. Kannan, and S. Vempala. Fast Monte Carlo algorithms for finding low-rank approximations. In Foundation of Computer Science, 1998.
16. G. Golub and C. V. Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, 2nd edition, 1983.
17. N. Halko, P. Martinsson, and J. Tropp. Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions. arXiv:0909.4061v1[math.NA], 2009.
18. J. Ham, D. D. Lee, S. Mika, and B. Scholkopf. A kernel view of the dimensionality reduction of manifolds. In International Conference on Machine Learning, 2004.
19. X. He, S. Yan, Y. Hu, and P. Niyogi. Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3):328-340, 2005. (Pubitemid 40377326)
20. T. Joachims. Making large-scale support vector machine learning practical. In Neural Information Processing Systems, 1999.
21. S. Kumar and H. Rowley. People Hopper. http://googleresearch.blogspot. com/2010/03/ hopping-on-face-manifold-via-people.html, 2010.
22. S. Kumar, M. Mohri, and A. Talwalkar. On sampling-based approximate spectral decomposition. In International Conference on Machine Learning, 2009.
23. S. Kumar, M. Mohri, and A. Talwalkar. Sampling methods for the Nystr om method. In Journal of Machine Learning Research, 2012.
24. Y. LeCun and C. Cortes. The MNIST database of handwritten digits. http://yann.lecun.com/ exdb/mnist/, 1998.
25. T. Liu, A. W. Moore, A. G. Gray, and K. Yang. An investigation of practical approximate nearest neighbor algorithms. In Neural Information Processing Systems, 2004.
26. L. Mackey, A. Talwalkar, and M. I. Jordan. Divide-and-conquer matrix factorization. In Neural Information Processing Systems, 2011.
27. M.W.Mahoney. Randomized algorithms for matrices and data. Foundations and Trends inMachine Learning, 3(2):123-224, 2011.
28. G. Mann, R. McDonald, M. Mohri, N. Silberman, and D. Walker. Efficient large-scale distributed training of conditional maximum entropy models. In Neural Information Processing Systems, 2009.
29. E. Nystr om. Uber die praktische auflosung von linearen integralgleichungen mit anwendungen auf randwertaufgaben der potentialtheorie. Commentationes Physico-Mathematicae, 4(15):1-52, 1928.
30. J. Platt. Fast training of Support Vector Machines using sequential minimal optimization. In Neural Information Processing Systems, 1999.
31. J. Platt. Fast embedding of sparse similarity graphs. In Neural Information Processing Systems, 2004.
32. S. Roweis and L. Saul. Nonlinear dimensionality reduction by Locally Linear Embedding. Science, 290(5500), 2000.
33. B. Scholkopf and A. Smola. Learning with Kernels. MIT Press: Cambridge, MA, 2002.
34. B. Shaw and T. Jebara. Structure preserving embedding. In International Conference on Machine Learning, 2009.
35. T. Sim, S. Baker, andM. Bsat. The CMU pose, illumination, and expression database. In Conference on Automatic Face and Gesture Recognition, 2002.
36. A. Talwalkar. Matrix Approximation for Large-scale Learning. Ph.D. thesis, Computer Science Department, Courant Institute, New York University, New York, NY, 2010.
37. A. Talwalkar and A. Rostamizadeh. Matrix coherence and the Nystr om method. In Conference on Uncertainty in Artificial Intelligence, 2010.
38. A. Talwalkar, S. Kumar, and H. Rowley. Large-scale manifold learning. In Conference on Vision and Pattern Recognition, 2008.
39. J. Tenenbaum, V. de Silva, and J. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2000.
40. K. Weinberger and L. Saul. An introduction to nonlinear dimensionality reduction by maximum variance unfolding. In AAAI Conference on Artificial Intelligence, 2006.
41. C. K. Williams and M. Seeger. Using the Nystr om method to speed p kernel machines. In Neural Information Processing Systems, 2000.