A randomized algorithm for principal component analysis

SIAM Journal on Matrix Analysis and Applications

Volumn 31, Issue 3, 2009, Pages 1100-1124

  Rokhlin, Vladimir ^a   Szlam, Arthur ^b   Tygert, Mark ^b  

^a YALE UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Lanczos; Low rank; Power; Principal component analysis; Singular value decomposition

Indexed keywords

EFFICIENT ALGORITHM; LANCZOS; LOW RANK; LOW RANK APPROXIMATIONS; MATRIX; NUMERICAL EXAMPLE; POWER; RANDOMIZED ALGORITHMS; SPECTRAL NORMS; THEORETICAL RESULT;

APPROXIMATION ALGORITHMS; COMPUTATIONAL EFFICIENCY; SINGULAR VALUE DECOMPOSITION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 72449140504     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/080736417     Document Type: Article

References (37)

1. D. ACHLIOPTAS AND F. MCSHERRY, Fast computation of low rank matrix approximations, in Proceedings of the Thirty-Third Annual Symposium on Theory of Computing, ACM, New York, 2001, pp. 611-618.
2. D. ACHLIOPTAS AND F. MCSHERRY, Fast computation of low-rank matrix approximations, J. ACM, 54 (2007), article 9.
3. T. F. CHAN AND P. C. HANSEN, Some applications of the rank revealing QR factorization, SIAM J. Sci. Statist. Comput., 13 (1992), pp. 727-741.
4. Z. CHEN AND J. J. DONGARRA, Condition numbers of Gaussian random matrices, SIAM J. Matrix Anal. Appl., 27 (2005), pp. 603-620.
5. K. L. CLARKSON AND D. P. WOODRUFF, Numerical linear algebra in the streaming model, in Proceedings of the 41st Annual ACM Symposium on Theory of Computing, ACM, New York, 2009, pp. 205-214.
6. S. DEERWESTER, S. T. DUMAIS, G. W. FURNAS, T. K. LANDAUER, AND R. HARSHMAN, Indexing by latent semantic analysis, J. Amer. Soc. Inform. Sci., 41 (1990), pp. 391-407.
7. A. DESHPANDE, L. RADEMACHER, S. VEMPALA, AND G. WANG, Matrix approximation and projective clustering via volume sampling, in Proceedings of the Seventeenth ACM-SIAM Symposium on Discrete Algorithms, ACM, New York, 2006, pp. 1117-1126.
8. A. DESHPANDE AND S. VEMPALA, Adaptive sampling and fast low-rank matrix approximation, Lecture Notes in Comput. Sci. 4110, Springer, Berlin, 2006, pp. 292-303. (Pubitemid 44577539)
9. J. D. DIXON, Estimating extremal eigenvalues and condition numbers of matrices, SIAM J. Numer. Anal., 20 (1983), pp. 812-814.
10. P. DRINEAS, E. DRINEA, AND P. S. HUGGINS, An experimental evaluation of a Monte-Carlo algorithm for singular value decomposition, in Revised Selected Papers from the Eighth Panhellenic Conference on Informatics, Lecture Notes in Comput. Sci. 2563, Y. Manolopoulos, S. Evripidou, and A. C. Kakas, eds., Springer, Berlin, 2003, pp. 279-296.
11. P. DRINEAS, R. KANNAN, AND M. W. MAHONEY, Fast Monte Carlo algorithms for matrices. II. Computing a low-rank approximation to a matrix, SIAM J. Comput., 36 (2006), pp. 158- 183.
12. P. DRINEAS, R. KANNAN, AND M. W. MAHONEY, Fast Monte Carlo algorithms for matrices. III. Computing a compressed approximate matrix decomposition, SIAM J. Comput., 36 (2006), pp. 184-206.
13. P. DRINEAS, M. W. MAHONEY, AND S. MUTHUKRISHNAN, Polynomial Time Algorithm for Column-Row-Based Relative-Error Low-Rank Matrix Approximation, Technical report 2006-04, DIMACS, Piscataway, NJ, 2006.
14. P. DRINEAS, M. W. MAHONEY, AND S. MUTHUKRISHNAN, Relative-error CUR matrix decompositions, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 844-881.
15. S. FRIEDLAND, M. KAVEH, A. NIKNEJAD, AND H. ZARE, Fast Monte-Carlo low rank approximations for matrices, in Proceedings of the First IEEE International Conference on System of Systems Engineering (SoSE 2006), Los Angeles, 2006, pp. 218-223.
16. A. FRIEZE AND R. KANNAN, Quick approximation to matrices and applications, Combinatorica, 19 (1999), pp. 175-220.
17. A. FRIEZE, R. KANNAN, AND S. VEMPALA, Fast Monte-Carlo algorithms for finding low-rank approximations, in Proceedings of the Thirty-Ninth Annual IEEE Symposium on Foundations of Computer Science, ACM, New York, 1998, pp. 370-378.
18. A. FRIEZE, R. KANNAN, AND S. VEMPALA, Fast Monte-Carlo algorithms for finding low-rank approximations, J. ACM, 51 (2004), pp. 1025-1041.
19. H. H. GOLDSTINE AND J. VON NEUMANN, Numerical inverting of matrices of high order, II, Proc. Amer. Math. Soc., 2 (1951), pp. 188-202.
20. G. H. GOLUB AND C. F. VAN LOAN, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, MD, 1996.
21. S. A. GOREINOV AND E. E. TYRTYSHNIKOV, The Maximal-Volume Concept in Approximation by Low-Rank Matrices, in Structured Matrices in Mathematics, Computer Science, and Engineering, I, Proceedings of an AMS-IMS-SIAM Joint Summer Research Conference, University of Colorado, Boulder, 1999, V. Olshevsky, ed., Contemp. Math. 280, AMS, Providence, RI, 2001, pp. 47-51.
22. S. A. GOREINOV, E. E. TYRTYSHNIKOV, AND N. L. ZAMARASHKIN, Pseudo-skeleton approximations by matrices of maximal volume, Math. Notes, 62 (1997), pp. 515-519.
23. S. A. GOREINOV, E. E. TYRTYSHNIKOV, AND N. L. ZAMARASHKIN, A theory of pseudoskeleton approximations, Linear Algebra Appl., 261 (1997), pp. 1-21.
24. M. GU AND S. C. EISENSTAT, Efficient algorithms for computing a strong rank-revealing QR factorization, SIAM J. Sci. Comput., 17 (1996), pp. 848-869.
25. S. HAR-PELED, Low-Rank Matrix Approximation in Linear Time, unpublished manuscript. Available online at http://valis.cs.uiuc.edu/~sariel/research/ papers/notes.html.
26. J. KUCZYŃSKI AND H. WOŹNIAKOWSKI, Estimating the largest eigenvalue by the power and Lanczos algorithms with a random start, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 1094- 1122.
27. E. LIBERTY, F. WOOLFE, P.-G. MARTINSSON, V. ROKHLIN, AND M. TYGERT, Randomized algorithms for the low-rank approximation of matrices, Proc. Natl. Acad. Sci. USA, 104 (2007), pp. 20167-20172.
28. M. W. MAHONEY AND P. DRINEAS, CUR matrix decompositions for improved data analysis, Proc. Natl. Acad. Sci. USA, 106 (2009), pp. 697-702.
29. P.-G. MARTINSSON, V. ROKHLIN, AND M. TYGERT, A Randomized Algorithm for the Approximation of Matrices, Technical report 1361, Yale University, Department of Computer Science, New Haven, CT, 2006. Available in the form referenced in the present paper at http://www.cs.yale.edu/publications/ techreports/tr1361.pdf and in revised form (with renumbered lemmas) at http://www.cs.yale.edu/~tygert/randapp.pdf.
30. C. H. PAPADIMITRIOU, P. RAGHAVAN, H. TAMAKI, AND S. VEMPALA, Latent semantic indexing: A probabilistic analysis, J. Comput. System Sci., 61 (2000), pp. 217-235.
31. S. ROWEIS, EM algorithms for PCA and SPCA, in Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems 10, MIT Press, Cambridge, MA, 1998, pp. 626-632.
32. T. SARLÓS, Improved approximation algorithms for large matrices via random projections, in Proceedings of the Forty-Seventh Annual IEEE Symposium on Foundations of Computer Science, Berkeley, CA, IEEE Computer Society, Washington, D.C., 2006, pp. 143-152.
33. T. SARLÓS, Improved Approximation Algorithms for Large Matrices via Random Projections, Long Form, Technical report, Eötvös Loránd University, Informatics Laboratory, Budapest, Hungary, 2006. Available online at http://www.ilab.sztaki.hu/~stamas/publications/rplong. pdf.
34. J. SUN, Y. XIE, H. ZHANG, AND C. FALOUTSOS, Less is more: Compact matrix decomposition for large sparse graphs, Stat. Anal. Data Min., 1 (2008), pp. 6-22.
35. E. E. TYRTYSHNIKOV, Incomplete cross approximation in the mosaic-skeleton method, Computing, 64 (2000), pp. 367-380.
36. J. H. WILKINSON, The Algebraic Eigenvalue Problem, Clarendon Press, Oxford, 1965.
37. F. WOOLFE, E. LIBERTY, V. ROKHLIN, AND M. TYGERT, A fast randomized algorithm for the approximation of matrices, Appl. Comput. Harmon. Anal., 25 (2008), pp. 335-366.