Improved matrix algorithms via the subsampled randomized hadamard transform

SIAM Journal on Matrix Analysis and Applications

Volumn 34, Issue 3, 2013, Pages 1301-1340

  Boutsidis, Christos ^a   Gittens, Alex ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Hadamard transform; Least squares regression; Low rank approximation; Randomized algorithms; Sampling

Indexed keywords

DIMENSION REDUCTION METHOD; LEAST SQUARES REGRESSION; LINEAR ALGEBRA ALGORITHMS; LOW RANK APPROXIMATIONS; LOW-RANK MATRIX APPROXIMATIONS; MATRIX MULTIPLICATION; MATRIX OPERATIONS; RANDOMIZED ALGORITHMS;

ALGORITHMS; LINEAR ALGEBRA; SAMPLING;

HADAMARD TRANSFORMS;

EID: 84887400770     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/120874540     Document Type: Article

References (43)

1. N. Ailon and B. Chazelle, Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform, in Proceedings of the ACM Symposium on Theory of Computing (STOC), 2006.
2. N. Ailon and E. Liberty, Fast dimension reduction using Rademacher series on dual BCH codes, in Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2008, pp. 1-9.
3. H. Avron, P. Maymounkov, and S. Toledo, Blendenpik: Supercharging LAPACK's leastsquares solver, SIAM J. Sci. Comput., 32 (2010), pp. 1217-1236.
4. C. Boutsidis, Topics in Matrix Sampling Algorithms, Ph.D. thesis, Rensselaer Polytechnic Institute, 2011.
5. C. Boutsidis, P. Drineas, and M. Magdon-Ismail, Near optimal column based matrix reconstruction, in Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS), 2011.
6. C. Boutsidis and P. Drineas, Random projections for the nonnegative least-squares problem, Linear Algebra Appl., 431 (2009), pp. 760-771.
7. C. Boutsidis, P. Drineas, and M. Magdon-Ismail, Rich Coresets for Unconstrained Linear Regression, preprint, arXiv:1202.3505, 2012.
8. C. Boutsidis, A. Zouzias, and P. Drineas, Random projections for k-means clustering, in Proceedings of the Conference on Neural Information Processing Systems (NIPS), 2010.
9. C. Boutsidis, M. W. Mahoney, and P. Drineas, An improved approximation algorithm for the column subset selection problem, in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009, pp. 968-977.
10. E. Cand́es and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
11. K. Clarkson, P. Drineas, M. Magdon-Ismail, M. W. Mahoney, X. Men, and D. Woodruff, The fast Cauchy transform and faster robust linear regression, in Proceedings of the 24th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2013, pp. 466-477.
12. K. Clarkson and D. Woodruff, Numerical linear algebra in the streaming model, in Proceedings of the ACM Symposium on Theory of Computing (STOC), 2009.
13. K. Clarkson and D. Woodruff, Low Rank Approximation and Regression in Input Sparsity Time, preprint, arXiv:1207.6365, 2012.
14. P. Drineas, Randomized Algorithms for Matrix Operations, Ph.D. thesis, Yale University, 2002.
15. P. Drineas and R. Kannan, Fast Monte-Carlo algorithms for approximate matrix multiplication, in Proceedings of the Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2001.
16. P. Drineas, R. Kannan, and M. W. Mahoney, Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication, SIAM J. Comput., 36 (2006), pp. 132-157.
17. P. Drineas, R. Kannan, and M. W. Mahoney, Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix, SIAM J. Comput., 36 (2006), pp. 158-183.
18. P. Drineas, R. Kannan, and M. W. Mahoney, Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition, SIAM J. Comput., 36 (2006), pp. 184-206.
19. P. Drineas, M. W. Mahoney, S. Muthukrishnan, and T. Sarlós, Faster least squares approximation, Numer. Math., 117 (2011), pp. 217-249.
20. A. Frieze, R. Kannan, and S. Vempala, Fast Monte-Carlo algorithms for finding low-rank approximations, in Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS), 1998.
21. M. Gu and S. C. Eisenstat, Efficient algorithms for computing a strong rank-revealing QR factorization, SIAM J. Sci. Comput., 17 (1996), pp. 848-869.
22. A. Gittens, The Spectral Norm Error of the Naive Nystrom Extension, preprint, arXiv: 1110.5305, 2011.
23. A. Gittens and J. Tropp, Tail Bounds for All Eigenvalues of a Sum of Random Matrices, preprint, arXiv:1104.4513, 2011.
24. D. Gross, Recovering low-rank matrices from few coefficients in any basis, IEEE Trans. Inform. Theory, 57 (2011), pp. 1548-1566.
25. D. Gross and V. Nesme, Note on Sampling without Replacing from a Finite Collection of Matrices, preprint, arXiv:1001.2738, 2010.
26. N. Halko, P. G. Martinsson, and J. A. Tropp, Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions, SIAM Rev., 53 (2011), pp. 217-288.
27. W. Hoeffding, Probability inequalities for sums of bounded random variables, J.Amer. Statist. Assoc., 58 (1963), pp. 13-30.
28. I. Ipsen and T. Wentworth, The Effect of Coherence on Sampling from Matrices with Orthonormal Columns, and Preconditioned Least Squares Problems, preprint, arXiv: 1203.4809, 2012.
29. M. Ledoux, On Talagrand's deviation inequalities for product measures, ESAIM Probab. Statist., 1 (1996), pp. 63-87.
30. E. Liberty, F. Woolfe, P.-G. Martinsson, V. Rokhlin, and M. Tygert, Randomized algorithms for the low-rank approximation of matrices, Proc. Natl. Acad. Sci. USA, 104 (2007), pp. 20167-20172.
31. M. Mahoney, Algorithmic and Statistical Perspectives on Large-Scale Data Analysis, Chapman & Hall/CRC Comput. Sci. Ser., CRC Press, Boca Raton, FL, 2011.
32. M. Mahoney, L. Lim, and G. Carlsson, Algorithmic and statistical challenges in modern largescale data analysis are the focus of MMDS 2008, ACMSIGKDD Explorations Newsletter, 10 (2008), pp. 57-60.
33. A. Magen and A. Zouzias, Low rank matrix-valued Chernoff bounds and approximate matrix multiplication, in Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2011, pp. 1422-1436.
34. P.-G. Martinsson, V. Rokhlin, and M. Tygert, A randomized algorithm for the decomposition of matrices, Appl. Comput. Harmonic Anal., 30 (2010), pp. 47-68.
35. N. H. Nguyen, T. T. Do, and T. D. Tran, A fast and efficient algorithm for low-rank approximation of a matrix, in Proceedings of the ACM Symposium on Theory of Computing (STOC), 2009.
36. C. C. Paige and M. A. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8 (1982), pp. 43-71.
37. V. Rokhlin, A. Szlam, and M. Tygert, A randomized algorithm for principal component analysis, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 1100-1124.
38. V. Rokhlin and M. Tygert, A fast randomized algorithm for overdetermined linear leastsquares regression, Proc. Natl. Acad. Sci. USA, 105 (2008), pp. 13212-13217.
39. A. Talwalkar and A. Rostamizadeh, Matrix coherence and the Nyström method, in Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2010.
40. T. Sarlós, Improved approximation algorithms for large matrices via random projections, in Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS), 2006.
41. J. A. Tropp, Improved analysis of the subsampled randomized Hadamard transform, Adv. Adaptive Data Anal., 3 (2011), pp. 115-126.
42. J. A. Tropp, User-friendly tail bounds for sums of random matrices, Found. Comput. Math., 12 (2012), pp. 389-434.
43. F. Woolfe, E. Liberty, V. Rokhlin, and M. Tygert, A fast randomized algorithm for the approximation of matrices, preliminary report, Appl. Comput. Harmonic Anal., 25 (2008), pp. 335-366.