Faster subset selection for matrices and applications

SIAM Journal on Matrix Analysis and Applications

Volumn 34, Issue 4, 2013, Pages 1464-1499

  Avron, Haim ^a   Boutsidis, Christos ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Feature selection; K means clustering; Low rank approximations; Low stretch spanning trees; Sparse approximation; Subset selection; Volume sampling

Indexed keywords

K-MEANS CLUSTERING; LOW RANK APPROXIMATIONS; SPANNING TREE; SPARSE APPROXIMATIONS; SUBSET SELECTION;

APPROXIMATION ALGORITHMS; FEATURE EXTRACTION;

SET THEORY;

EID: 84892412888     PISSN: 08954798     EISSN: 10957162     Source Type: Journal    
DOI: 10.1137/120867287     Document Type: Article

References (57)

1. I. Abraham, Y. Bartal, and O. Neiman, Nearly tight low stretch spanning trees, in Proceedings of the 49th IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 781-790.
2. N. Alon, R. M. Karp, D. Peleg, and D. West, A graph-theoretic game and its application to the k-server problem, SIAM J. Comput., 24 (1995), pp. 78-100.
3. H. Avron, P. Maymounkov, and S. Toledo, Blendenpik: Supercharging LAPACK's leastsquares solver, SIAM J. Sci. Comput., 32 (2010), pp. 1217-1236.
4. J. Batson, D. Spielman, and N. Srivastava, Twice-Ramanujan sparsifiers, in Proceedings of the 41st Annual ACM Symposium on Theory of Computing (STOC), ACM, New York, pp. 255-262, 2009.
5. D. S. Bernstein, Matrix Mathematics: Theory, Facts, and Formulas with Applications to Linear Systems Theory, Princeton University Press, Princeton, NJ, 2005.
6. E. G. Boman and B. Hendrickson, On Spanning Tree Preconditioners, Sandia National Laboratories, Albuquerque, NM, unpublished manuscript, 2001.
7. J. Bourgain and L. Tzafriri, Invertibility of large submatrices with applications to the geometry of banach spaces and harmonic analysis, Israel J. Math., 57 (1987), pp. 137-224.
8. C. Boutsidis, On Truncated-SVD-Like Sparse Solutions to Least-Squares of Arbitrary Dimensions, manuscript, available online at http://www.cs.rpi.edu/ boutsc/SparseChristos.pdf.
9. C. Boutsidis, Topics in matrix sampling algorithms, Ph.D. thesis, Rensselaer Polytechnic Institute, May, 2011.
10. C. Boutsidis, P. Drineas, and M. Magdon-Ismail, Near optimal column based matrix reconstruction, in Proceedings of the 52nd Annual IEEE Symposium on Foundations of Computer Science (FOCS), IEEE, Los Alamitos, CA, 2011, pp. 305-314. Full version available online at http://arxiv.org/pdf/1103.0995v2.pdf.
11. C. Boutsidis and M. Magdon-Ismail, Deterministic feature selection for k-means clustering, arxiv: 1109.5664, 2011.
12. C. Boutsidis, M. W. Mahoney, and P. Drineas, An improved approximation algorithm for the column subset selection problem, in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), SIAM, Philadelphia, 2009, pp. 968-977.
13. C. Boutsidis, A. Zouzias, M. W. Mahoney, and P. Drineas, Stochastic dimensionality reduction for k-means clustering, arxiv: 1110.2897, 2013.
14. M. Broadbent, M. Brown, K. Penner, I. Ipsen, and R. Rehman, Subset Selection Algorithms: Randomized vs. Deterministic, SIAM Undergraduate Research Online, 3, 2010, pp. 50-71.
15. T. F. Chan and P. C. Hansen, Some applications of the rank revealing QR factorization, SIAM J. Sci. Statist. Comput., 13 (1992), pp. 727-741.
16. T. F. Chan and P. C. Hansen, Low-rank revealing QR factorizations, Numer. Linear Algebra Appl., 1 (1994), pp. 33-44.
17. A. Çivril and M. Magdon-Ismail, Exponential inapproximability of selecting a maximum volume sub-matrix, Algorithmica, 65 (2013), pp. 159-176.
18. A. Çivril and M. Magdon-Ismail, On selecting a maximum volume sub-matrix of a matrix and related problems, Theoret. Comput. Sci., 410 (2009), pp. 4801-4811.
19. F. R. de Hoog and R. M. M. Mattheij, Subset selection for matrices, Linear Algebra Appl., 422 (2007), pp. 349-359.
20. F. R. de Hoog and R. M. M. Mattheij, A note on subset selection for matrices, Linear Algebra Appl., 434 (2011), pp. 1845-1850.
21. A. Deshpande and L. Rademacher, Efficient volume sampling for row/column subset selection, in Proceedings of the 42th Annual ACM Symposium on Theory of Computing (STOC), ACM, New York, 2010, pp. 329-338.
22. A. Deshpande, L. Rademacher, S. Vempala, and G. Wang, Matrix approximation and projective clustering via volume sampling, in Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms, ACM, New York, 2006, pp. 1117-1126.
23. P. Drineas, R. Kannan, and M. W. Mahoney, Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication, SIAM J. Comput., 36 (2006), pp. 132-157.
24. P. Drineas, M. Magdon-Ismail, M.W. Mahoney, and D. P.Woodruff, Fast approximation of matrix coherence and statistical leverage, 13 (2012), pp. 3475-3506.
25. P. Drineas, M. W. Mahoney, S. Muthukrishnan, and T. Saros, Faster least squares approximation, Numer. Math., 117 (2011), pp. 219-249.
26. S. C. Eisenstat and I. C. F. Ipsen, Relative perturbation techniques for singular value problems, SIAM J. Numer. Anal., 32 (1995), pp. 1972-1988.
27. M. Elkin, Y. Emerk, D. A. Spielman, and S. H. Teng, Lower-stretch spanning trees, in Proceedings of the 37th Annual ACM Symposium on Theory of Computing (STOC), ACM, New York, 2005, pp. 494-503.
28. L. Foster and R. Kommu, Algorithm 853: An efficient algorithm for solving rank-deficient least squares problems, ACM Trans. Math. Software, 32 (2006), pp. 157-165.
29. A. Gittens, The spectral norm error of the naive Nystrom extension, arXiv:1110.5305, 2011.
30. G. Golub and C. V. Loan, Matrix Computations, Johns Hopkins University Press, 3rd ed., Baltimore, 1996.
31. S. Goreinov, E. Tyrtyshnikov, and N. Zamarashkin, A theory of pseudoskeleton approximations, Linear Algebra Appl., 261 (1997), pp. 1-21.
32. M. Gu and S. C. Eisenstat, Downdating the singular value decomposition, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 793-810.
33. M. Gu and S. C. Eisenstat, Efficient algorithms for computing a strong rank-revealing QR factorization, SIAM J. Sci. Comput., 17 (1996), pp. 848-869.
34. M. Gu and L. Miranian, Strong rank revealing Cholesky factorization, Electron. Trans. Numer. Anal., 17 (2004), pp. 76-92.
35. V. Guruswami and A. K. Sinop, Optimal column-based low-rank matrix reconstruction, in Proceedings of the Twenty-Third Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '12, SIAM, Philadelphia, 2012, pp. 1207-1214.
36. P. C. Hansen, The truncated SVD as a method for regularization, BIT, 27 (1987), pp. 534-553.
37. Y. P. Hong and C. T. Pan, Rank-revealing QR factorizations and the singular value decomposition, Math. Comput., 58 (1992), pp. 213-232.
38. R. Horn and C. Johnson, Matrix Analysis, Cambridge University Press, New York, 1985.
39. I. C. F. Ipsen, C. T. Kelley, and S. R. Pope, Rank-deficient nonlinear least squares problems and subset selection, SIAM J. Numer. Anal., 49 (2011), pp. 1244-1266.
40. S. Joshi and S. Boyd, Sensor selection via convex optimization, IEEE Trans. Signal Process., 57 (2009), pp. 451-462.
41. I. Koutis, Parameterized complexity and improved inapproximability for computing the largest j-simplex in a v-polytope, Inform. Process. Lett., 100 (2006), pp. 8-13.
42. I. Koutis, G. L. Miller, and R. Peng, Solving SDD linear systems in time O(mlog n log(1/-) ), in Proceedings of the 52nd Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2011.
43. P.-G. Martinsson, V. Rokhlin, and M. Tygert, A randomized algorithm for the decomposition of matrices, Appl. Comput. Harmon. Anal., 30 (2011), pp. 47-68.
44. A. Packer, NP-hardness of largest contained and smallest containing simplices for v-and h-polytopes, Discrete Comput. Geom., 28 (2002), pp. 349-377.
45. C.-T. Pan, On the existence and computation of rank-revealing LU factorizations, Linear Algebra Appl., 316 (2000), pp. 199-222.
46. D. Peleg, Distributed Computing: A Locality-Sensitive Approach, SIAM, Philadelphia, 2000.
47. M. Rudelson and R. Vershynin, Sampling from large matrices: An approach through geometric functional analysis, J. ACM, 54 (2007) 21.
48. T. Saros, Improved approximation algorithms for large matrices via random projections, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS), IEEE, Washington, DC, 2006, pp. 143-152.
49. D. A. Spielman and N. Srivastava, Graph sparsifications by effective resistances, in Proceedings of the 40th ACM Symposium on Theory of Computing (STOC), ACM, New York, 2008, pp 563-568.
50. D. A. Spielman and N. Srivastava, An elementary proof of the restricted invertibility theorem, Israel J. Math., 190 (2012), pp. 1-9.
51. D. A. Spielman and S.-H. Teng, Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems, in Proceedings of the 36th Annual ACM Symposium on Theory of Computing (STOC), ACM, New York, 2004, pp. 81-90.
52. D. A. Spielman and J. Woo, A note on preconditioning by low-stretch spanning trees, arXiv:0903.2816, 2009.
53. J. Tropp, Improved analysis of the subsampled randomized Hadamard transform, Adv. Adapt. Data Anal., 3 (2011), pp. 115-126.
54. J. A. Tropp, Column subset selection, matrix factorization, and eigenvalue optimization, in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), SIAM, Philadelphia, 2009, pp. 978-986.
55. R. Vershynin, A Note on Sums of Independent Random Matrices after Ahlswede-Winter, Lecture Notes, 2010, available online at http://www-personal. umich.edu/romanv/teaching/reading-group/ahlswede-winter.pdf.
56. D. B. Wilson, Generating random spanning trees more quickly than the cover time, in Proceedings of the 28th Annual ACM Symposium on Theory of Computing (STOC), ACM, New York, 1996, pp. 296-303.
57. A. Zouzias, A matrix hyperbolic cosine algorithm and applications, arXiv:1103.2793, 2011.