Fast approximation of matrix coherence and statistical leverage

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 3475-3506

  Drineas, Petros ^a   Magdon Ismail, Malik ^a   Mahoney, Michael W ^b   Woodruff, David P ^c  

^a Rensselaer Polytechnic Institute   (United States)

^b STANFORD UNIVERSITY   (United States)

^c IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Matrix coherence; Randomized algorithm; Statistical leverage

Indexed keywords

APPROXIMATION PROBLEMS; BASIC RESULTS; FAST APPROXIMATION; LOW-RANK MATRICES; MATRIX ALGORITHMS; MATRIX COMPLETION; NONUNIFORMITY; ORTHOGONAL BASIS; RANDOMIZED ALGORITHMS; SINGULAR VECTORS; STATISTICAL DATA ANALYSIS; STATISTICAL LEVERAGE; UNDER-CONSTRAINED;

MATRIX ALGEBRA;

APPROXIMATION ALGORITHMS;

EID: 84873435224     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (47)

1. D. Achlioptas. Database-friendly random projections: Johnson- lindenstrauss with binary coins. Journal of Computer and System Sciences, 66(4):671-687, 2003.
2. N. Ailon and B. Chazelle. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform. In Proceedings of the 38th Annual ACM Symposium on Theory of Computing, pages 557- 563, 2006. (Pubitemid 44306590)
3. N. Ailon and B. Chazelle. The fast Johnson-Lindenstrauss transformand approximate nearest neighbors. SIAM Journal on Computing, 39(1):302-322, 2009.
4. N. Ailon and E. Liberty. Fast dimension reduction using Rademacher series on dual BCH codes. In Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1-9, 2008.
5. A. Andoni, R. Krauthgamer, and K. Onak. Streaming algorithms from precision sampling. Technical report, 2010. Preprint: arXiv:1011.1263.
6. H. Avron, P. Maymounkov, and S. Toledo. Blendenpik: Supercharging LAPACK's least-squares solver. SIAM Journal on Scientific Computing, 32:1217-1236, 2010.
7. C. Bekas, E. Kokiopoulou, and Y. Saad. An estimator for the diagonal of a matrix. Applied Numer- ical Mathematics, 57:1214-1229, 2007. (Pubitemid 47348166)
8. C. Bekas, A. Curioni, and I. Fedulova. Low cost high performance uncertainty quantification. In Proceedings of the 2nd Workshop on High Performance Computational Finance, page Article No.: 8, 2009.
9. P. Bonacich. Power and centrality: A family of measures. The American Journal of Sociology, 92 (5):1170-1182, 1987.
10. C. Boutsidis,M.W.Mahoney, and P. Drineas. An improved approximation algorithm for the column subset selection problem. In Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 968-977, 2009.
11. C. Boutsidis, P. Drineas, andM.Magdon-Ismail. Near-optimal column-based matrix reconstruction. In Proceedings of the 52nd Annual IEEE Symposium on Foundations of Computer Science, pages 305-314, 2011a.
12. C. Boutsidis, P. Drineas, andM.Magdon-Ismail. Near-optimal column-based matrix reconstruction. Technical report, 2011b. Preprint: arXiv:1103.0995.
13. E.J. Candes and B. Recht. Exact matrix completion via convex optimization. Technical report, 2008. Preprint: arXiv:0805.4471.
14. S. Chatterjee and A.S. Hadi. Influential observations, high leverage points, and outliers in linear regression. Statistical Science, 1(3):379-393, 1986.
15. S. Chatterjee and A.S. Hadi. Sensitivity Analysis in Linear Regression. John Wiley & Sons, New York, 1988.
16. S. Chatterjee, A.S. Hadi, and B. Price. Regression Analysis by Example. John Wiley & Sons, New York, 2000.
17. A. Dasgupta, R. Kumar, and T. Sarlos. A sparse Johnson-Lindenstrauss transform. In Proceedings of the 42nd Annual ACM Symposium on Theory of Computing, pages 341-350, 2010.
18. P. Drineas, R. Kannan, and M.W. Mahoney. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. SIAM Journal on Computing, 36:132-157, 2006a. (Pubitemid 46374021)
19. P. Drineas, M.W. Mahoney, and S. Muthukrishnan. Sampling algorithms for ℓ2 regression and applications. In Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1127-1136, 2006b.
20. P. Drineas, M.W. Mahoney, and S. Muthukrishnan. Relative-error CUR matrix decompositions. SIAM Journal on Matrix Analysis and Applications, 30:844-881, 2008.
21. P. Drineas, J. Lewis, and P. Paschou. Inferring geographic coordinates of origin for Europeans using small panels of ancestry informative markers. PLoS ONE, 5(8):e11892, 2010a.
22. P. Drineas, M.W. Mahoney, S. Muthukrishnan, and T. Sarlos. Faster least squares approximation. Numerische Mathematik, 117(2):219-249, 2010b.
23. S. Ganguly, M. Bansal, and S. Dube. Estimating hybrid frequency moments of data streams. In Proceedings of the 2nd Annual International Workshop on Frontiers in Algorithmics, pages 55- 66, 2008.
24. S. Georgiev and S. Mukherjee, 2011. Unpublished results. (2011).
25. GittensM.W.M., 2012. In preparation. (2012).
26. G.H. Golub and C.F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, 1996.
27. N. Halko, P.-G. Martinsson, and J. A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2):217-288, 2011.
28. N.J.A. Harvey, J. Nelson, and K. Onak. Sketching and streaming entropy via approximation theory. In Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science, pages 489-498, 2008.
29. D.C. Hoaglin and R.E. Welsch. The hat matrix in regression and ANOVA. The American Statisti- cian, 32(1):17-22, 1978.
30. E. A. Jonckheere, M. Lou, J. Hespanha, and P. Barooah. Effective resistance of Gromov-hyperbolic graphs: Application to asymptotic sensor network problems. In Proceedings of the 46th IEEE Conference on Decision and Control, pages 1453-1458, 2007. (Pubitemid 351277358)
31. M. Magdon-Ismail. Row sampling for matrix algorithms via a non-commutative Bernstein bound. Technical report, 2010. Preprint: arXiv:1008.0587.
32. M.W.Mahoney. Randomized Algorithms forMatrices and Data. Foundations and trends in machine learning. NOW Publishers, Boston, 2011. Also available at: arXiv:1104.5557.
33. M.W.Mahoney and P. Drineas. CUR matrix decompositions for improved data analysis. Proc. Natl. Acad. Sci. USA, 106:697-702, 2009.
34. X. Meng, M. A. Saunders, and M.W. Mahoney. LSRN: A parallel iterative solver for strongly overor under-determined systems. Technical report, 2011. Preprint: arXiv:1109.5981.
35. M. Monemizadeh and D. P. Woodruff. 1-pass relative-error lp-sampling with applications. In Proceedings of the 21st Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1143-1160, 2010.
36. S. Muthukrishnan. Data Streams: Algorithms and Applications. Foundations and Trends in Theoretical Computer Science. Now Publishers Inc, Boston, 2005.
37. M.Z. Nashed, editor. Generalized Inverses and Applications. Academic Press, New York, 1976.
38. M.E.J. Newman. A measure of betweenness centrality based on random walks. Social Networks, 27:39-54, 2005. (Pubitemid 40272648)
39. C.H. Papadimitriou, P. Raghavan, H. Tamaki, and S. Vempala. Latent semantic indexing: a probabilistic analysis. Journal of Computer and System Sciences, 61(2):217-235, 2000.
40. P. Paschou, E. Ziv, E.G. Burchard, S. Choudhry, W. Rodriguez-Cintron, M.W. Mahoney, and P. Drineas. PCA-correlated SNPs for structure identification in worldwide human populations. PLoS Genetics, 3:1672-1686, 2007. (Pubitemid 47510260)
41. P. Paschou, J. Lewis, A. Javed, and P. Drineas. Ancestry informative markers for finescale individual assignment to worldwide populations. Journal of Medical Genetics, page doi:10.1136/jmg.2010.078212, 2010.
42. M. Richardson and P. Domingos. Mining knowledge-sharing sites for viral marketing. In Proceedings of the 8th Annual ACM SIGKDD Conference, pages 61-70, 2002.
43. V. Rokhlin and M. Tygert. A fast randomized algorithm for overdetermined linear least- squares regression. Proc. Natl. Acad. Sci. USA, 105(36):13212-13217, 2008.
44. V. Rokhlin, A. Szlam, and M. Tygert. A randomized algorithm for principal component analysis. SIAM Journal on Matrix Analysis and Applications, 31(3):1100-1124, 2009.
45. T. Sarl os. Improved approximation algorithms for large matrices via random projections. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, pages 143- 152, 2006. (Pubitemid 351175504)
46. A. Talwalkar and A. Rostamizadeh. Matrix coherence and the Nystr om method. In Proceedings of the 26th Conference in Uncertainty in Artificial Intelligence, 2010.
47. P.F. Velleman and R.E. Welsch. Efficient computing of regression diagnostics. The American Statistician, 35(4):234-242, 1981.