Sketching for M-estimators: A unified approach to robust regression

Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms

Volumn 2015-January, Issue January, 2015, Pages 921-939

  Clarkson, Kenneth L ^a   Woodruff, David P ^a  

^a IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST FUNCTIONS;

CONSTANT FACTORS; HIGH PROBABILITY; M-ESTIMATORS; RELATIVE ERRORS; RESIDUAL ERROR; ROBUST REGRESSIONS; ROBUSTNESS PROPERTIES; UNIFIED APPROACH;

MATRIX ALGEBRA;

EID: 84938243406     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1137/1.9781611973730.63     Document Type: Conference Paper

References (43)

1. Dimitris Achlioptas. Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci., 66 (4): 671-687, 2003.
2. Dimitris Achlioptas and Frank McSherry. Fast computation of low-rank matrix approximations. J. ACM, 54 (2), 2007.
3. Nir Ailon and Bernard Chazelle. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform. In STOC, pages 557-563, 2006.
4. Nir Ailon and Edo Liberty. An almost optimal unrestricted fast Johnson-Lindenstrauss transform. ACM Trans. Algorithms, 9 (3): 21:1-21:12, June 2013.
5. S. Bernstein. Theory of Probability. Moscow, 1927.
6. Jean Bourgain and Jelani Nelson. Toward a unified theory of sparse dimensionality reduction in euclidean space. CoRR, abs/1311.2542, 2013.
7. C. Boutsidis and A. Gittens. Improved matrix algorithms via the Subsampled Randomized Hadamard Transform. ArXiv e-prints, March 2012.
8. Vladimir Braverman and Rafail Ostrovsky. Zero-one frequency laws. In STOC, pages 281-290, 2010.
9. Kenneth L Clarkson and David P Woodruff. Low rank approximation and regression in input spar-sity time. In STOC, 2013. Full version at http://arxiv.org/abs/1207.6365.
10. Anirban Dasgupta, Petros Drineas, Boulos Harb, Ravi Kumar, and Michael W. Mahoney. Sampling algo-rithms and coresets for ℓp regression. SIAM J. Corn-put, 38 (5): 2060-2078, 2009.
11. Anirban Dasgupta, Ravi Kumar, and Tamas Sarlos. A sparse Jofmson-Lindenstrauss transform. In STOC, pages 341-350, 2010.
12. Amit Deshpande, Luis Rademacher, Santosh Vempala, and Grant Wang. Matrix approximation and projective clustering via volume sampling. In SODA, pages 1117-1126, 2006.
13. Amit Deshpande and Santosh Vempala. Adaptive sampling and fast low-rank matrix approximation. In APPROX-RANDOM, pages 292-303, 2006.
14. Petros Drineas, Ravi Kannan, and Michael W. Mahoney. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. SI AM J. Corn-put, 36 (1): 132-157, 2006.
15. Petros Drineas, Ravi Kannan, and Michael W. Mahoney. Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix. SIAM J. Comput, 36 (1): 158-183, 2006.
16. Petros Drineas, Ravi Kannan, and Michael W. Mahoney. Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition. SIAM J. Comput, 36 (1): 184-206, 2006.
17. Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, and David P. Woodruff. Fast approximation of matrix coherence and statistical leverage. CoRR, abs/1109.3843, 2011.
18. Petros Drineas, Michael W. Mahoney, and S. Muthukr-ishnan. Sampling algorithms for ℓ2 regression and ap-plications. In SODA, pages 1127-1136, 2006.
19. Petros Drineas, Michael W. Mahoney, and S. Muthukr-ishnan. Subspace sampling and relative-error matrix approximation: Column-based methods. In APPROX-RANDOM, pages 316-326, 2006.
20. Petros Drineas, Michael W. Mahoney, and S. Muthukr-Ishnan. Subspace sampling and relative-error matrix approximation: Column-row-based methods. In ESA, [40 pages 304-314, 2006.
21. Petros Drineas, Michael W. Mahoney, S. Muthukrish-nan, and Tamas Sarlos. Faster least squares approximation. CoRR, abs/0710.1435, 2007.
22. Antoine Guitton and William W. Symes. Robust and stable velocity analysis using the Huber function, 1999.
23. S. J. Haberman. Convexity and estimation. Ann. Statist., 17:1631-1661, 1989.
24. Peter J. Huber. Robust estimation of a location parameter. Ann. Math. Statist., 35 (1): 73-101, 1964.
25. Piotr Indyk and David P. Woodruff. Optimal approximations of the frequency moments of data streams. In STOC, pages 202-208, 2005.
26. William B. Johnson and Joram Lindenstrauss. Extensions of Lipschitz mappings into a Hilbert space. In Conference in Modern Analysis and Probability, 1982.
27. Daniel M. Kane and Jelani Nelson. Sparser Johnson-Lindenstrauss transforms. In SODA, pages 1195-1206, 2012.
28. Ioannis Koutis, Gary L. Miller, and Richard Peng. Approaching optimality for solving SDD linear systems. In FOCS, pages 235-244, 2010.
29. Ioannis Koutis, Gary L. Miller, and Richard Peng. A nearly-m log n time solver for SDD linear systems. In FOCS, pages 590-598, 2011.
30. Michael W. Mahoney. Randomized algorithms for matrices and data. Foundations and Trends in Machine Learning, 3 (2): 123-224, 2011.
31. Olvi L. Mangasarian and David R. Musicant. Robust linear and support vector regression. IEEE 'Trans. Pattern Anal. Mach. Inteli, 22 (9): 950-955, 2000.
32. Jiri Matousek. Lectures on Discrete Geometry. Springer, 2002.
33. A. Maurer. A bound on the deviation probability for sums of non-negative random variables. Journal of Inequalities in Pure and Applied Mathematics, 4 (1), 2003.
34. X. Meng and M. W. Mahoney. Low-distortion Sub-space Embeddings in Input-sparsity Time and Applications to Robust Linear Regression. ArXiv e-prints, October 2012.
35. Gary L. Miller and Richard Peng. Iterative approaches to row sampling. CoRR, abs/1211.2713, 2012.
36. Jelani Nelson and Huy L. Nguyen. OSNAP: Faster numerical linear algebra algorithms via sparser subspace embeddings. CoRR, abs/1211.1002, 2012.
37. Wojciech Niemiro. Asymptotics for m-estimators defined by convex minimization. Ann. Statist., 20 (3): 1514-1533, 1992.
38. Tamas Sarlos. Improved approximation algorithms for large matrices via random projections. In FOCS, pages 143-152, 2006.
39. D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In 36th Annual ACM Symposium on Theory of Computing, pages 81-90. ACM, 2004.
40. Elad Verbin and Qin Zhang. Rademacher-sketch: A dimensionality-reducing embedding for sum-product norms, with an application to earth-mover distance. In ICALP (1), pages 834-845, 2012.
41. p-regression using exponential random variables. CoRR, abs/1305.5580, 2013.
42. Jiyan Yang, Xiangrui Meng, and Michael W. Mahoney. Quantile regression for large-scale applications. In ICML (3), pages 881-887, 2013.
43. Zhengyou Zhang. Parameter estimation techniques: A tutorial with application to conic fitting. Image and Vision Computing Journal, 15 (1): 59-76, 1997. http://research.microsoft.com/en-us/um/people/zhang/INRIA/Publis/Tutorial-Estim/node24.html.