lp-Recovery of the Most Significant Subspace Among Multiple Subspaces with Outliers

Constructive Approximation

Volumn 40, Issue 3, 2014, Pages 329-385

  Lerman, Gilad ^a   Zhang, Teng ^a  

^a University of Minnesota   (United States)

Author keywords

Best approximating subspace; Geometric probability; Hybrid linear modeling; lpminimization; Optimization on the Grassmannian; Robust statistics

Indexed keywords

EID: 84919903045     PISSN: 01764276     EISSN: 14320940     Source Type: Journal    
DOI: 10.1007/s00365-014-9242-6     Document Type: Article

References (49)

1. Arias-Castro, E., Donoho, D.L., Huo, X., Tovey, C.A.: Connect the dots: how many random points can a regular curve pass through? Adv. Appl. Probab. 37(3), 571–603 (2005)
2. Arias-Castro, E., Chen, G., Lerman, G.: Spectral clustering based on local linear approximations. Electron. J. Stat. 5, 1537–1587 (2011)
3. Bargiela, A., Hartley, J.K.: Orthogonal linear regression algorithm based on augmented matrix formulation. Comput. Oper. Res. 20, 829–836 (1993)
4. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 11 (2011)
5. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
6. Candès, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
7. David, G., Semmes, S.: Singular integrals and rectifiable sets in $$\mathbb{R}^n$$Rn: au-delà des graphes Lipschitziens. Astérisque 193, 1–145 (1991)
8. Deming, W.E., Colcord, C.G.: The minimum in the gamma function. Nature 135(3422), 917 (1935)
9. Ding, C., Zhou, D., He, X., Zha, H.: R1-PCA: rotational invariant $${L}_1$$L1-norm principal component analysis for robust subspace factorization. In: ICML ’06: Proceedings of the 23rd international Conference on Machine Learning, pp. 281–288. ACM, New York (2006)
10. Dodge, Y.: An introduction to $$L_1$$L1-norm based statistical data analysis. Comput. Stat. Data Anal. 5(4), 239–253 (1987)
11. Donoho, D.L.: For most large underdetermined systems of equations, the minimal $$l_1$$l1-norm near-solution approximates the sparsest near-solution. Commun. Pure Appl. Math. 59(7), 907–934 (2006)
12. Donoho, D.L.: For most large underdetermined systems of linear equations the minimal $$l_1$$l1-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)
13. Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (electronic) (1999)
14. Elhamifar, E., Vidal, R.:. Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)
15. Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
16. Golub, G., Loan, C.V.: Matrix Computations. John Hopkins University Press, Baltimore (1996)
17. Hardt, M., Moitra, A.: Can we reconcile robustness and efficiency in unsupervised learning? In: Proceedings of the Twenty-Sixth Annual Conference on Learning Theory (COLT 2013) (2013)
18. Harter, H.L.: The method of least squares and some alternatives: Part II. Int. Stat. Rev. 42(3), 235–264+282 (1974)
19. Harter, H.L.: The method of least squares and some alternatives. I. Int. Stat. Rev. 42, 147–174 (1974)
20. Huber, P.J., Ronchetti, E.: Robust Statistics. Wiley Series in Probability and Mathematical Statistics. Probability and Mathematical Statistics. Wiley, New York (2009)
21. Lerman, G., Maunu, T.: Fast algorithm for robust subspace recovery (2014). arXiv:1406.6145
22. Lerman, G., Zhang, T.: Robust recovery of multiple subspaces by geometric $${{l_p}}$$lp minimization. Ann. Stat. 39(5), 2686–2715 (2011)
23. Lerman, G., McCoy, M., Tropp, J.A., Zhang, T.: Robust computation of linear models, or how to find a needle in a haystack. Found. Math. Comput (2014) (to appear)
24. Locantore, N., Marron, J., Simpson, D., Tripoli, N., Zhang, J., Cohen, K., Boente, G., Fraiman, R., Brumback, B., Croux, C., Fan, J., Kneip, A., Marden, J., Robust, D.P.: Principal component analysis for functional data. TEST 8(1), 1–73 (1999)
25. Maronna, R.A., Martin, R.D., Yohai, V.J.: Robust Statistics: Theory and Methods. Wiley Series in Probability and Statistics. Wiley, Chichester (2006)
26. Mattila, P.: Geometry of Sets and Measures in Euclidean Spaces. Cambridge University Press, Cambridge (1995)
27. McCoy, M., Tropp, J.: Two proposals for robust PCA using semidefinite programming. Electron. J. Stat. 5, 1123–1160 (2011)
28. Nyquist, H.: Least orthogonal absolute deviations. Comput. Stat. Data Anal. 6(4), 361–367 (1988)
29. Osborne, M.R., Watson, G.A.: An analysis of the total approximation problem in separable norms, and an algorithm for the total $$l_1 $$l1 problem. SIAM J. Sci. Stat. Comput. 6(2), 410–424 (1985)
30. Qi, F., Guo, B.-N., Chen, C.-P.: The best bounds in Gautschi-Kershaw inequalities. Math. Inequal. Appl. 9(3), 427–436 (2006)
31. Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. John Wiley & Sons Inc., New York (1987)
32. Soltanolkotabi, M., Candés, E.J.: A geometric analysis of subspace clustering with outliers. Ann. Stat. 40(4), 2195–2238 (2012)
33. Soltanolkotabi, M., Elhamifar, E., Candès, E.J.: Robust subspace clustering. CoRR (2013). arXiv:1301.2603
34. Späth, H., Watson, G.A.: On orthogonal linear approximation. Numer. Math. 51, 531–543 (1987)
35. Szarek, S.J.: The finite-dimensional basis problem with an appendix on nets of Grassmann manifolds. Acta Math. 151(3–4), 153–179 (1983)
36. Szarek, S.J.: Metric entropy of homogeneous spaces. In: Quantum Probability (Gdańsk, 1997), vol. 43 of Banach Center Publication, pp. 395–410. Polish Academy Sciences, Warsaw (1998)
37. Torr, P.H.S., Zisserman, A.: Robust computation and parametrization of multiple view relations. In: ICCV ’98: Proceedings of the Sixth International Conference on Computer Vision, p. 727. IEEE Computer Society, Washington (1998)
38. Torr, P.H.S., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Understand. 78(1), 138–156 (2000)
39. Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. In: Eldar, Y.C., Kutyniok, G. (eds.) Compressed Sensing: Theory and Applications. Cambridge University Press (to appear)
40. Watson, G.A.: Some Problems in Orthogonal Distance and Non-Orthogonal Distance Regression. Defense Technical Information Center (2001)
41. Watson, G.A.: On the gauss-newton method for $$l_1$$l1 orthogonal distance regression. IMA J. Numer. Anal. 22(3), 345–357 (2002)
42. Wong, Y.-C.: Differential geometry of Grassmann manifolds. Proc. Natl Acad. Sci. USA 57, 589–594 (1967)
43. Xu, H., Caramanis, C., Sanghavi, S.: Robust PCA via outlier pursuit. In: NIPS, pp. 2496–2504, 2010
44. Xu, H., Caramanis, C., Sanghavi, S.: Robust PCA via outlier pursuit. IEEE Trans. Inf. Theory 58(5), 3047–3064 (2012)
45. Yan, J., Pollefeys, M.: A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and nondegenerate. ECCV 4, 94–106 (2006)
46. Zhang, T., Lerman, G.: A novel m-estimator for robust PCA. J. Mach. Learn. Res. 15, 749–808 (2014)
47. Zhang, T., Szlam, A., Lerman, G.: Median $$K$$K-flats for hybrid linear modeling with many outliers. In: 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops). pp. 234–241. Kyoto, Japan (2009)
48. Zhang, T., Szlam, A., Wang, Y., Lerman, G.: Randomized hybrid linear modeling by local best-fit flats. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1927–1934, June 2010
49. Zhang, T., Szlam, A., Wang, Y., Lerman, G.: Hybrid linear modeling via local best-fit flats. Int. J. Comput. Vis. 100, 217–240 (2012)