Outlier-robust PCA: The high-dimensional case

IEEE Transactions on Information Theory

Volumn 59, Issue 1, 2013, Pages 546-572

  Xu, Huan ^a   Caramanis, Constantine ^b   Mannor, Shie ^c  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b The University of Texas at Austin   (United States)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Dimension reduction; outlier; principal component analysis (PCA); robustness; statistical learning

Indexed keywords

BREAKDOWN POINTS; CORRUPTED DATA; DATA SETS; DIMENSION REDUCTION; HIGH-DIMENSIONAL; OPTIMAL SOLUTIONS; OUTLIER; REAL-WORLD APPLICATION; ROBUST ALGORITHM; ROBUST PRINCIPAL COMPONENT ANALYSIS; STATISTICAL LEARNING;

PRINCIPAL COMPONENT ANALYSIS; ROBUST CONTROL; ROBUSTNESS (CONTROL SYSTEMS); STATISTICS;

ALGORITHMS;

EID: 84871742039     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2012.2212415     Document Type: Article

References (41)

1. D. L. Donoho, "High-dimensional data analysis: The curses and blessings of dimensionality," presented at the presented at the Amer. Math. Soc.-Math. Challenges 21st Century, Los Angeles, CA, 2000.
2. I. M. Johnstone, "On the distribution of the largest eigenvalue in principal components analysis," Ann. Statist., vol. 29, no. 2, pp. 295-327, 2001.
3. B. Schölkopf and A. J. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
4. K. Pearson, "On lines and planes of closest fit to systems of points in space," Philosoph. Mag., vol. 2, no. 6, pp. 559-572, 1901.
5. I. T. Jolliffe, Principal Component Analysis, ser. Springer Series in Statistics. Berlin, Germany: Springer, 1986.
6. P. J. Huber, Robust Statistics. New York: Wiley, 1981.
7. S. J. Devlin, R. Gnanadesikan, and J. R. Kettenring, "Robust estimation of dispersion matrices and principal components," J. Amer. Statist. Assoc., vol. 76, no. 374, pp. 354-362, 1981.
8. L. Xu and A. L. Yuille, "Robust principal component analysis by selforganizing rules based on statistical physics approach," IEEE Trans. Neural Netw., vol. 6, no. 1, pp. 131-143, Jan. 1995.
9. T. N. Yang and S. D. Wang, "Robust algorithms for principal component analysis," Pattern Recognit. Lett., vol. 20, no. 9, pp. 927-933, 1999.
10. C. Croux and G. Hasebroeck, "Principal component analysis based on robust estimators of the covariance or correlation matrix: Influence functions and efficiencies," Biometrika, vol. 87, no. 3, pp. 603-618, 2000.
11. F. De la Torre and M. J. Black, "Robust principal component analysis for computer vision," in Proc. 8th Int. Conf. Comput. Vision, 2001, pp. 362-369.
12. F. De la Torre and M. J. Black, "A framework for robust subspace learning," Int. J. Comput. Vision, vol. 54, no. 1/2/3, pp. 117-142, 2003.
13. C. Croux, P. Filzmoser, and M. Oliveira, "Algorithms for Projection-Pursuit robust principal component analysis," Chemometrics Intell. Lab. Syst., vol. 87, no. 2, pp. 218-225, 2007.
14. S. C. Brubaker, "Robust PCA and clustering on noisy mixtures," in Proc. 19th Annu. ACM-SIAM Symp. Discrete Algorithms, 2009, pp. 1078-1087.
15. S. Dasgupta, "Subspace detection: A robust statistics formulation," in Proc. 16th Annu. Conf. Learning Theory, 2003, pp. 734-734.
16. A. Klivans, P. Long, and R. Servedio, "Learning halfspaces with malicious noise," J. Mach. Learning Res., vol. 10, pp. 2715-2740, 2009.
17. D. L. Donoho, Breakdown Properties ofMultivariate Location Estimators. Cambridge, MA, Harvard Univ., 1982, Qualifying paper.
18. M. Hubert, P. J. Rousseeuw, and K. Branden, "ROBPCA: A new approach to robust principal component analysis," Technometr., vol. 47, no. 1, pp. 64-79, 2005.
19. P. J. Rousseeuw, "Multivariate estimation with high breakdown point," in Mathematical Statistics and Applications, W. Grossman, G. Pflug, I. Vincze, and W. Wertz, Eds. Dordrecht, The Netherlands: Reidel, 1985, pp. 283-297.
20. R. Maronna, "Robust-estimators of multivariate location and scatter," Ann. Statist., vol. 4, pp. 51-67, 1976.
21. V. Barnett, "The ordering of multivariate data," J. Royal Statist. Soc. Series, A, vol. 138, pp. 318-344, 1976.
22. A. Bebbington, "A method of bivariate trimming for robust estimation of the correlation coefficient," Appl. Statist., vol. 27, pp. 221-228, 1978.
23. D. Titterington, "Estimation of correlation coefficients by ellipsoidal trimming," Appl. Statist., vol. 27, pp. 227-234, 1978.
24. J. Helbling, "Ellipsoïdes minimaux de couverture en statistique multivariée," Ph.D. dissertation, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, 1983.
25. V. Barnett and T. Lewis, Outliers in Statistical Data. NewYork: Wiley, 1978.
26. H. David, Order Statistics. New York: Wiley, 1981.
27. A. Dempster and M. Gasko-Green, "New tools for residual analysis," Ann. Statist., vol. 9, no. 5, pp. 945-959, 1981.
28. R. Gnanadesikan and J. R. Kettenring, "Robust estimates, residuals, and outlier detection with multiresponse data," Biometrics, vol. 28, pp. 81-124, 1972.
29. S. J. Devlin, R. Gnanadesikan, and J. R. Kettenring, "Robust estimation and outlier detection with correlation coefficients," Biometrika, vol. 62, pp. 531-545, 1975.
30. C. Croux and A. Ruiz-Gazen, "High breakdown estimators for principal components: the projection-pursuit approach revisited," J. Multivariate Anal., vol. 95, no. 1, pp. 206-226, 2005.
31. E. Candès and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, pp. 717-772, 2009.
32. B. Recht, A simpler approach to matrix completion 2009 [Online]. Available: ArXiv: 0910.0651
33. V. Chandrasekaran, S. Sanghavi, P. Parrilo, and A.Willsky, Rank-sparsity incoherence for matrix decomposition 2009 [Online]. Available: ArXiv:0906.2220
34. Z. Zhou, X. Li, J.Wright, E. Candès, and Y. Ma, Stable principal component pursuit 2010 [Online]. Available: ArXiv:1001.2363
35. H. Xu, C. Caramanis, and S. Sanghavi, "Robust PCA via outlier pursuit," in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 2496-2504.
36. H. Xu, C. Caramanis, and S. Sanghavi, "Robust PCA via outlier pursuit," IEEE Trans. Inf. Theory, vol. 58, no. 5, pp. 3047-3064, May 2012, to be published.
37. M. McCoy and J. Tropp, "Two proposals for robust PCA using semidefinite programming," Electron. J. Statist., vol. 5, pp. 1123-1160, 2011.
38. R. Vershynin, Introduction to the non-asymptotic analysis of random matrices 2011 [Online]. Available: ArXiv: 1011.3027v6
39. B. Schölkopf, A. J. Smola, and K. R. Müller, "Kernel principal component analysis," in Advances in Kernel Methods-Support Vector Learning, B. Schölkopf, C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 327-352.
40. E. Candès, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?," J. ACM, vol. 58, pp. 1-37, 2011.
41. K. Davidson and S. Szarek, "Local operator theory, random matrices and banach spaces," in Handbook on the Geometry of Banach Spaces, W. Johnson and J. Lindenstrauss, Eds. New York: Elsevier, 2001, pp. 317-366.