Robust PCA as bilinear decomposition with outlier-sparsity regularization

IEEE Transactions on Signal Processing

Volumn 60, Issue 10, 2012, Pages 5176-5190

  Mateos, Gonzalo ^a   Giannakis, Georgios B ^a  

^a University of Minnesota   (United States)

Author keywords

(Group) Lasso; outlier rejection; principal component analysis; robust statistics; sparsity

Indexed keywords

(GROUP) LASSO; COMPRESSIVE SAMPLING; CONVEX RELAXATION; DIMENSIONALITY REDUCTION; FACTOR ANALYSIS MODEL; FEATURE SPACE; HIGH DIMENSIONAL DATA; OUTLIER REJECTION; PERSONALITY ASSESSMENTS; PREFERENCE MEASUREMENT; PRINCIPAL COMPONENTS; REAL TIME; REGULARIZATION PARAMETERS; ROBUST ESTIMATORS; ROBUST PCA; ROBUST STATISTICS; ROBUSTIFICATION; SCALABLE ALGORITHMS; SELECTION OPERATORS; SIGNAL SUB-SPACE; SOCIAL NETWORKS; SPARSITY; SYNTHETIC AND REAL DATA; VARIABLE SELECTION; VIDEO SURVEILLANCE;

BIOINFORMATICS; COMPUTER VISION; MATRIX ALGEBRA; PRINCIPAL COMPONENT ANALYSIS; RELAXATION PROCESSES; SECURITY SYSTEMS;

STATISTICS;

EID: 84866502467     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2204986     Document Type: Article

References (45)

1. D. Angelosante, J. A. Bazerque, and G. B. Giannakis, "Online adaptive estimation of sparse signals: Where RLS meets the-norm," IEEE Trans. Signal Process., vol. 58, pp. 3436-3447, Jul. 2010.
2. L. Balzano, R. Nowak, and B. Recht, "Online identification and tracking of subspaces from highly incomplete information," in Proc. 48th Allerton Conf., Monticello, IL, Sep./Oct. 2010, pp. 704-711.
3. D. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena-Scientific, 1999.
4. N. A. Campbell, "Robust procedures in multivariate analysis I: Robust covariance estimation," Appl. Stat., vol. 29, pp. 231-237, 1980.
5. E. J. Candes, X. Li, Y. Ma, and J. Wright, "Robust principal component analysis?," J. ACM, vol. 58, Mar. 2011, Article No. 11.
6. E. J. Candes, M. B. Wakin, and S. Boyd, "Enhancing sparsity by reweighted minimization," J. Fourier Anal. Appl., vol. 14, pp. 877-905, Dec. 2008.
7. V. Chandrasekaran, S. Sanghavi, P. A. Parillo, and A. S. Willsky, "Rank-sparsity incoherence for matrix decomposition," SIAM J. Optim., vol. 21, pp. 572-596, 2011.
8. F. de la Torre and M. J. Black, "A framework for robust subspace learning," Int. J. Comput. Vis., vol. 54, pp. 1183-209, 2003.
9. X. Ding, L. He, and L. Carin, "Bayesian robust principal component analysis," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3419-3430, 2011.
10. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Stat. Assoc., vol. 96, pp. 1348-1360, 2001. (Pubitemid 33695585)
11. M. Fazel, H. Hindi, and S. Boyd, "Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices," in Proc. Amer. Control Conf., Denver, CO, Jun. 2003, pp. 2156-2162.
12. P. Forero, V. Kekatos, and G. B. Giannakis, "Outlier-aware robust clustering," in Proc. Int. Conf. Acoust., Speech, Signal Process., Prague, Czech Republic, May 2011, pp. 2244-2247.
13. J. J. Fuchs, "An inverse problem approach to robust regression," in Proc. Int. Conf. Acoust., Speech, Signal Process., Phoenix, AZ, Mar. 1999, pp. 180-188.
14. G. B. Giannakis, G. Mateos, S. Farahmand, V. Kekatos, and H. Zhu, "USPACOR: Universal sparsity-controlling outlier rejection," in Proc. Int. Conf. Acoust., Speech, Signal Process., Prague, Czech Republic, May 2011, pp. 1952-1955.
15. M. Girvan and M. E. J. Newman, "Community structure in social and biological networks," in Proc. Natl. Acad. Sci., 2002, vol. 99, pp. 7821-7826. (Pubitemid 34650963)
16. L. R. Goldberg, "The Eugene-Springfield community sample: Information available from the research participants," Oregon Research Institute, Eugene, OR, 2008, vol. 48.
17. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York: Springer, 2009.
18. J. He, L. Balzano, and J. C. S. Lui, "Online robust subspace tracking from partial information," 2011 [Online]. Available: http://arxiv.org/abs/1109.3827
19. P. J. Huber and E. Ronchetti, Robust Statistics. New York: Wiley, 2009.
20. Y. Jin and B. D. Rao, "Algorithms for robust linear regression by exploiting the connection to sparse signal recovery," in Proc. Int. Conf. Acoust., Speech, Signal Process., Dallas, TX, Mar. 2010, pp. 3830-3833.
21. O. P. John, L. P. Naumann, and C. J. Soto, "Paradigm shift to the inte-grative big-five trait taxonomy: History, measurement, and conceptual issues," in Handbook of Personality: Theory and Research, O.P. John, R.W. Robins, and L.A. Pervin, Eds. New York: Guilford Press, 2008.
22. I. T. Jolliffe, Principal Component Analysis. New York: Springer, 2002.
23. V. Kekatos and G. B. Giannakis, "From sparse signals to sparse residuals for robust sensing," IEEE Trans. Signal Process., vol. 59, pp. 3355-3368, Jul. 2011.
24. J. Mairal, J. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Jan. 2010.
25. M. Mardani, G. Mateos, and G. B. Giannakis, "In-network sparsity-regularized rank minimization: Algorithms and applications," IEEE Trans. Signal Process., 2012 [Online]. Available: http://arxiv.org/abs/1203. 1570, submitted for publication
26. B. K. Natarajan, "Sparse approximate solutions to linear systems," SIAM J. Comput., vol. 24, pp. 227-234, 1995.
27. A. T. Puig, A. Wiesel, and A. O. Hero, "Multidimensional shrinkage-thresholding operator and group LASSO penalties," IEEE Signal Process. Lett., vol. 18, pp. 363-366, Jun. 2011.
28. I. Ramirez, F. Lecumberry, and G. Sapiro, "Universal priors for sparse modeling," in Proc. 3rd Int. Workshop Comput. Adv. Multi-Sensor Adapt. Process., Aruba, Dutch, Antilles, Dec. 2009, pp. 197-200.
29. B. Recht, M. Fazel, and P. A. Parrilo, "Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization," SIAM Rev., vol. 52, pp. 471-501, 2010.
30. B. Recht and C. Re, "Parallel stochastic gradient algorithms for large-scale matrix completion," 2011 [Online]. Available: http://pages.cs. wisc.edu/brecht/papers/11.Rec.Re.IPGM.pdf
31. S. P. Reise and N. G. Waller, "Traitedness and the assessment of response pattern scalability," J. Personality Social Psychol., vol. 65, pp. 143-151, 1993.
32. P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection. New York: Wiley, 1987.
33. B. Scholkopf, R. Herbrich, and A. J. Smola, "A generalized representer theorem," Computation Learning Theory, ser. Lecture Notes in Computer Science, vol. 2111, pp. 416-426, 2001. (Pubitemid 33312837)
34. B. Scholkopf, A. J. Smola, and K. R. Muller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, pp. 1299-1319, 1998. (Pubitemid 128463674)
35. M. Storer, P. M. Roth, M. Urschler, and H. Bischof, "Fast-robust PCA," Image Analysis, ser. Lecture Notes in Computer Science, vol. 5575, pp. 430-439, 2009.
36. I. Tošic and P. Frossard, "Dictionary learning," IEEE Signal Process. Mag., vol. 28, pp. 27-38, Mar. 2010.
37. J. Tropp, "Just relax: Convex programming methods for identifying sparse signals," IEEE Trans. Inf. Theory, vol. 51, pp. 1030-1051, Mar. 2006. (Pubitemid 46444890)
38. P. Tseng, "Convergence of block coordinate descent method for non-differentiable maximization," J. Optim. Theory Appl., vol. 109, pp. 473-492, 2001.
39. N. Waller and S. Reise, "Measuring psychopathology with nonstandard irt models: Fitting the four parameter model to the MMPI," in New Directions in Psychological Measurement With Model-Based Approaches, S. Embretson, Ed. Washington, DC: Amer. Psych. Assoc.
40. H. Xu, C. Caramanis, and S. Sanghavi, "Robust PCA via outlier pursuit," IEEE Trans. Inf. Theory, vol. 58, pp. 3047-3064, May 2012.
41. L. Xu and A. L. Yuille, "Robust principal component analysis by self-organizing rules based on statistical physics approach," IEEE Trans. Neural Netw., vol. 6, pp. 131-143, Jan. 1995.
42. B. Yang, "Projection approximation subspace tracking," IEEE Trans. Signal Process., vol. 43, pp. 95-107, Jan. 1995.
43. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Roy. Stat. Soc B, vol. 68, pp. 49-67, 2006. (Pubitemid 43415335)
44. Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, "Stable principal component pursuit," in Proc. Int. Symp. Inf. Theory, Austin, TX, Jun. 2010, pp. 1518-1522.
45. H. Zou, T. Hastie, and R. Tibshirani, "Sparse principal component analysis," J. Comput. Graph. Stat., vol. 15, no. 2, pp. 265-286, 2006. (Pubitemid 43935978)