Robust stochastic principal component analysis

Journal of Machine Learning Research

Volumn 33, Issue , 2014, Pages 266-274

  Goes, John ^a   Zhang, Teng ^b   Arora, Raman ^c   Lerman, Gilad ^a  

^a UNIVERSITY OF MINNESOTA   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; NUMERICAL METHODS; STATISTICS; STOCHASTIC SYSTEMS;

DIMENSIONAL SUBSPACE; MATRIX EXPONENTIATED GRADIENTS; RUN TIME COMPLEXITY; STANDARD ALGORITHMS; STOCHASTIC APPROXIMATION ALGORITHMS; STOCHASTIC PRINCIPALS; STOCHASTIC SETTINGS; STORAGE REQUIREMENTS;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84955462631     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (45)

1. R. Arora, A. Cotter, K. Livescu, and N. Srebro. Stochastic optimization for PCA and PLS. In Allerton Conference, pages 861-868, 2012.
2. R. Arora, A. Cotter, and N. Srebro. Stochastic optimization of PCA with capped MSG. In Advances in Neural Information Processing Systems (NIPS), 2013.
3. R. Basri and D. Jacobs. Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2):218-233, February 2003.
4. L. Bottou and O. Bousquet. The tradeoffs of large scale learning. In NIPS'07, pages 161-168, 2007.
5. M. Brand. Incremental singular value decomposition of uncertain data with missing values. In ECCV, pages 707-720, 2002.
6. T. Budavári, V. Wild, A. S. Szalay, L. Dobos, and C.-W. Yip. Reliable eigenspectra for new generation surveys. Monthly Notices of the Royal Astronomical Society, 394(3):1496-1502, 2009.
7. E. J. Candès, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? Journal of the ACM (JACM), 58(3):11, 2011.
8. N. Cesa-Bianchi, A. Conconi, and C. Gentile. On the generalization ability of on-line learning algorithms. Information Theory, IEEE Transactions on, 50(9):2050-2057, 2004.
9. V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky. Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim., 21(2):572-596, 2011.
10. M. Collins, A. Globerson, T. Koo, X. Carreras, and P. L. Bartlett. Exponentiated gradient algorithms for conditional random fields and max-margin markov networks. J. Mach. Learn. Res., 9:1775-1822, June 2008.
11. C. Croux and A. Ruiz-Gazen. High breakdown estimators for principal components: the projection-pursuit approach revisited. Journal of Multivariate Analysis, 95(1):206-226, 2005.
12. F. De La Torre and M. J. Black. A framework for robust subspace learning. International Journal of Computer Vision, 54(1-3):117-142, 2003.
13. L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Comput. Vis. Image Underst., 106(1):59-70, Apr. 2007.
14. J. Feng, H. Xu, and S. Yan. Robust PCA in highdimension: A deterministic approach. International conf. on machine learning (ICML), 2012.
15. J. Feng, H. Xu, and S. Yan. Online robust PCA via stochastic optimization. In Advances in Neural Information Processing Systems (NIPS), 2013.
16. O. L. Fèvre et al. The VIMOS VLT deep survey. first epoch VVDS-Deep survey: 11564 spectra with 17.5
17. M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
18. R. Gnanadesikan and J. R. Kettenring. Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics, pages 81-124, 1972.
19. F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. Robust Statistics: The Approach Based on Influence Functions. Wiley-Interscience, New York, April 2005.
20. M. Hardt and A. Moitra. Can we reconcile robustness and efficiency in unsupervised learning? In Proceedings of the Twenty-sixth Annual Conference on Learning Theory (COLT 2013), 2013.
21. J. He, L. Balzano, and J. Lui. Online robust sub-space tracking from partial information. arXiv preprint arXiv:1109.3827, 2011.
22. P. J. Huber and E. M. Ronchetti. Robust Statistics. Wiley Series in Probability and Statistics. Wiley, Hoboken, NJ, 2nd edition, 2009.
23. M. Hubert, P. J. Rousseeuw, and K. V. Branden. ROBPCA: a new approach to robust principal component analysis. Technometrics, 47(1), 2005.
24. I. T. Jolliffe. Principal Component Analysis. Springer Series in Statistics. Springer, 2nd edition, 2002.
25. 1 norm factorization in the presence of outliers and missing data by alternative convex programming. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), pages 739-746, June 2005.
26. K. C. Lee, J. Ho, and D. Kriegman. Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intelligence, 27(5):684-698, 2005.
27. G. Lerman, M. McCoy, J. A. Tropp, and T. Zhang. Robust computation of linear models, or how to find a needle in a haystack. ArXiv e-prints, Feb. 2012.
28. p-Recovery of the Most Significant Subspace among Multiple Subspaces with Outliers. ArXiv e-prints, Dec. 2010.
29. p minimization. Ann. Statist, 39(5):2686-2715, 2011.
30. G. Li and Z. Chen. Projection-Pursuit Approach to Robust Dispersion Matrices and Principal Components: Primary Theory and Monte Carlo. Journal of the American Statistical Association, 80(391):759-766, 1985.
31. Y. Li. On incremental and robust subspace learning. Pattern recognition, 37(7):1509-1518, 2004.
32. R. A. Maronna, R. D. Martin, and V. J. Yohai. Robust statistics: Theory and methods. Wiley Series in Probability and Statistics. John Wiley & Sons Ltd., Chichester, 2006.
33. M. McCoy and J. Tropp. Two proposals for robust PCA using semidefinite programming. Elec. J. Stat., 5:1123-1160, 2011.
34. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19(4):1574-1609, January 2009.
35. B. Recht, M. Fazel, and P. A. Parrilo. Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM review, 52(3):471-501, 2010.
36. S. Shalev-Shwartz, Y. Singer, and N. Srebro. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In ICML'07, pages 807-814, 2007.
37. S. Shalev-Shwartz and N. Srebro. SVM optimization: Inverse dependence on training set size. In ICML'08, pages 928-935, 2008.
38. 1 regularized loss minimization. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML'09, pages 929-936, 2009.
39. N. Srebro, K. Sridharan, and A. Tewari. On the universality of online mirror descent. In Advances in Neural Information Processing Systems 24, pages 2645-2653, 2011.
40. M. K. Warmuth and D. Kuzmin. Randomized Online PCA Algorithms with Regret Bounds that are Logarithmic in the Dimension. Journal of Machine Learning Research, 9:2287-2320, Oct. 2008.
41. H. Xu, C. Caramanis, and S. Mannor. Principal component analysis with contaminated data: The high dimensional case. In COLT, 2010.
42. H. Xu, C. Caramanis, and S. Sanghavi. Robust PCA via outlier pursuit. Information Theory, IEEE Transactions on, PP(99):1, 2012.
43. L. Xu and A. Yuille. Robust principal component analysis by self-organizing rules based on statistical physics approach. Neural Networks, IEEE Transactions on, 6(1):131-143, 1995.
44. T. Zhang and G. Lerman. A novel m-estimator for robust PCA. Submitted, available at arXiv:1112.4863.
45. T. Zhang, A. Szlam, and G. Lerman. Median K-flats for hybrid linear modeling with many outliers. In Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on Computer Vision, pages 234-241, Kyoto, Japan, 2009.