Linearized alternating direction method with parallel splitting and adaptive penalty for separable convex programs in machine learning

Journal of Machine Learning Research

Volumn 29, Issue , 2013, Pages 116-132

  Liu, Risheng ^a   Lin, Zhouchen ^b   Su, Zhixun ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^b Peking University   (China)

Author keywords

Adaptive penalty; Convex programs; Linearized alternating direction method; Matrix completion; Parallel splitting; Subspace clustering

Indexed keywords

ADAPTIVE PENALTY; ALTERNATING DIRECTION METHODS; CONVEX PROGRAMS; MATRIX COMPLETION; PARALLEL SPLITTING; SUB-SPACE CLUSTERING;

EID: 84908476631     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (31)

1. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sciences, 2(1):183-202, 2009.
2. S. Boyd and L. Van den berghe. Convex optimization. Cambridge University Press, 2004.
3. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. In M. Jordan, editor, Foundations and Trends in Machine Learning, 2010.
4. J. Cai, E. Candés, and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM J. Optimization, 20(4):1956-1982, 2010.
5. E. Candés and B. Recht. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717-772, 2009.
6. E. Candés, X. Li, Y. Ma, and J. Wright. Robust principal component analysis? J. ACM, 58(3):No.11, 2011.
7. V. Chandrasekaran, P. Parrilo, and A. Willsky. Latent variable graphical model selection via convex optimization. The Annals of Statistics, 40(4):1935-1967, 2012.
8. E. Esser. Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Report 09-31, UCLA, 2009.
9. P. Favaro, R. Vidal, and A. Ravichandran. A closed form solution to robust subspace estimation and clustering. In CVPR, 2011.
10. M. Fazel. Matrix rank minimization with applications. PhD thesis, 2002.
11. M. Fortin and R. Glowinski. Augmented Lagrangian methods. North-Holland, 1983.
12. D. Goldfarb and S. Ma. Fast multiple splitting algorithms for convex optimization. SIAM J. Optimization, 22(2):533-556, 2012.
13. T. Goldstein and S. Osher. The split Bregman method for L1 regularized problems. SIAM J. Imaging Sciences, 2(2):323-343, 2008.
14. B. S. He and X. Yuan. On the O(1/t) convergence rate of alternating direction method. Preprint, 2011.
15. B. S. He and X. Yuan. Linearized alternating direction method with Gaussian back substitution for separable convex programming. Numerical Algebra, Control and Optimization, 3(2):247-260, 2013.
16. B. S. He, M. Tao, and X. Yuan. Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optimization, 22(2):313-340, 2012.
17. H. Ji, C. Liu, Z. Shen, and Y. Xu. Robust video denoising using low rank matrix completion. In CVPR, 2010.
18. R. Larsen. Lanczos bidiagonalization with partial reorthogonalization. Department of Computer Science, Aarhus University, Technical report, DAIMI PB-357, 1998.
19. Z. Lin, M. Chen, and Y. Ma. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report UILU-ENG-09-2215, arXiv:1009.5055, 2009a.
20. Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma. Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix. UIUC Technical Report UILU-ENG-09-2214, 2009b.
21. Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rank representation. In NIPS, 2011.
22. G. Liu and S. Yan. Latent low-rank representation for subspace segmentation and feature extraction. In ICCV, 2011.
23. G. Liu, Z. Lin, and Y. Yu. Robust subspace segmentation by low-rank representation. In ICML, 2010.
24. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma. Robust recovery of subspace structures by low-rank representation. IEEE Trans. on PAMI, 35(1):171-184, 2012.
25. X. Shen and Y. Wu. A unified approach to salient object detection via low rank matrix recovery. In CVPR, 2012.
26. M. Tao. Some parallel splitting methods for separable convex programming with O(1/t) convergence rate. Preprint, 2011.
27. K. Toh and S. Yun. An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems. Pacific J. Optimization, 6(15):615-640, 2010.
28. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma. Robust face recognition via sparse representation. IEEE Trans. on PAMI, 31(2):210-227, 2009.
29. Y. Xu, W. Yin, and Z. Wen. An alternating direction algorithm for matrix completion with nonnegative factors. CAAM Technical Report TR11-03, 2011.
30. J. Yang and X. Yuan. Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization. Mathematics of Computation, 82(281):301-329, 2013.
31. J. Ye, S. Ji, and J. Chen. Multi-class discriminant kernel learning via convex programming. JMLR, 9:719-758, 2008.