A coordinate gradient descent method for l1-regularized convex minimization

Computational Optimization and Applications

Volumn 48, Issue 2, 2011, Pages 273-307

  Yun, Sangwoon ^a,b   Toh, Kim Chuan ^a  

^a SINGAPORE MIT ALLIANCE   (Singapore)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Compressed sensing; Convex optimization; Coordinate gradient descent; Image deconvolution; L1 Regularization; Linear least squares; Logistic regression; Q linear convergence

Indexed keywords

COMPRESSED SENSING; COORDINATE GRADIENT DESCENT; IMAGE DECONVOLUTION; L1- REGULARIZATION; LINEAR LEAST SQUARES; LOGISTIC REGRESSION; Q-LINEAR CONVERGENCE;

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; CONVEX OPTIMIZATION; FEATURE EXTRACTION; LINEAR SYSTEMS; OPTIMIZATION; REGRESSION ANALYSIS; SIGNAL RECONSTRUCTION; SURFACE RECONSTRUCTION;

LEAST SQUARES APPROXIMATIONS;

EID: 79955559521     PISSN: 09266003     EISSN: 15732894     Source Type: Journal    
DOI: 10.1007/s10589-009-9251-8     Document Type: Conference Paper

References (35)

1. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
2. Bradley, P.S., Fayyad, U.M., Mangasarian, O.L.: Mathematical programming for data mining: Formulations and challenges. INFORMS J. Comput. 11, 217-238 (1999)
3. Candès, E.J., Tao, T.: Nearly optimal signal recovery from random projections: Universal encoding strategies. IEEE Trans. Inf. Theory 52, 5406-5425 (2006) (Pubitemid 46445381)
4. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489-509 (2006) (Pubitemid 43174093)
5. Chang, C.-C., Lin, C.-J.: LIBSVM-A library for support vector machines. http://www.csie.ntu. edu.tw/~cjlin/libsvmtools/datasets/
6. Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33-61 (1999) (Pubitemid 128689501)
7. Daubechies, I., De Friese, M., De Mol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57, 1413-1457 (2004) (Pubitemid 39427442)
8. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289-1306 (2006)
9. 1-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54, 4789-4812 (2008)
10. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407-499 (2004) (Pubitemid 41250302)
11. Figueiredo, M., Nowak, R.: An EM algorithm for wavelet-based image restoration. IEEE Trans. Image Process. 12, 906-916 (2003)
12. Figueiredo, M., Nowak, R.: A bound optimization approach to wavelet-based image deconvolution. In: IEEE Int. Conf. on Image Processing-ICIP'05, 2005
13. Figueiredo, M., Nowak, R., Wright, S.J.: Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1, 586-598 (2007)
14. Fuchs, J.-J.: On sparse representations in arbitrary redundant bases. IEEE Trans. Inf. Theory 50, 1341-1344 (2004)
15. Genkin, A., Lewis, D., Madigan, D.: Large-scale Bayesian logistic regression for text categorization. Technometrics 49, 291-304 (2007) (Pubitemid 47292487)
16. Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D., Lander, E.S.: Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286, 531-537 (1999)
17. 1-regularized minimization with applications to compressed sensing. CAAM Technical report TR07-07, Department of Computational and Applied Mathematics, Rice University (July 2007)
18. 1-regularized least squares. IEEE J. Sel. Top. Signal Process. 1, 606-617 (2007) (Pubitemid 351266626)
19. 1-regularized logistic regression. J. Mach. Learn. Res. 8, 1519-1555 (2007) (Pubitemid 47210273)
20. 1-regularized logistic regression. In: Proceedings of the 21st National Conference on Artificial Intelligence, 2006
21. Lewis, D.D., Yang, Y., Rose, T.G., Li, F.: RCV1: A new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361-397 (2004)
22. Lin, C.-J., Weng, R.C., Keerthi, S.S.: Trust region Newton method for large-scale logistic regression. J. Mach. Learn. Res. 9, 627-650 (2008)
23. Luo, Z.-Q., Tseng, P.: Error bounds and convergence analysis of feasible descent methods: A general approach. Ann. Oper. Res. 46, 157-178 (1993)
24. Mangasarian, O.L., Musicant, D.R.: Large scale kernel regression via linear programming. Mach. Learn. 46, 255-269 (2002) (Pubitemid 34129971)
25. 2 regularization, and rotational invariance, In: Proceedings of the 21st International Conference on Machine Learning, 2004
26. Osborne,M., Presnell, B., Turlach, B.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20, 389-403 (2000)
27. 1 regularization-path algorithm for generalized linear models. J. R. Stat. Soc. B 69, 659-677 (2007)
28. Sardy, S., Tseng, P.: AMlet, RAMlet, and GAMlet: Automatic nonlinear fitting of additive models, robust and generalized, with wavelets. J. Comput. Graph. Stat. 13, 283-309 (2004) (Pubitemid 38845886)
29. Shi, W., Wahba, G., Wright, S.J., Lee, K., Klein, R., Klein, B.: Lasso-patternsearch algorithm with application to ophthalmology and genomic data. Stat. Interface 1, 137-153 (2008)
30. Starck, J.-L., Nguyen, M., Murtagh, F.: Wavelets and curvelets for image deconvolution: A combined approach. Signal Process. 83, 2279-2283 (2003)
31. Tropp, J.A.: Just relax: Convex programming methods for identifying sparse signals. IEEE Trans. Inf. Theory 51, 1030-1051 (2006) (Pubitemid 46444890)
32. Tsaig, Y., Donoho, D.: Extensions of compressed sensing. Signal Process. 86, 533-548 (2005)
33. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117, 387-423 (2009)
34. Wang, L.: Efficient regularized solution path algorithms with applications in machine learning and data mining. Ph.D. thesis, University of Michigan (2008)
35. Wright, S.J., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. (2007, to appear)