A variable fixing version of the two-block nonlinear constrained Gauss–Seidel algorithm forℓ1-Regularized least-squares

Computational Optimization and Applications

Volumn 59, Issue 3, 2014, Pages 565-589

  Porcelli, Margherita ^a   Rinaldi, Francesco ^b  

^a UNIVERSITY OF BOLOGNA   (Italy)

^b UNIVERSITY OF PADOVA   (Italy)

Author keywords

Active set; Gauss Seidel algorithm; Sparse approximation; 1 Regularized least squares

Indexed keywords

APPROXIMATION ALGORITHMS; GAUSSIAN DISTRIBUTION; ITERATIVE METHODS;

ACTIVE SETS; ALGORITHM FOR SOLVING; GLOBAL CONVER-GENCE; LEAST-SQUARES APPROACH; NUMERICAL EXPERIMENTS; REGULARIZED LEAST SQUARES; SPARSE APPROXIMATIONS; SPARSE SOLUTIONS TO UNDERDETERMINED SYSTEMS;

ALGORITHMS;

EID: 84926636639     PISSN: 09266003     EISSN: 15732894     Source Type: Journal    
DOI: 10.1007/s10589-014-9653-0     Document Type: Article

References (29)

1. Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2–45 (2010)
2. Beck, A., Teboulle, M.: A fast iterative shrinkage–threshold algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)
3. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
4. Bioucas-Dias, J., Figueiredo, M.: A new twist: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16, 2992–3004 (2007)
5. Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)
6. Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
7. Candès, E., Romberg, J.: Quantitative robust uncertainty principles and optimally sparse decompositions. Found. Comput. Math. 6(2), 227–254 (2006)
8. Candès, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
9. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition basis pursuit. SIAM Rev. 43, 129–159 (2001)
10. Daubechies, I., Defrise, M., Mol, C.D.: An iterative thresholding algorithm for linear inverse problems with sparsity constraints. Commun. Pure Appl. Math. 57, 1413–1457 (2004)
11. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program 91, 201221 (2002)
12. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
13. Facchinei, F., Fischer, A., Kanzow, C.: On the accurate identification of active constraints. SIAM J. Optim. 9, 14–32 (1999)
14. Figueiredo, M., Nowak, R., Wright, S.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1, 586–598 (2007)
15. Fletcher, R.: A class of methods for nonlinear programming with termination and convergence properties. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 157–175. North-Holland, The Netherlands (1970)
16. Fountoulakis, K., Gondzio, J., Zhlobich, P.: Matrix-free interior point method for compressed sensing problems. Math. Program. Comput. 6(1), 1–31 (2014)
17. Grippo, L., Sciandrone, M.: Globally convergent block-coordinate techniques for unconstrained optimization. Optim. Methods Softw. 10(4), 587–637 (1999)
18. Grippo, L., Sciandrone, M.: On the convergence of the block nonlinear Gauss–Seidel method under convex constraints. Oper. Res. Lett. 26(3), 127–136 (2000)
19. Hale, E.T., Yin, W., Zhang, Y.: A numerical study of fixed-point continuation applied to compressed sensing. J. Comput. Math. 28(2), 170194 (2010)
20. Haynsworth, E.V.: On the Schur Complement. Basel Mathematical Notes, BNB 20. Wiley, Hoboken (1968)
21. Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Methods Appl. 4, 303–320 (1969)
22. Kim, S.-J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale l1-regularized least squares. IEEE J. Sel. Top. Signal Process. 1(4), 606–617 (2007)
23. Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)
24. Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B 58(1), 267–288 (1996)
25. Tseng, P., Yun, S.: A coordinate gradient descent method for nonsmooth separable minimization. Math. Program. 117(1), 387–423 (2009)
26. Wen, Z., Yin, W., Goldfarb, D., Zhang, Y.: A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation. SIAM J. Sci. Comput. 32(4), 1832–1857 (2010)
27. Wen, Z., Yin, W., Zhang, H., Goldfarb, D.: On the convergence of an active-set method for $$\ell _1$$ℓ1 minimization. Optim. Methods Softw. 27(6), 1127–1146 (2012)
28. Wright, S., Nowak, R., Figueiredo, M.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479–2493 (2009)
29. Zhang, J.J., Morini, B.: Solving regularized linear least-squares problems by alternating direction methods with applications to image restoration. Electron. Trans. Numer. Anal. (ETNA) 40, 356–372 (2013)