Sparsity constrained nonlinear optimization: Optimality conditions and algorithms

SIAM Journal on Optimization

Volumn 23, Issue 3, 2013, Pages 1480-1509

  Beck, Amir ^a   Eldar, Yonina C ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Compressed sensing; Numerical methods; Optimality conditions; Sparsity constrained problems; Stationarity

Indexed keywords

CONSTRAINED NON-LINEAR OPTIMIZATIONS; CONSTRAINED PROBLEM; CONTINUOUSLY DIFFERENTIABLE; GRADIENT PROJECTION METHODS; ITERATIVE HARD THRESHOLDING; NUMERICAL ALGORITHMS; OPTIMALITY CONDITIONS; STATIONARITY;

ALGORITHMS; COMPRESSED SENSING; CONSTRAINED OPTIMIZATION; NUMERICAL METHODS;

ITERATIVE METHODS;

EID: 84886247578     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120869778     Document Type: Article

References (29)

1. S. Bahmani, B. Raj, and P. Boufounos, Greedy Sparsity-Constrained Optimization, arxiv:1203.5483v2.pdf, 2012.
2. A. Beck and M. Teboulle, Gradient-based algorithms with applications to signal recovery problems, in Convex Optimization in Signal Processing and Communications, Y. Eldar and D. Palomar, eds., Cambridge University Press, Cambridge, UK, 2010.
3. E. V. D. Berg and M. P. Friedlander, Sparse optimization with least-squares constraints, SIAM J. Optim., 21 (2011), pp. 1201-1229.
4. D. P. Bertsekas, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
5. T. Blumensath, Compressed Sensing with Nonlinear Observations, http://users.fmrib.ox.ac. uk/~tblumens/papers/B Nonlinear.pdf (2010).
6. T. Blumensath and M. E. Davies, Iterative thresholding for sparse approximations, J. Fourier Anal. Appl., 14 (2008), pp. 629-654.
7. T. Blumensath and M. E. Davies, Normalised iterative hard thresholding: Guaranteed stability and performance, IEEE J. Selected Topics in Signal Processing, 4 (2010), pp. 298-309.
8. E. Candés, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
9. G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Constr. Approx., 13 (1997), pp. 57-98.
10. R. DeVore, Nonlinear approximation, Acta Numerica, 7 (1998), pp. 51-150.
11. D. Donoho, Denoising by soft-thresholding, IEEE Trans. Inform. Theory, 41 (1995), pp. 613-627.
12. D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306
13. D. L. Donoho and M. Elad, Optimally sparse representation in general (non-orthogonal) dictionaries via l1 minimization, in Proc. Natl. Acad. Sci. USA, 100 (2003), pp. 2197- 2202.
14. A. Szameit et al., Sparsity-based single-shot sub-wavelength coherent diffractive imaging, Nature Materials, 11 (2012), pp. 455-459.
15. J. R. Fienup, Phase retrieval algorithms: A comparison, Appl. Optics, 21 (1982), pp. 2758-2769.
16. R. W. Gerchberg and W. O. Saxton, A practical algorithm for the determination of phase from image and diffraction plane pictures, Optik, 35 (1972), pp. 237-246.
17. I. F. Gorodnitsky and B. D. Rao, Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm, IEEE Trans. Signal Process., 45 (1997), pp. 600-616.
18. N. Hurt, Phase Retrieval and Zero Crossings, Kluwer Academic, Norwell, MA, 1989.
19. M. A. Davenport, M. F. Duarte, Y. C. Eldar, and G. Kutyniok, Introduction to Compressed Sensing, in Compressed Sensing: Theory and Applications, Cambridge University Press, Cambridge, MA, 2012.
20. S. Mallat, A Wavelet Tour of Signal Processing: The Sparse Way, Academic Press, New York, 2008.
21. S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Signal Process., 41 (1993), pp. 3397-3415.
22. B. Olshausen and D. Field, Emergence of simple-cell receptive field properties by learning sparse representation Nature 381 (1996) pp. 607-609.
23. S. Scholtes, Nonconvex structures in nonlinear programming, Oper. Res., 52 (2004), pp. 368-383.
24. Y. Shechtman, Y. C. Eldar, A. Szameit, and M. Segev, Sparsity-based sub-wavelength imaging with partially spatially incoherent light via quadratic compressed sensing, Optics Express, 19 (2011), pp. 14807-14822.
25. D. Taubman and M. Marcellin, JPEG 2000: Image Compression Fundamentals, Standards and Practice, Kluwer, Dordrecht, Netherlands, 2001.
26. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Royal Statist. Soc. B, 58 (1996), pp. 267-288.
27. J. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. Inform. Theory, 50 (2004), pp. 2231-2242.
28. J. Tropp and S. J. Wright, Computational methods for sparse solution of linear inverse problems, Proc. IEEE, 98 (2010), pp. 948-958.
29. R. Vershynin, Introduction to the Non-asymptotic Analysis of Random Matrices, in Compressed Sensing: Theory and Applications, Cambridge University Press, Cambridge, UK, 2012.