Solving basis pursuit: Heuristic optimality check and solver comparison

ACM Transactions on Mathematical Software

Volumn 41, Issue 2, 2015, Pages

  Lorenz, Dirk A ^a   Pfetsch, Marc E ^b   Tillmann, Andreas M ^b  

^a TECHNICAL UNIVERSITY OF BRAUNSCHWEIG   (Germany)

^b DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

Algorithms; Experimentation; Performance; Reliability; Verification

Indexed keywords

ALGORITHMS; COMPRESSED SENSING; LINEAR SYSTEMS; RELIABILITY; VERIFICATION;

BASIS PURSUITS; EARLY TERMINATION; EXPERIMENTATION; NUMERICAL COMPARISON; PERFORMANCE; RIGHT-HAND SIDES; STATE OF THE ART; UNDERDETERMINED LINEAR SYSTEMS;

OPTIMIZATION;

EID: 84922761775     PISSN: 00983500     EISSN: 15577295     Source Type: Journal    
DOI: 10.1145/2689662     Document Type: Article

References (44)

1. M. Salman Asif. 2008. Primal dual pursuit-A homotopy based algorithm for the dantzig selector. Master's thesis, Georgia Institute of Technology.
2. Stephen Becker, Jérome Bobin, and Emmanuel J. Candes. 2011. NESTA: A fast and accurate first-order method for sparse recovery. SIAM J. Imag Sci. 4, 1, 1-39.
3. Ernesto G. Birgin, Jo͆e M. Martínez, and Marcos Raydán. 2000. Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optimiz. 10, 4, 1196-1211.
4. Stephen Boyd and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press, New York, NY.
5. Jian-Feng Cai, Stanley Osher, and Zuowei Shen. 2009. Linearized Bregman iterations for compressed sensing. Math. Comp. 78, 267, 1515-1536.
6. PaoloM. Camerini, Luigi Fratta, and FrancescoMaffioli. 1975. On improving relaxationmethods by modified gradient techniques. Math. Prog. Study 3, 26-34.
7. Emmanuel J. Candes, Justin Romberg, and Terence Tao. 2006. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 2, 489-509.
8. Giacomo D'Antonio and Antonio Frangioni. 2009. Convergence analysis of deflected conditional approximate subgradient methods. SIAM J. Optimiz 20, 1, 357-386.
9. Geoffrey M. Davies, Stéphane G. Mallat, and Zhifeng Zhang. 1994. Adaptive time-frequency decompositions. Opt. Eng. 33, 7, 2183-2191.
10. David L. Donoho. 2006. Compressed Sensing. IEEE Trans. Inf. Theory 52, 4, 1289-1306.
11. David L. Donoho and Xiaoming Huo. 2001. Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47, 7, 2845-2862.
12. David L. Donoho and Yaakov Tsaig. 2008. Fast solution ofℓ 1-norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54, 11, 4789-4812.
13. Junbo Duan, Charles Soussen, David Brie, Jérome Idier, and Yu-Ping Wang. 2011. A sufficient condition on monotonic increase of the number of nonzero entry in the optimizer of L1 norm penalized least-square problem. arXiv:1104.3792 [stat.ML].
14. Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani. 2004. Least angle regression. Ann. Stat. 32, 2, 407-499.
15. Mário A. T. Figueiredo, Robert D. Nowak, and Stephen J. Wright. 2007. Gradient projection for sparse reconstruction: Applications to compressed sensing and other inverse problems. IEEE J. Select Topics Sig. Proc. 4, 1, 586-597.
16. Jean-Jacques Fuchs. 2004. On sparse representations in arbitrary redundant bases. IEEE Trans. Inf. Theory 50, 6, 1341-1344.
17. Markus Grasmair, Markus Haltmeier, and Otmar Scherzer. 2011. Necessary and sufficient conditions for linear convergence ofℓ 1-regularization. Commun. Pure Appl. Math. 64, 2, 161-182.
18. Michael Held, Philip Wolfe, and Harlan P. Crowder. 1974. Validation of subgradient optimization. Math. Prog. 6, 62-88.
19. Victor Klee and George J. Minty. 1972. How good is the simplex algorithm? In Inequalities, (III Proceedings of the 3rd Symposium Dedicated to the Memory of Theodore S. Motzkin). Academic Press, New York, NY, 159-175.
20. Torbjörn Larsson, Michael Patriksson, and Ann-Brith Strömberg. 1996. Conditional subgradient optimization-Theory and applications. Europ. J. Oper. Res. 88, 2, 382-403.
21. Churlzu Lim and Hanif D. Sherali. 2005. Convergence and computational analyses for some variable target value and subgradient deflection methods. Computat. Opt. Appl. 34, 3, 409-428.
22. Andreas Löbel. 1997. Optimal vehicle scheduling in public transit. Ph.D. Dissertation, Technische Universit ät Berlin.
23. Dirk A. Lorenz. 2013. Constructing test instances for basis pursuit denoising. IEEE Trans. Sign. Process. 61, 5, 2010-2014.
24. D. A. Lorenz,M. E. Pfetsch, and A. M. Tillmann. 2014. An infeasible-point subgradientmethod using adaptive approximate projections. Computat. Optimiz. Appl. 57, 2, 271-306.
25. Dirk A. Lorenz, Stefan Schiffler, and Dennis Trede. 2011. Beyond convergence rates: Exact inversion with Tikhonov regularization with sparsity constraints. Inve. Prob. 27, 085009.
26. J. Mairal and B. Yu. 2012. Complexity analysis of the Lasso regularization path. In Proceedings of the 29th International Conference on Machine Learning (ICML). J. Langford and J. Pineau (Eds.), Omnipress, Madison, WI, 353-360.
27. Dmitry M. Malioutov, Mujdat Cetin, and Alan S. Willsky. 2005. Homotopy continuation for sparse signal representation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'05). Vol. 5, IEEE, New York, NY, 733-736.
28. Arkadi S. Nemirovskiy and Boris T. Polyak. 1984. Iterative methods for solving linear ill-posed problems under precise information. I. Izvestiya Akademii Nauk SSSR. Tekhnicheskaya Kibernetika 2, 13-25, 203.
29. Yurii Nesterov. 2005. Smooth minimization of non-smooth functions. Math. Prog. 103, 1, 127-152.
30. Michael R. Osbourne, Brett Presnell, and Berwin A. Turlach. 2000. A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20, 3, 389-402.
31. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad. 1993. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of the 27th Annual Asilomar Conference on Signals, Systems and Computers. Vol. 1, CA, 40-44.
32. Andrzej Ruszczynski 2006. Nonlinear Optimization. Princeton University Press, Princeton, NJ.
33. Naum Z. Shor. 1985. Minimization Methods for Non-Differentiable Functions. Springer, New York, NY.
34. Joel A. Tropp. 2004. Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inf. Theory. 50, 10, 2231-2242.
35. Joel A. Tropp. 2006. Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theory 52, 3 (March 2006), 1030-1051.
36. Joel A. Tropp and Anna C. Gilbert. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53, 12, 4655-5666.
37. Ewout van den Berg and Michael P. Friedlander. 2008. Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31, 2, 890-912.
38. YilunWang andWotao Yin. 2010. Sparse signal reconstruction via iterative support detection. SIAM J. Imag. Sci. 3, 3, 462-491.
39. Zaiwen Wen, Wotao Yin, Donald Goldfarb, and Yin Zhang. 2010. A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation. SIAM J. Sci. Comput. 32, 4, 1832-1857.
40. Roland Wunderling. 1996. Paralleler und objektorientierter Simplex-Algorithmus. Ph.D. Dissertation, Technische Universität Berlin. In German.
41. Allen Y. Yang, Zihan Zhou, Arvind Ganesh, S. Shankar Sastry, and Yi Ma. 2010. A review of fastℓ 1-minimization algorithms for robust face recognition. arXiv:1007.3753 [cs.CV].
42. Junfeng Yang and Yin Zhang. 2011. Alternating direction algorithms forℓ 1-problems in compressive sensing. SIAM J. Sci. Comput. 33, 1, 250-278.
43. Wotao Yin. 2010. Analysis and generalizations of the linearized Bregman model. SIAM J. Imag. Sci. 3, 4, 856-877.
44. Wotao Yin, Stanley J. Osher, Donald Goldfarb, and Jérome Darbon. 2008. Bregman iterative algorithms forℓ 1-minimization with applications to compressed sensing. SIAM J. Imag Sci. 1, 1, 143-168.