Greedy sparsity-constrained optimization

Journal of Machine Learning Research

Volumn 14, Issue 1, 2013, Pages 807-841

  Bahmani, Sohail ^a   Raj, Bhiksha ^b   Boufounos, Petros T ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Carnegie Mellon University   (United States)

^c MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

Author keywords

Compressed sensing; Greedy algorithm; Optimization; Sparsity

Indexed keywords

GREEDY ALGORITHMS; LOGISTIC REGRESSIONS; QUADRATIC COST FUNCTIONS; SIGNAL PROCESSING PROBLEMS; SPARSE ESTIMATION; SPARSE LINEAR REGRESSIONS; SPARSITY; SYNTHETIC AND REAL DATA;

COMPRESSED SENSING; CONSTRAINED OPTIMIZATION; COST FUNCTIONS; LOGISTICS; OPTIMIZATION; SIGNAL RECONSTRUCTION;

ALGORITHMS;

EID: 84876255064     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (39)

1. A. Agarwal, S. Negahban, and M. Wainwright. Fast global convergence rates of gradient methods for high-dimensional statistical recovery. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 37-45. 2010. long version available at arXiv:1104.4824v1 [stat.ML].
2. S. Bahmani, P. Boufounos, and B. Raj. Greedy sparsity-constrained optimization. In Conference Record of the Forty-Fifth Asilomar Conference on Signals, Systems, and Computers, pages 1148-1152, 2011.
3. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.
4. P. Bickel, Y. Ritov, and A. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. The Annals of Statistics, 37(4):1705-1732, 2009.
5. T. Blumensath. Compressed sensing with nonlinear observations. Preprint, 2010. URL http://users.fmrib.ox.ac.uk/~tblumens/papers/B Nonlinear.pdf.
6. T. Blumensath and M. E. Davies. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3):265-274, Nov. 2009.
7. F. Bunea. Honest variable selection in linear and logistic regression models via l1 and l1 +l2 penalization. Electronic Journal of Statistics, 2:1153-1194, 2008.
8. E. J. Candès. The restricted isometry property and its implications for compressed sensing. Comptes Rendus Mathematique, 346(9-10):589-592, 2008.
9. E. J. Candès, J. K. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8):1207-1223, 2006.
10. V. Chandrasekaran, B. Recht, P. A. Parrilo, and A. S.Willsky. The convex geometry of linear inverse problems. Foundations of Computational Mathematics, 12(6):805-849, Dec. 2012.
11. A. Cohen,W. Dahmen, and R. DeVore. Compressed sensing and best k-term approximation. American Mathematical Society, 22(1):211-231, 2009.
12. W. Dai and O. Milenkovic. Subspace pursuit for compressive sensing signal reconstruction. IEEE Transactions on Information Theory, 55(5):2230-2249, 2009.
13. A. J. Dobson and A. Barnett. An Introduction to Generalized Linear Models. Chapman and Hall/CRC, 3rd edition, May 2008.
14. D. L. Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52(4):1289-1306, 2006.
15. D. L. Donoho and M. Elad. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of the National Academy of Sciences, 100(5):2197-2202, 2003.
16. D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition. IEEE Transactions on Information Theory, 47(7):2845-2862, 2001.
17. S. Foucart. Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants. In Approximation Theory XIII: San Antonio 2010, volume 13 of Springer Proceedings in Mathematics, pages 65-77. Springer New York, 2012.
18. J. H. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2 2010. ISSN 1548-7660.
19. Software available online at http://www-stat.stanford.edu/~tibs/glmnet- matlab/.
20. I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror. Result analysis of the NIPS 2003 feature selection challenge. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 545-552. MIT-Press, Cambridge, MA, 2005.
21. J. D. Hamilton. Time Series Analysis. Princeton University Press, Princeton, NJ, 1994.
22. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Verlag, 2009.
23. A. Jalali, C. C. Johnson, and P. K. Ravikumar. On learning discrete graphical models using greedy methods. In J. Shawe-Taylor, R. S. Zemel, P. Bartlett, F. C. N. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 1935-1943. 2011.
24. S. M. Kakade, O. Shamir, K. Sridharan, and A. Tewari. Learning exponential families in highdimensions: strong convexity and sparsity. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, volume 9 of JMLR W&CP, pages 381-388, 2010.
25. J. Liu, J. Chen, and J. Ye. Large-scale sparse logistic regression. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '09, pages 547-556, New York, NY, USA, 2009. ACM.
26. A. Lozano, G. Swirszcz, and N. Abe. Group orthogonal matching pursuit for logistic regression. In G. Gordon, D. Dunson, and M. Dudik, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of JMLR W&CP, pages 452-460, 2011.
27. B. K. Natarajan. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24 (2):227-234, 1995.
28. D. Needell and J. A. Tropp. CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Applied and Computational Harmonic Analysis, 26(3):301-321, 2009.
29. S. Negahban, P. Ravikumar, M. Wainwright, and B. Yu. A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1348-1356. 2009. long version available at arXiv:1010.2731v1 [math.ST].
30. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, Jan. 2012.
31. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad. Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In Conference Record of the Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, volume 1, pages 40-44, 1993.
32. S. Shalev-Shwartz, N. Srebro, and T. Zhang. Trading accuracy for sparsity in optimization problems with sparsity constraints. SIAM Journal on Optimization, 20(6):2807-2832, 2010.
33. A. Tewari, P. K. Ravikumar, and I. S. Dhillon. Greedy algorithms for structurally constrained high dimensional problems. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 882-890. 2011.
34. J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12(4):389-434, Aug. 2012.
35. J. A. Tropp and A. C. Gilbert. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12):4655-4666, 2007.
36. S. A. van de Geer. Empirical Processes in M-estimation. Cambridge University Press, Cambridge, UK, 2000.
37. S. A. van de Geer. High-dimensional generalized linear models and the Lasso. The Annals of Statistics, 36(2):614-645, 2008.
38. V. Vapnik. Statistical Learning Theory. Wiley, New York, NY, 1998.
39. T. Zhang. Sparse recovery with orthogonal matching pursuit under RIP. IEEE Transactions on Information Theory, 57(9):6215-6221, Sept. 2011.