Sparse approximation via penalty decomposition methods

SIAM Journal on Optimization

Volumn 23, Issue 4, 2013, Pages 2448-2478

  Lu, Zhaosong ^a   Zhang, Yong ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

Block coordinate descent method; Compressed sensing; L0 minimization; Penalty decomposition methods; Sparse inverse covariance selection; Sparse logistic regression

Indexed keywords

BLOCK COORDINATE DESCENTS; COMPUTATIONAL RESULTS; DECOMPOSITION METHODS; FIRST-ORDER OPTIMALITY CONDITION; INVERSE COVARIANCE; LOGISTIC REGRESSIONS; MINIMIZATION PROBLEMS; SPARSE APPROXIMATIONS;

COMPRESSED SENSING; REGRESSION ANALYSIS; SIGNAL RECONSTRUCTION;

INVERSE PROBLEMS;

EID: 84892875448     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/100808071     Document Type: Article

References (59)

1. O. Banerjee, L. E. Ghaoui, and A. D'Aspremont, Model selection through sparse maximum likelihood estimation, J. Mach. Learn. Res., 9 (2008), pp. 485-516.
2. J. A. Bilmes, Factored sparse inverse covariance matrices, in International Conference on Acoustics, Speech and Signal Processing, Washington, D.C., 2000, pp. 1009-1012.
3. C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2007.
4. E. G. Birgin, J. M. Martínez, and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim., 10 (2000), pp. 1196-1211.
5. T. Blumensath and M. E. Davies, Iterative thresholding for sparse approximations, J. Fourier Anal. Appl., 14 (2008), pp. 629-654.
6. T. Blumensath and M. E. Davies, Iterative hard thresholding for compressed sensing, Appl. Comput. Harmon. Anal., 27 (2009), pp. 265-274.
7. E. J. Candés and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
8. E. J. Candés, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
9. R. Chartrand, Exact reconstruction of sparse signals via nonconvex minimization, IEEE Signal Process. Lett., 14 (2007), pp. 707-710.
10. S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33-61.
11. p minimization, SIAM J. Sci. Comput., 32 (2010), pp. 2832-2852.
12. p Minimization, Tech. report, Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, 2010.
13. J. Claerbout and F. Muir, Robust modelling of erratic data, Geophys., 38 (1973), pp. 826-844.
14. A. D'Aspremont, O. Banerjee, and L. El Ghaoui, First-order methods for sparse covariance selection, SIAM J. Matrix Anal. Appl., 30 (2008), pp. 56-66.
15. J. Dahl, L. Vandenberghe, and V. Roychowdhury, Covariance selection for nonchordal graphs via chordal embedding, Optim. Methods Softw., 23 (2008), pp. 501-520.
16. A. Dempster, Covariance selection, Biometrics, 28 (1978), pp. 157-175.
17. A. Dobra, Dependency networks for genome-wide data, Biostatist., 8 (2007), pp. 1-28.
18. A. Dobra, C. Hans, B. Jones, J. R. Nevins, G. Yao, and M. West, Sparse graphical models for exploring gene expression data, J. Multivariate Anal., 90 (2004), pp. 196-212.
19. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Statist., 32 (2004), pp. 407-499.
20. M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems, IEEE J. Select. Topics Signal Process., 1 (2007), pp. 586-598.
21. J. Friedman, T. Hastie, and R. Tibshirani, Sparse inverse covariance estimation with the graphical lasso, Biostatist., 9 (2008), pp. 432-441.
22. A. D. Gerson, L. C. Parra, and P. Sajda, Cortical origins of response time variability during rapid discrimination of visual objects, Neuroimage, 28 (2005), pp. 342-353.
23. G. Golub and C. Van Loan, Matrix Computations, 3rd ed., Stud. Appl. Math. 13, Johns Hopkins University Press, Baltimore, MD, 1996.
24. T. R. Golub, D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, M. Loh, J. R. Downing, M. A. Caligiuri, C. D. Bloomfield, and E. S. Lander, Molecular classification of cancer: Class discovery and class prediction by expression monitoring, Science, 286 (1999), pp. 531-537.
25. E. T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation applied to compressed sensing: Implementation and numerical experiments, J. Comput. Math., 28 (2010), pp. 170-194.
26. K. K. Herrity, A. C. Gilbert, and J. A. Tropp, Sparse approximation via iterative thresholding, in IEEE International Conference on Acoustics, Speech and Signal Processing, 2006, pp. 624-627.
27. C. J. Hsien, M. A. Sustik, I. S. Dhillon, and P. Ravikumar, Sparse Inverse Covariance Matrix Estimation Using Quadratic Approximation, in Advances in Neural Information Processing Systems, 2011.
28. 1-regularized least squares, IEEE J. Select. Topics Signal Process., 1 (2007), pp. 606-617.
29. 1-regularized logistic regression, J. Mach. Learn. Res., 8 (2007), pp. 1519-1555.
30. Linear Algebra PACKage, http://www.netlib.org/lapack/index.html.
31. 1-regularized logistic regression, in Proceedings of the 21th National Conference on Artificial Intelligence (AAAI), 2006.
32. S. Levy and P. Fullagar, Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution, Geophys., 46 (1981), pp. 1235-1243.
33. J. G. Liao and K. V. Chin, Logistic regression for disease classification using microarray data: Model selection in a large p and small n case, Bioinformatics, 23 (15), 2007, pp. 1945-1951.
34. J. Liu, S. Ji, and J. Ye, SLEP: Sparse Learning with Efficient Projections, Arizona State University, 2009; available from http://www.public. asu.edu/~jye02/Software/SLEP.
35. 1-regularized sparse covariance selection, Math. Program. Comput., 2 (2010), pp. 291-315.
36. Z. Lu, Smooth optimization approach for sparse covariance selection, SIAM J. Optim., 19 (2009), pp. 1807-1827.
37. Z. Lu, Adaptive first-order methods for general sparse inverse covariance selection, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2000-2016.
38. S. Mallat and Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Trans. Image Process., 41 (1993), pp. 3397-3415.
39. A. Miller, Subset Selection in Regression, Chapman and Hall, London, 2002.
40. D. Newman, S. Hettich, C. Blake, and C. Merz, UCI Repository of Machine Learning Databases, www.ics.uci.edu/~mlearn/MLRepository.html, 1998.
41. 2 regularization, and rotational invariance, in Proceedings of the 21st International Conference on Machine learning (ICML), 2004, pp. 72-85.
42. M. Y. Park and T. Hastie, Regularization Path Algorithms for Detecting Gene Interactions, Department of Statistics, Stanford University, 2006.
43. L. C. Parra, C. D. Spence, A. D. Gerson, and P. Sajda, Recipes for the linear analysis of EEG, Neuroimage, 28 (2005), pp. 326-341.
44. M. G. Philiastides and P. Sajda, Temporal characterization of the neural correlates of perceptual decision making in the human brain, Cereb. Cortex, 16 (2006), pp. 509-518.
45. J. Pittman, E. Huang, H. Dressman, C. F. Horng, S. H. Cheng, M. H. Tsou, C. M. Chen, A. Bild, E. S. Iversen, A. T. Huang, J. R. Nevins, and M. West, Integrated modeling of clinical and gene expression information for personalized prediction of disease outcomes, Proc. Natl. Acad. Sci. USA, 101 (2004), pp. 8431-8436.
46. B. Recht, M. Fazel, and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., 52 (2010), pp. 471-501.
47. A. Ruszczyński, Nonlinear Optimization, Princeton University Press, Princeton, NJ, 2006.
48. F. Santosa and W. W. Symes, Linear inversion of band-limited reflection seismograms, SIAM J. Sci. Statist. Comput., 7 (1986), pp. 1307-1330.
49. 1-regularized logistic regression, J. Mach. Learn. Res., 11 (2010), pp. 713-741.
50. 1-norm, Geophys., 44 (1979), pp. 39-52.
51. R. Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B, 58 (1996), pp. 267-288.
52. J. A. Tropp, Greed is good: Algorithmic results for sparse approximation, IEEE Trans. Inform. Theory, 50 (2004), pp. 2231-2242.
53. J. Tropp, Just relax: Convex programming methods for identifying sparse signals, IEEE Trans. Inform. Theory, 51 (2006), pp. 1030-1051.
54. Y. Tsuruoka, J. McNaught, J. Tsujii, and S. Ananiadou, Learning string similarity measures for gene/protein name dictionary look-up using logistic regression, Bioinformatics, 23 (2007), pp. 2768-2774.
55. E. Van Den Berg and M. P. Friedlander, Probing the Pareto frontier for basis pursuit solutions, SIAM J. Sci. Comput., 31 (2008), pp. 890-912.
56. E. Van Den Berg, M. P. Friedlander, G. Hennenfent, F. Herrmann, R. Saab, and O. Yilmaz, Algorithm 890: Sparco: A testing framework for sparse reconstruction, ACM Trans. Math. Software, 35 (2009), pp. 1-16.
57. C. Wang, D. Sun and K.-C. Toh, Solving log-determinant optimization problems by a Newton-CG primal proximal point algorithm, SIAM J. Optim., 20 (2010), pp. 2994-3013.
58. K. Y. Yeung, R. E. Bumgarner, and A. E. Raftery, Bayesian model averaging: Development of an improved multi-class, gene selection and classification tool for microarray data, Bioinformatics, 21 (2005), pp. 2394-2402.
59. 0 minimization for wavelet frame based image restoration, Math. Comp., 82 (2013), pp. 995-1015.