Gradient hard thresholding pursuit for sparsity-constrained optimization

31st International Conference on Machine Learning, ICML 2014

Volumn 2, Issue , 2014, Pages 1322-1330

  Yuan, Xiao Tong ^a,b   Li, Ping ^b   Zhang, Tong ^b  

^a Department of Statistical Science Cornell University ^*  (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPRESSED SENSING; CONVEX OPTIMIZATION; ITERATIVE METHODS; LEARNING SYSTEMS; LINEAR SYSTEMS; NUMERICAL METHODS; OPTIMIZATION;

CONSTRAINED CONVEX OPTIMIZATIONS; GREEDY SELECTIONS; LOGISTIC REGRESSIONS; NUMERICAL EVIDENCE; NUMERICAL PERFORMANCE; RATE OF CONVERGENCE; THEORETICAL GUARANTEES; UNDERDETERMINED LINEAR SYSTEMS;

CONSTRAINED OPTIMIZATION;

EID: 84919934927     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Bahmani, S., Raj, B., and Boufounos, P. Greedy sparsity- constrained optimization. Journal of Machine Learning Research, 14:807-841, 2013.
2. Beck, A. and Teboulle, Marc. A fast iterative shrinkage- thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1): 183-202, 2009.
3. Bishop, C.M. Pattern Recognition and Machine Learning. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006. ISBN 978-0-387-31073-2.
4. Blumensath, T. Compressed sensing with nonlinear observations and related nonlinear optimization problems. IEEE Transactions on Information Theory, 59(6):3466- 3474, 2013.
5. Blumensath, T. and Davies, M. E. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3):265-274, 2009.
6. Candes, E. J., Romberg, J. K., and Tao, T. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8): 1207-1223, 2006.
7. Chappelle, O. Training a support vector machine in the primal. Neural Computation, 19(5): 1155-1178, 2007.
8. Dai, W. and Milenkovic, O. Subspace pursuit for compressive sensing signal reconstruction. IEEE Transactions on Information Theory, 55(5):2230-2249, 2009.
9. Donoho, D. L. Compressed sensing. IEEE Transactions on Information Theory, 52(4): 1289-1306, 2006.
10. Foucart, S. Hard thresholding pursuit: An algorithm for compressive sensing. SIAM Journal on Numerical Analysis, 49(6):2543-2563, 2011.
11. Foucart, S. Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants. In Approximation Theory XIII: San Antonio 2010, volume 13 of Springer Proceedings in Mathematics, pp. 65-77, 2012.
12. Frank, M. and Wolfe, P. An algorithm for quadratic programming. Naval Res. Logist. Quart., 5:95-110, 1956.
13. Jalali, A., Johnson, C. C and Ravikumar, P. K. On learning discrete graphical models using greedy methods. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS'11), 2011.
14. Kim, Yongdai and Kim, Jinseog. Gradient lasso for feature selection. In Proceedings Of The Twenty-First International Conference On Machine Learning (ICML'04), 2004.
15. Langford, J., Li, L., and Zhang, T. Sparse online learning via truncated gradient. Journal of Machine Learning Research, 10:777-801, 2009.
16. Ma, Z. Sparse principal component analysis and iterative thresholding. Annals of Statistics, 41(2):772-801, 2013.
17. Mallat, S. and Zhang, Zhifeng. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12):3397-3415, 1993.
18. Natarajan, B. K. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24(2):227-234, 1995.
19. Needell, D. and Tropp, J. A. Cosamp: iterative signal recovery from incomplete and inaccurate samples. IEEE Transactions on Information Theory, 26(3):301- 321, 2009.
20. Pati, Y.C., Rezaiifar, R., and Krishnaprasad, PS. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of the 27th Annual Asilomar Conference on Signals, Systems, and Computers, pp. 40-44, 1993.
21. 1-penalized log-determinant divergence. Electronic Journal of Statistics, 5:935-980, 2011.
22. Shalev-Shwartz, Shai, Srebro, Nathan, and Zhang, Tong. Trading accuracy for sparsity in optimization problems with sparsity constraints. SIAM Journal on Optimization, 20:2807-2832, 2010.
23. Tropp, J. and Gilbert, A. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12):4655-4666.
24. Wainwright, M.J. and Jordan, M.I. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1-2):1-305.
25. Yuan, X.-T. and Yan, S. Forward basis selection for pursuing sparse representations over a dictionary. IEEE Transactions on Pattern Analysis And Machine Intelligence, 35(12):3025-3036, 2013.
26. Yuan, X.-T. and Zhang, T. Truncated power method for sparse eigenvalue problems. Journal of Machine Learning Research, 14:899-925, 2013.
27. Zhang, T. Adative forward-backward greedy algorithm for sparse learning with linear models. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS'08), 2008.