Forward-backward greedy algorithms for general convex smooth functions over a cardinality constraint

31st International Conference on Machine Learning, ICML 2014

Volumn 1, Issue , 2014, Pages 765-773

  Liu, Ji ^a   Fujimaki, Ryohei ^b   Ye, Jieping ^c  

^a University of Wisconsin   (United States)

^b NEC LABORATORIES AMERICA   (United States)

^c Arizona State University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONSTRAINT THEORY; FEATURE EXTRACTION; FUNCTIONS; ITERATIVE METHODS; LEARNING SYSTEMS; LEAST SQUARES APPROXIMATIONS; OPTIMIZATION;

CARDINALITY CONSTRAINTS; CONDITIONAL RANDOM FIELD; FORWARD-BACKWARD GREEDY ALGORITHMS; GRADIENT INFORMATIONS; INDEPENDENT SAMPLES; OPTIMIZATION PROBLEMS; SENSOR SELECTION PROBLEM; THEORETICAL PERFORMANCE;

ALGORITHMS;

EID: 84919783289     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Bahmani, S., Boufounos, P., and Raj, B. Greedy sparsityconstrained optimization. ASILOMAR, pp. 1148-1152, 2011.
2. Buhlmann, P. Boosting for high-dimensional linear mod els. Annals of Statistics, 34:559-583, 2006.
3. Candes, E. J. and Tao, T. Decoding by linear program ming. IEEE Transactions on Information Theory, 51 (12):4203-4215, 2005.
4. Candes, E. J. and Tao, T. The Dantzig selector: statisti cal estimation when p is much larger than n. Annals of Statistics, 35(6):2313-2351, 2007.
5. Chickering, D. M. and Boutilier, C. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3:507-554, 2002.
6. Jalali, A., Johnson, C. C, and Ravikumar, P. D. On learning discrete graphical models using greedy methods. NIPS, 2011.
7. Johnson, C. C, Jalali, A., and Ravikumar, P. D. Highdimensional sparse inverse covariance estimation using greedy methods. Journal of Machine Learning Research-Proceedings Track, 22:574-582, 2012.
8. Kleinbaum, D. G. and Klein, M. Logistic regression. Statistics for Biology and Health, pp. 103-127, 2010.
9. Lafferty, J. D., McCallum, A., and Pereira, F. C. N. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML, pp. 282-289, 2001.
10. Liu, J., Wonka, P., and Ye, J. A multi-stage framework for dantzig selector and LASSO. Journal of Machine Learning Research, 13:1189-1219, 2012.
11. Liu, J., Fujimaki, R., and Ye, J. Forward-backward greedy algorithms for general convex smooth functions over a cardinality constraint. arXiv:1401.0086, 2013.
12. Needell, D. and Tropp, J. A. CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Applied and Computational Harmonic Analysis, 26:301-321, 2008.
13. Needell, D. and Vershynin, R. Uniform uncertainty prin ciple and signal recovery via regularized orthogonal matching pursuit. Foundations of Computational Mathematics, 9(3):317-334, 2009.
14. Negahban, S., Ravikumar, P. D., Wainwright, M. J., and Yu, B. A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. CoRR, abs/1010. 2731, 2010.
15. Ravikumar, R, Wainwright, M. I, Raskutti, G., and Yu, B. High-dimensional covariance estimation by minimizing i\-penalized log-determinant divergence. Electronic Journal of Statistics, 5:935-980, 2011.
16. Shalev-Shwartz, S., Srebro, N., and Zhang, T. Trading ac curacy for sparsity in optimization problems with sparsity constraints. SIAM Journal on Optimization, 20(6): 2807-2832, 2010.
17. Sutton, C. and McCallum, A. An introduction to conditional random fields for relational learning. Introduction to Statistical Relational Learning, pp. 93-128, 2006.
18. Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1994.
19. Tropp, J. A. Greed is good: algorithmic results for sparse approximation. IEEE Transactions on Information Theory, 50(10):2231-2242, 2004.
20. Vershynin, R. Introduction to the non-asymptotic analysis of random matrices. arXiv:1011.3027, 2011.
21. Zhang, C.-H. and Zhang, T. A general theory of con cave regularization for high dimensional sparse estima tion problems. Statistical Science, 27(4), 2012.
22. Zhang, T. On the consistency of feature selection using greedy least squares regression. Journal of Machine Learning Research, 10:555-568, 2009.
23. Zhang, T. Adaptive forward-backward greedy algorithm for learning sparse representations. IEEE Transactions on Information Theory, 57(7):4689-4708, 2011a.
24. Zhang, T. Sparse recovery with orthogonal matching pur suit under RIP. IEEE Transactions on Information Theory, 57(9): 6215-6221, 2011b.
25. Zhao, P. and Yu, B. On model selection consistency of lasso. Journal of Machine Learning Research, 7:2541-2563, 2006.