Efficient nonconvex sparse group feature selection via continuous and discrete optimization

Artificial Intelligence

Volumn 224, Issue , 2015, Pages 28-50

  Xiang, Shuo ^a,b   Shen, Xiaotong ^c   Ye, Jieping ^a,b  

^a ARIZONA STATE UNIVERSITY   (United States)

^b Arizona State University   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Application; Discrete optimization; EEG data analysis; Error bound; Nonconvex optimization

Indexed keywords

APPLICATIONS; CLUSTERING ALGORITHMS; NUMERICAL METHODS; OPTIMIZATION; PARAMETER ESTIMATION;

CONTINUOUS AND DISCRETE OPTIMIZATION; DISCRETE FORMULATIONS; DISCRETE OPTIMIZATION; EEG DATUM; ERROR BOUND; HIGH DIMENSIONAL DATA; NONCONVEX OPTIMIZATION; OPTIMIZATION ALGORITHMS;

FEATURE EXTRACTION;

EID: 84925439254     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2015.02.008     Document Type: Article

References (52)

1. Francis Bach, Rodolphe Jenatton, Julien Mairal, and Guillaume Obozinski Convex Optimization with Sparsity-Inducing Norms 2010
2. Jonathan Barzilai, and Jonathan M. Borwein Two-point step size gradient methods IMA J. Numer. Anal. 8 1 1988 141 148
3. A. Beck, and M. Teboulle A fast iterative shrinkage-thresholding algorithm for linear inverse problems SIAM J. Imaging Sci. 2 1 2009 183 202
4. P.J. Bickel, Y. Ritov, and A.B. Tsybakov Simultaneous analysis of Lasso and Dantzig selector Ann. Stat. 37 4 2009 1705 1732
5. Thomas Blumensath, and Mike E. Davies Iterative hard thresholding for compressed sensing Appl. Comput. Harmon. Anal. 27 3 2009 265 274
6. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers 2011 Now Publishers
7. S. Boyd, and L. Vandenberghe Convex Optimization 2004 Cambridge University Press
8. P. Breheny, and J. Huang Penalized methods for bi-level variable selection Stat. Interface 2 3 2009 369 380
9. Peter Brucker An O (n) algorithm for quadratic knapsack problems Oper. Res. Lett. 3 3 1984 163 166
10. Emmanuel J. Candes, and Terence Tao Decoding by linear programming IEEE Trans. Inf. Theory 51 12 2005 4203 4215
11. P.L. Combettes, and J.C. Pesquet Proximal splitting methods in signal processing Fixed-Point Algorithms for Inverse Problems in Science and Engineering 2010
12. D.L. Donoho De-noising by soft-thresholding IEEE Trans. Inf. Theory 41 3 2002 613 627
13. 1-ball for learning in high dimensions Proceedings of the 25th International Conference on Machine Learning 2008 ACM 272 279
14. J. Fan, and R. Li Variable selection via nonconcave penalized likelihood and its oracle properties J. Am. Stat. Assoc. 96 456 2001 1348 1360
15. Simon Foucart Sparse recovery algorithms: sufficient conditions in terms of restricted isometry constants Approximation Theory XIII: San Antonio 2010 2012 Springer 65 77
16. A. Frank, and A. Asuncion UCI Machine Learning Repository 2010
17. J. Friedman, T. Hastie, and R. Tibshirani A note on the group lasso and a sparse group lasso arXiv preprint arXiv:1001.0736 2010
18. Pinghua Gong, Changshui Zhang, Zhaosong Lu, Jianhua Huang, and Jieping Ye A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems The 30th International Conference on Machine Learning ICML 2013 37 45
19. M. Grant, S. Boyd, CVX: Matlab software for disciplined convex programming, version 1.21, http://cvxr.com/cvx, April 2011.
20. J. Huang, S. Ma, H. Xie, and C.H. Zhang A group bridge approach for variable selection Biometrika 96 2 2009 339 355
21. J. Huang, and T. Zhang The benefit of group sparsity Ann. Stat. 38 4 2010 1978 2004
22. Jian Huang, Patrick Breheny, and Shuangge Ma A selective review of group selection in high-dimensional models Stat. Sci. 27 4 2012
23. Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, and Francis Bach Proximal methods for hierarchical sparse coding J. Mach. Learn. Res. 12 2011 2297 2334
24. A.N. Kolmogorov, and V.M. Tikhomirov e-Entropy and e-capacity of sets in functional spaces Transl. Am. Math. Soc. 1961
25. Andrei Nikolaevich Kolmogorov, and Vladimir Mikhailovich Tikhomirov ε-Entropy and ε-capacity of sets in function spaces Usp. Mat. Nauk 14 2 1959 3 86
26. Charles E. Leiserson, Ronald L. Rivest, Clifford Stein, and Thomas H. Cormen Introduction to Algorithms 2001 The MIT Press
27. J. Liu, S. Ji, and J. Ye SLEP: Sparse Learning with Efficient Projections 2009 Arizona State University
28. 0 regularized convex cone programming arXiv preprint arXiv:1211.0056 2013
29. R. Mazumder, J.H. Friedman, and T. Hastie Sparsenet: coordinate descent with nonconvex penalties J. Am. Stat. Assoc. 106 495 2011 1125 1138
30. L. Meier, S. Van De Geer, and P. Bühlmann The group lasso for logistic regression J. R. Stat. Soc., Ser. B, Stat. Methodol. 70 1 2008 53 71
31. B.K. Natarajan Sparse approximation solutions to linear systems SIAM J. Comput. 24 2 1995 227 234
32. Y. Nesterov Gradient methods for minimizing composite objective function CORE discussion papers 2007
33. Jorge Nocedal, and Stephen J. Wright Numerical Optimization 2000 Springer
34. M. Schmidt, N. Le Roux, and F. Bach et al. Convergence rates of inexact proximal-gradient methods for convex optimization NIPS'11 - 25th Annual Conference on Neural Information Processing Systems 2011
35. X. Shen, W. Pan, and Y. Zhu Likelihood-based selection and sharp parameter estimation J. Am. Stat. Assoc. 107 2012 223 232
36. X. Shen, W. Pan, Y. Zhu, and H. Zhou On constrained and regularized high-dimensional regression Ann. Inst. Stat. Math. 1 2013 1 26
37. H. Su, W. Yu, and F.F. Li Efficient Euclidean projections onto the intersection of norm balls Proceedings of the 29th International Conference on Machine Learning vol. 951 2012 International Machine Learning Society 12
38. P.D. Tao, and L.T.H. An Convex analysis approach to dc programming: theory, algorithms and applications Acta Math. Vietnam. 22 1 1997 289 355
39. R. Tibshirani Regression shrinkage and selection via the lasso J. R. Stat. Soc., Ser. B, Stat. Methodol. 1996 267 288
40. Berwin A. Turlach, William N. Venables, and Stephen J. Wright Simultaneous variable selection Technometrics 47 3 2005 349 363
41. S.A. Van De Geer, and P. Bühlmann On the conditions used to prove oracle results for the lasso Electron. J. Stat. 3 2009 1360 1392
42. L. Wang, G. Chen, and H. Li Group SCAD regression analysis for microarray time course gene expression data Bioinformatics 23 12 2007 1486 1494
43. W.H. Wong, and X. Shen Probability inequalities for likelihood ratios and convergence rates of sieve MLEs Ann. Stat. 23 2 1995 339 362
44. Stephen J. Wright, Robert D. Nowak, and Mário A.T. Figueiredo Sparse reconstruction by separable approximation IEEE Trans. Signal Process. 57 7 2009 2479 2493
45. S. Xiang, X. Shen, and J. Ye Efficient sparse group feature selection via nonconvex optimization The 30th International Conference on Machine Learning ICML 2013 284 292
46. Shuo Xiang, Lei Yuan, Wei Fan, Yalin Wang, Paul M. Thompson, and Jieping Ye Bi-level multi-source learning for heterogeneous block-wise missing data NeuroImage 2013
47. Shuo Xiang, Lei Yuan, Wei Fan, Yalin Wang, Paul M. Thompson, and Jieping Ye Multi-source learning with block-wise missing data for Alzheimer's disease prediction Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2013 ACM 185 193
48. H. Yang, Z. Xu, I. King, and M. Lyu Online learning for group lasso Proceedings of the 27th International Conference on Machine Learning 2010 ACM 1191 1198
49. M. Yuan, and Y. Lin Model selection and estimation in regression with grouped variables J. R. Stat. Soc., Ser. B, Stat. Methodol. 68 1 2006 49 67
50. T. Zhang Analysis of multi-stage convex relaxation for sparse regularization J. Mach. Learn. Res. 11 2010 1081 1107
51. T. Zhang Multi-stage convex relaxation for feature selection Bernoulli 19 5B 2013 2277 2293
52. P. Zhao, and B. Yu On model selection consistency of lasso J. Mach. Learn. Res. 7 2006 2541 2563