Smoothing proximal gradient method for general structured sparse regression

Annals of Applied Statistics

Volumn 6, Issue 2, 2012, Pages 719-752

  Chen, Xi ^a   Lin, Qihang ^b   Kim, Seyoung ^a   Carbonell, Jaime G ^a   Xing, Eric P ^a  

^a Carnegie Mellon University ^*  (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Optimization; Proximal gradient; Smoothing; Sparse regression; Structured sparsity

Indexed keywords

EID: 84866248858     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/11-AOAS514     Document Type: Article

References (43)

1. ABATE, N., CHANDALIA, M., SATIJA, P. and ADAMS HUET, B., et al. (2005). Enpp1/pc-1 k121q polymorphism and genetic susceptibility to type 2 diabetes. Diabetes 54 1027-1213.
2. BECK, A. and TEBOULLE, M. (2009). A fast iterative shrinkage-thre sholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2 183-202. MR2486527
3. BERTSEKAS, D. (1999). Nonlinear Programming. Athena Scientific, Nashua, NH.
4. DUCHI, J. and SINGER, Y. (2009). Efficient online and batch learning using forward backward splitting. J. Mach. Learn. Res. 10 2899-2934. MR2579916
5. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2010). A note on the group lasso and a sparse group lasso. Dept. Statistics, Stanford Univ.
6. FRIEDMAN, J., HASTIE, T., HÖFLING, H. and TIBSHIRANI, R. (2007). Pathwise coordinate optimization. Ann. Appl. Stat. 1302-1332. MR2415737
7. HUANG, D. W., SHERMAN, B. T. and LEMPICKI, R. A. (2009). Systematic and integrative analysis of large gene lists using david bioinformatics resources. Nature Protoc. 4 44-57.
8. JACOB, L., OBOZINSKI, G. and VERT, J. P. (2009). Group lasso with overlap and graph lasso. In Proceedings of the International Conference on Machine Learning. ACM, Montreal, QC
9. JENATTON, R., AUDIBERT, J. Y. and BACH, F. (2009). Structured variable selection with sparsityinducing norms. Technical report, INRIA.
10. JENATTON, R., MAIRAL, J., OBOZINSKI, G. and BACH, F. (2010). Proximal methods for sparse hierarchical dictionary learning. In Proceedings of the International Conference on Machine Learning. Omnipress, Haifa.
11. KIM, S., SOHN, K. A. and XING, E. P. (2009). A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics 25 204-212.
12. KIM, S. and XING, E. P. (2009). Statistical estimation of correlated genome associations to a quantitative trait network. PLoS Genet. 5 e1000587.
13. KIM, S. and XING, E. P. (2010). Tree-guided group lasso for multi-task regression with structured sparsity. In Proceedings of the International Conference on Machine Learning. Omnipress, Haifa.
14. LAN, G., LU, Z and MONTEIRO, R. (2011). Primal-dual first-order methods with O(1/ε) iteration complexity for cone programming. Mathematical Programming 126 1-29.
15. LANGE, K. (2004). Optimization. Springer, Berlin.
16. LIU, J., JI, S. and YE, J. (2009). Multi-task feature learning via efficient l2,1-norm minimization. In Proceedings of the Uncertainty in AI. AUAI Press, Montreal, QC.
17. LIU, J. and YE, J. (2010a). Fast overlapping group lasso. Available at arXiv:1009.0306v1.
18. LIU, J. and YE, J. (2010b). Moreau-yosida regularization for grouped tree structure learning. In Advances in Neural Information Processing Systems (NIPS). Curran Associates, Inc., Vancouver, BC.
19. LIU, J., YUAN, L. and YE, J. (2010). An efficient algorithm for a class of fused lasso problems. In The ACM SIG Knowledge Discovery and Data Mining. ACM, Washington, DC.
20. MA, S. and KOSOROK, M. R. (2010). Detection of gene pathways with predictive power for breast cancer prognosis. BMC Bioinformatics 11 1.
21. MAIRAL, J., JENATTON, R., OBOZINSKI, G. and BACH, F. (2010). Network flow algorithms for structured sparsity. In Advances in Neural Information Processing Systems (NIPS). Curran Associates, Inc., Vancouver, BC.
22. NESTEROV, Y. (2003). Excessive gap technique in non-smooth convex minimization. Technical report, Univ. Catholique de Louvain, Center for Operations Research and Econometrics (CORE).
23. NESTEROV, Y. (2005). Smooth minimization of non-smooth functions. Mathematical Programming 103 127-152.
24. NESTEROV, Y. (2007). Gradient methods for minimizing composite objective function. ECORE Discussion Paper 2007.
25. OBOZINSKI, G., TASKAR, B. and JORDAN, M. I. (2009). High-dimensional union support recovery in multivariate regression. In Advances in Neural Information Processing Systems (NIPS). Curran Associates, Inc., Vancouver, BC.
26. QI, L. and SUN, J. (1993). A nonsmooth version of newton's method. Mathematical Programming 58 353-367.
27. ROCKAFELLAR, R. (1996). Convex Analysis. Princeton Univ. Press, Princeton.
28. SUBRAMANIAN, A., TAMAYO, P. and MOOTHA, V., et al. (2005). Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proc. Natl. Acad. Sci. USA 102 15545-15550.
29. SUN, D., WOMERSLEY, R. and QI, H. (2002). A feasible semis mooth asymptotically Newton method for mixed complementarity problems. Mathematical Programming, Ser. A 94 167-187.
30. THE INTERNATIONAL HAPMAP CONSORTIUM. (2005). A haplotype map of the human genome. Nature 437 1399-1320.
31. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58 267-288. MR1379242
32. TIBSHIRANI, R. and SAUNDERS, M. (2005). Sparsity and smoothness via the fused lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 67 91-108. MR2136641
33. TIBSHIRANI, R. and TAYLOR, J. (2010). The solution path of the generalized lasso. Ann. Statist. 39 1335-1371. MR2850205
34. TSENG, P. (2008). On accelerated proximal gradient methods for convex-concave optimization. SIAM J. Optim. To appear.
35. TÜTÜNCÜ, R. H., TOH, K. C. and TODD, M. J. (2003). Solving semidefinite-quadratic-linear programs using sdpt3. Mathematical Programming 95 189-217.
36. VAN DE VIJVER, M. J. ET AL. (2002). A gene-expression signature as a predictor of survival in breast cancer. New England Journal of Medicine 347 1999-2009.
37. WU, T. and LANGE, K. (2008). Coordinate descent algorithms for lasso penalized regression. Ann. Appl. Stat. 2 224-244. MR2415601
38. YUAN, M. and LIN, Y. (2006).Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 49-67. MR2212574
39. ZHANG, Z., LANGE, K., OPHOFF, R. and SABATTI, C. (2010). Reconstructing DNA copy number by penalized estimation and imputation. Ann. Appl. Stat. 4 1749-1773. MR2829935
40. ZHAO, P., ROCHA, G. and YU, B. (2009a). The composite absolute penalties family for grouped and hierarchical variable selection. Ann. Statist. 37 3468-3497. MR2549566
41. ZHAO, P., ROCHA, G. and YU, B. (2009b). Grouped and hierarchical model selection through composite absolute penalties. Ann. Statist. 37 3468-3497.
42. ZHOU, H., ALEXANDER, D. and LANGE, K. (2011). A quasi-Newton acceleration for highdimensional optimization algorithms. Stat. Comput. 21 261-273. MR2774856
43. ZHOU, H. and LANGE, K. (2011). A path algorithm for constrained estimation. Available at arXiv:1103.3738v1.