SparseNet: Coordinate descent with nonconvex penalties

Journal of the American Statistical Association

Volumn 106, Issue 495, 2011, Pages 1125-1138

  Mazumder, Rahul ^a   Friedman, Jerome H ^a   Hastie, Trevor ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Degrees of freedom; LASSO; Nonconvex optimization; Regularization surface; Sparse regression; Variable selection

Indexed keywords

EID: 80052694528     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/jasa.2011.tm09738     Document Type: Article

References (27)

1. Borwein, J. M., and Lewis, A. S. (2006), Convex Analysis and Nonlinear Optimization (2nd ed.), New York: Springer. [1137]
2. Breiman, L. (1996), "Heuristics of Instability and Stabilization in Model Selection, " The Annals of Statistics, 24, 2350-2383. [1129]
3. Candes, E. J., Wakin, M. B., and Boyd, S. (2008), "Enhancing Sparsity by Reweighted l1 Minimization, " Journal of Fourier Analysis and Applications, 14 (5), 877-905. [1127,1133-1135]
4. Chen, S., and Donoho, D. (1994), "On Basis Pursuit, " technical report, Dept. of Statistics, Stanford University. [1125]
5. 1-Norm Solution Is the Sparsest Solution, " Communications on Pure and Applied Mathematics, 59, 797-829. [1125]
6. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004), "Least Angle Regression" (with discussion), The Annals of Statistics, 32 (2), 407-499. [1125,1129]
7. Fan, J., and Li, R. (2001), "Variable Selection via Nonconcave Penalized Likelihood and Its Oracle Properties, " Journal of the American Statistical Association, 96 (456), 1348-1360. [1126,1128,1129,1134]
8. Frank, I., and Friedman, J. (1993), "A Statistical View of Some Chemometrics Regression Tools" (with discussion), Technometrics, 35 (2), 109-148. [1128]
9. Friedman, J. (2008), "Fast Sparse Regression and Classification, " technical report, Dept. of Statistics, Stanford University. [1125,1126,1128,1135]
10. Friedman, J., Hastie, T., and Tibshirani, R. (2009), "Glmnet: Lasso and Elastic-Net Regularized Generalized Linear Models, " R package version 1.1-4, available at http://www-stat.stanford.edu/~hastie/Papers/glmnet.pdf. [1125]
11. Gao, H.-Y., and Bruce, A. G. (1997), "Waveshrink With Firm Shrinkage, " Statistica Sinica, 7, 855-874. [1127,1128]
12. Knight, K., and Fu, W. (2000), "Asymptotics for Lasso-Type Estimators, " The Annals of Statistics, 28 (5), 1356-1378. [1125]
13. Meinshausen, N., and Bühlmann, P. (2006), "High-Dimensional Graphs and Variable Selection With the Lasso, " The Annals of Statistics, 34, 1436-1462. [1125]
14. Nikolova, M. (2000), "Local Strong Homogeneity of a Regularized Estimator, " SIAM Journal of Applied Mathematics, 61, 633-658. [1128]
15. Osborne, M., Presnell, B., and Turlach, B. (2000), "A New Approach to Variable Selection in Least Squares Problems, " IMA Journal of Numerical Analysis, 20, 389-404. [1125]
16. She, Y. (2010), "Sparse Regression With Exact Clustering, " Electronic Journal of Statistics, 4, 1055-1096. [1127,1136]
17. Stein, C. (1981), "Estimation of the Mean of a Multivariate Normal Distribution, " The Annals of Statistics, 9, 1131-1151. [1129,1131]
18. Tibshirani, R. (1996), "Regression Shrinkage and Selection via the Lasso, " Journal of the Royal Statistical Society, Ser. B, 58, 267-288. [1125]
19. Tseng, P. (2001), "Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization, " Journal of Optimization Theory and Applications, 109, 475-494. [1126]
20. Tseng, P., and Yun, S. (2009), "A Coordinate Gradient Descent Method for Nonsmooth Separable Minimization, " Mathematical Programming B, 117, 387-423. [1126,1132]
21. Zhang, C.-H. (2010a), "Nearly Unbiased Variable Selection Under Minimax Concave Penalty, " The Annals of Statistics, 38, 894-942. [1125-1128,1135]
22. Zhang, C.-H., and Huang, J. (2008), "The Sparsity and Bias of the Lasso Selection in High-Dimensional Linear Regression, " The Annals of Statistics, 36 (4), 1567-1594. [1125]
23. Zhang, T. (2010b), "Analysis of Multi-Stage Convex Relaxation for Sparse Regularization, " Journal of Machine Learning Research, 11, 1081-1107. [1127,1128,1133,1135,1136]
24. Zhao, P., and Yu, B. (2006), "On Model Selection Consistency of Lasso, " Journal of Machine Learning Research, 7, 2541-2563. [1125]
25. Zou, H. (2006), "The Adaptive Lasso and Its Oracle Properties, " Journal of the American Statistical Association, 101, 1418-1429. [1125,1126,1135]
26. Zou, H., and Li, R. (2008), "One-Step Sparse Estimates in Nonconcave Penalized Likelihood Models, " The Annals of Statistics, 36 (4), 1509-1533. [1125-1127,1133-1135]
27. Zou, H., Hastie, T., and Tibshirani, R. (2007), "On the Degrees of Freedom of the Lasso, " The Annals of Statistics, 35 (5), 2173-2192. [1129,1131]