Regularization paths for generalized linear models via coordinate descent

Journal of Statistical Software

Volumn 33, Issue 1, 2010, Pages 1-22

  Friedman, Jerome ^a   Hastie, Trevor ^a   Tibshirani, Rob ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

Coordinate descent; Elastic net; L1 penalty; Lasso; Logistic regression; Regularization path

Indexed keywords

EID: 77950537175     PISSN: 15487660     EISSN: 15487660     Source Type: Journal    
DOI: 10.18637/jss.v033.i01     Document Type: Article

References (49)

1. Andrew G, Gao J (2007a). OWL-QN: Orthant-Wise Limited-Memory Quasi-Newton Optimizer for L1-Regularized Objectives. URL http://research.microsoft.com/en-us/downloads/b1eb1016-1738-4bd5-83a9-370c9d498a03.
2. Andrew G, Gao J (2007b). "Scalable Training of L1-Regularized Log-Linear Models. In ICML '07: Proceedings of the 24th International Conference on Machine Learning, pp. 33-40. ACM, New York, NY, USA. doi:10.1145/1273496.1273501.
3. Bates D, Maechler M (2009). Matrix: Sparse and Dense Matrix Classes and Methods. R package version 0.999375-30, URL http://CRAN.R-project.org/package=Matrix.
4. Candes E, Tao T (2007). "The Dantzig Selector: Statistical Estimation When p is much Larger than n. The Annals of Statistics, 35(6), 2313-2351.
5. Chen SS, Donoho D, Saunders M (1998). "Atomic Decomposition by Basis Pursuit. SIAM Journal on Scientific Computing, 20(1), 33-61.
6. Daubechies I, Defrise M, De Mol C (2004). "An Iterative Thresholding Algorithm for Linear Inverse Problems with a Sparsity Constraint. Communications on Pure and Applied Mathematics, 57, 1413-1457.
7. Dettling M (2004). "BagBoosting for Tumor Classification with Gene Expression Data. Bioinformatics, 20, 3583-3593.
8. Donoho DL, Johnstone IM (1994). "Ideal Spatial Adaptation by Wavelet Shrinkage. Biometrika, 81, 425-455.
9. Efron B, Hastie T, Johnstone I, Tibshirani R (2004). "Least Angle Regression. The Annals of Statistics, 32(2), 407-499.
10. Fan J, Li R (2005). "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties. Journal of the American Statistical Association, 96, 1348-1360.
11. Friedman J (2008). "Fast Sparse Regression and Classification." Technical report, Department of Statistics, Stanford University. URL http://www-stat.stanford.edu/~jhf/ftp/GPSpub.pdf.
12. Friedman J, Hastie T, Hoeing H, Tibshirani R (2007). "Pathwise Coordinate Optimization. The Annals of Applied Statistics, 2(1), 302-332.
13. Friedman J, Hastie T, Tibshirani R (2008). "Sparse Inverse Covariance Estimation with the Graphical Lasso. Biostatistics, 9, 432-441.
14. Friedman J, Hastie T, Tibshirani R (2009). Glmnet: Lasso and Elastic-Net Regularized Generalized Linear Models. R package version 1.1-4, URL http://CRAN.R-project.org/package=glmnet.
15. Fu W (1998). "Penalized Regressions: The Bridge vs. the Lasso. Journal of Computational and Graphical Statistics, 7(3), 397-416.
16. Genkin A, Lewis D, Madigan D (2007). "Large-scale Bayesian Logistic Regression for Text Categorization. Technometrics, 49(3), 291-304.
17. Goeman J (2009a). "L1 Penalized Estimation in the Cox Proportional Hazards Model." Biometrical Journal. doi:10.1002/bimj.200900028. Forthcoming.
18. Goeman J (2009b). Penalized: L1 (Lasso) and L2 (Ridge) Penalized Estimation in GLMs and in the Cox Model. R package version 0.9-27, URL http://CRAN.R-project.org/package=penalized.
19. Golub T, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD, Lander ES (1999). "Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring. Science, 286, 531-536.
20. Hastie T, Efron B (2007). lars: Least Angle Regression, Lasso and Forward Stagewise. R package version 0.9-7, URL http://CRAN.R-project.org/package=Matrix.
21. Hastie T, Rosset S, Tibshirani R, Zhu J (2004). "The Entire Regularization Path for the Support Vector Machine. Journal of Machine Learning Research, 5, 1391-1415.
22. Hastie T, Tibshirani R, Friedman J (2009). The Elements of Statistical Learning: Prediction, Inference and Data Mining. 2nd edition. Springer-Verlag, New York.
23. Jiang H (2009). "A MATLAB Implementation of glmnet." Stanford University, URL http: //www-stat.stanford.edu/~tibs/glmnet-matlab/
24. Koh K, Kim SJ, Boyd S (2007a). "An Interior-Point Method for Large-Scale L1-Regularized Logistic Regression. Journal of Machine Learning Research, 8, 1519-1555.
25. Koh K, Kim SJ, Boyd S (2007b). l1logreg: A Solver for L1-Regularized Logistic Regression. R package version 0.1-1. Avaliable from Kwangmoo Koh (deneb1@stanford.edu).
26. Krishnapuram B, Hartemink AJ (2005). "Sparse Multinomial Logistic Regression: Fast Algorithms and Generalization Bounds." IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6), 957-968. Fellow-Lawrence Carin and Senior Member-Mario A. T. Figueiredo.
27. Kushmerick N (1999). "Learning to Remove Internet Advertisements. In AGENTS '99: Proceedings of the Third Annual Conference on Autonomous Agents, pp. 175-181. ACM, New York, NY, USA. ISBN 1-58113-066-X. doi:10.1145/301136.301186.
28. Lang K (1995). "NewsWeeder: Learning to Filter Netnews." In A Prieditis, S Russell (eds.), Proceedings of the 12th International Conference on Machine Learning, pp. 331-339. San Francisco.
29. Lee S, Lee H, Abbeel P, Ng A (2006). "Eficient L1 Logistic Regression." In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-06). URL https: //www.aaai.org/Papers/AAAI/2006/AAAI06-064.pdf.
30. Madigan D, Lewis D (2007). BBR, BMR: Bayesian Logistic Regression. Open-source standalone software, URL http://www.bayesianregression.org/
31. Meier L, van de Geer S, Bühlmann P (2008). The Group Lasso for Logistic Regression. Journal of the Royal Statistical Society B, 70(1), 53-71.
32. Osborne M, Presnell B, Turlach B (2000). "A New Approach to Variable Selection in Least Squares Problems. IMA Journal of Numerical Analysis, 20, 389-404.
33. Park MY, Hastie T (2007a). "L1-Regularization Path Algorithm for Generalized Linear Models. Journal of the Royal Statistical Society B, 69, 659-677.
34. Park MY, Hastie T (2007b). glmpath: L1 Regularization Path for Generalized Linear Models and Cox Proportional Hazards Model. R package version 0.94, URL http: //CRAN.R-project.org/package=glmpath.
35. Ramaswamy S, Tamayo P, Rifkin R, Mukherjee S, Yeang C, Angelo M, Ladd C, Reich M, Latulippe E, Mesirov J, Poggio T, Gerald W, Loda M, Lander E, Golub T (2002). "Multiclass Cancer Diagnosis Using Tumor Gene Expression Signature. Proceedings of the National Academy of Sciences, 98, 15149-15154.
36. R Development Core Team (2009). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http: //www.R-project.org/.
37. Rosset S, Zhu J (2007). "Piecewise Linear Regularized Solution Paths. The Annals of Statistics, 35(3), 1012-1030.
38. Shevade K, Keerthi S (2003). "A Simple and Eficient Algorithm for Gene Selection Using Sparse Logistic Regression. Bioinformatics, 19, 2246-2253.
39. Tibshirani R (1996). "Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society B, 58, 267-288.
40. Tibshirani R (1997). "The Lasso Method for Variable Selection in the Cox Model. Statistics in Medicine, 16, 385-395.
41. Tseng P (2001). "Convergence of a Block Coordinate Descent Method for Nondifierentiable Minimization. Journal of Optimization Theory and Applications, 109, 475-494.
42. Van der Kooij A (2007). Prediction Accuracy and Stability of Regrsssion with Optimal Scaling Transformations. Ph.D. thesis, Department of Data Theory, University of Leiden. URL https://openaccess.leidenuniv.nl/dspace/handle/1887/12096.
43. Wu T, Chen Y, Hastie T, Sobel E, Lange K (2009). "Genome-Wide Association Analysis by Penalized Logistic Regression. Bioinformatics, 25(6), 714-721.
44. Wu T, Lange K (2008). "Coordinate Descent Procedures for Lasso Penalized Regression. The Annals of Applied Statistics, 2(1), 224-244.
45. Yuan M, Lin Y (2007). "Model Selection and Estimation in Regression with Grouped Variables. Journal of the Royal Statistical Society B, 68(1), 49-67.
46. Zhu J, Hastie T (2004). "Classification of Expression Arrays by Penalized Logistic Regression. Biostatistics, 5(3), 427-443.
47. Zou H (2006). "The Adaptive Lasso and its Oracle Properties. Journal of the American Statistical Association, 101, 1418-1429.
48. Zou H, Hastie T (2004). elasticnet: Elastic Net Regularization and Variable Selection. R package version 1.02, URL http://CRAN.R-project.org/package=elasticnet.
49. Zou H, Hastie T (2005). "Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society B, 67(2), 301-320.