Lasso logistic regression, Gsoft and the cyclic coordinate descent algorithm: Application to gene expression data

Statistical Applications in Genetics and Molecular Biology

Volumn 9, Issue 1, 2010, Pages

  Garcia Magariños, Manuel ^a   Antoniadis, Anestis ^b   Cao, Ricardo ^c   Gonázlez Manteiga, Wenceslao ^a  

^a UNIVERSITY OF SANTIAGO DE COMPOSTELA   (Spain)

^b UNIV GRENOBLE ALPES   (France)

^c UNIVERSITY OF A CORUÑA   (Spain)

Author keywords

CCD algorithm; gene expression; GSoft; lasso; logistic regression; optimization; penalized regression

Indexed keywords

ALGORITHM; ARTICLE; COLON; CYCLIC COORDINATE DESCENT ALGORITHM; GENE EXPRESSION; GENERALIZED SOFT THRESHOLD ESTIMATOR; LEUKEMIA; LOGISTIC REGRESSION ANALYSIS; PHENOTYPE; STATISTICAL ANALYSIS; COLON TUMOR; DNA MICROARRAY; FACTUAL DATABASE; GENE EXPRESSION PROFILING; GENETICS; METHODOLOGY; STATISTICAL MODEL;

ALGORITHMS; COLONIC NEOPLASMS; DATABASES, FACTUAL; GENE EXPRESSION; GENE EXPRESSION PROFILING; LEUKEMIA; LOGISTIC MODELS; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS;

EID: 77956417185     PISSN: None     EISSN: 15446115     Source Type: Journal    
DOI: 10.2202/1544-6115.1536     Document Type: Article

References (59)

1. Alon U., Barkai N., Notterman D., Gish K., Ybarra S., Mack D. and Levine A.J. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences USA, 96(12), (1999), 6745-6750.
2. Antoniadis A. and Fan J. Regularization of wavelet approximations. Journal of the American Statistical Association, 96(455), (2001), 939-967.
3. Antoniadis A., Gijbels I. and Nikolova M. Penalized likelihood regression for generalized linear models with nonquadratic 'penalties. Annals of the Institute of Statistical Mathematics (in press), (2009).
4. Bakin S. Adaptive regression and model selection in data mining problems. PhD Thesis, Australian National University, Canberra, (1999).
5. Boulesteix A.L., Tutz G. and Strimmer K. A CART-based approach to discover emerging patterns in microarray data. Bioinformatics, 19(18), (2003), 2465-2472.
6. De Leeuw J. Block-relaxation methods in statistics. In Bock H.H., Lenski W. and Richter M.M., editors, Information Systems and Data Analysis, Springer-Verlag, Berlin, (1994).
7. De Mol C., De Vito E. and Rosasco L. Elastic-net regularization in learning theory. Journal of Complexity, 25(2), (2009), 201-230.
8. Dettling M. BagBoosting for tumor classification with gene expression data. Bioinformatics, 20(18), (2004), 3583-3593.
9. Dettling M. and Buhlmann P. Finding predictive gene groups from microarray data. Journal of Multivariate Analysis, 90, (2004), 106-131.
10. Dettling M. and Buhlmann P. Supervised clustering of genes. Genome Biology, 3(12), (2002), 0069.1-0069.15.
11. Dudoit S., Fridlyand J. and Speed T.P. Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association, 97(457), (2002), 77-87.
12. Efron B., Hastie T., Johnstone I. and Tibshirani R. Least angle regression. Annals of Statistics, 32(2) (2004), 407-499.
13. Eggermont P.P. and LaRiccia V.N. Maximum penalized likelihood estimation. Vol. 2 Regression, Springer Series in Statistics, New York, (2009).
14. Fan J. and Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), (2001), 1348-1360.
15. Frank I.E. and Friedman J.H. A statistical view of some chemometrics tools. Technometrics, 35(2), (1993), 109-135.
16. Friedman J., Hastie T. and Tibshirani R. Regularization paths for generalized linear models via coordinate descent. Technical Report, Department of Statistics, Stanford University, (2008).
17. Genkin A., Lewis D.D. and Madigan D. Sparse logistic regression for text categorization. DIMACS Working Group on Monitoring Message Streams, Project Report, (2005).
18. Goldstein T. and Osher S. The Split Bregman method for L1 regularized problems. UCLA CAAM Report 08-29, (2008).
19. Golub T.R., Slonim D.K., Tamayo P., Huard C., Gaasenbeck M., Mesirov P., Coller H., Loh M.L., Downing J.R., Caligiuri M.A. et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science, 286(5439), (1999), 531-537.
20. Guyon I. and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research, 3(Mar), (2003), 1157-1182.
21. Guyon I., Weston J., Barnhill S. and Vapnik V. Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), (2002), 389-422.
22. Hall M. Correlation-based feature selection for machine learning. PhD Thesis, Department of Computer Science, Waikato University, New Zealand, (1999).
23. Hoerl A.E. and Kennard R. Ridge regression: biased estimation for nonorthog-onal problems. Technometrics, 12(1), (1970), 55-67.
24. Huang J., Horowitz J.L. and Ma S. Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Technical Report, Department of Statistics and Actuarial Science, The University of Iowa, (2006).
25. Huang J., Ma S. and Zhang C.H. Adaptive lasso for sparse high-dimensional regression models. Statistica Sinica, 18(4), (2008), 1603-1618.
26. Huang J., Ma S. and Zhang C.H. The iterated lasso for high-dimensional logistic regression. Technical report No. 392, The University of Iowa, (2008).
27. Huber P.J. The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Simposium on Mathematical Statistics and Probability, (1967).
28. Hunter D.R. and Li R. Variable selection using MM algorithms. Annals of Statistics, 33(4), (2005), 1617-1642.
29. Kim Y., Choi H. and Oh H.S. Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103(484), (2008), 1665-1673.
30. Klinger A. Inference in high dimensional generalized linear models based on soft thresholding. Journal of the Royal Statistical Society Series B, 63(2), 377-392.
31. Knight K. and Fu W.J. Asymptotics for lasso-type estimators. Annals of Statistics, 28(5), (2000), 1356-1378.
32. Krishnapuram B., Carin L., Figueiredo M.A.T. and Hartemink A.J. Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6), 957-968.
33. Lee S.I., Lee H., Abbeel P. and Ng A.Y. Efficient L1 regularized logistic regression. Proceedings of the Twenty-first International Conference on Machine Learning (AAAI-06), (2006).
34. Lee K.E., Sha N., Dougherty E.R., Vannucci M. and Mallick B.K. Gene selection: a bayesian variable selection approach. Bioinformatics, 19(1), 90-97.
35. Liu Z., Jiang F., Tian G., Wang S., Sato F., Meltzer S.J. and Tan M. Sparse logistic regression with Lp penalty for biomarker identification. Statistical Applications in Genetics and Molecular Biology, 6(1), (2007), article 6.
36. Lv J. and Fan Y. A unified approach to model selection and sparse recovery using regularized least squares. Annals of Statistics, 37(6A), (2009), 34983528.
37. Meier L., van de Geer S. and Buhlmann P. The group lasso for logistic regression. Journal of the Royal Statistical Society Series B, 70(1), (2008), 53-71.
38. Meinshausen N. and Buhlmann P. High dimensional graphs and variable selection with the lasso. Annals of Statistics, 34(3), (2006), 1436-1462.
39. Nguyen D.V. and Rocke D.M. Tumor classification by partial least squares using microarray gene expression data. Bioinformatics, 18(1), (2002), 3950.
40. Park M.Y. and Hastie T. An Li regularization-path algorithm for generalized linear models. Manuscript, Department of Statistics, Stanford University.
41. Roth V. The generalized lasso. IEEE Transactions on Neural Networks, 15, (2004), 16-28.
42. Schmidt M., Fung G. and Rosales R. Fast optimization methods for Li regularization: a comparative study and two new approaches. European Conference on Machine Learning (ECML), (2007).
43. Shevade S. and Keerthi S. A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics, 19(17), (2003), 22462253.
44. Tarigan B. and van de Geer S. Classifiers of support vector machine type with Li complexity regularization. Bernoulli, 12(6), (2006), 1045-1076.
45. Thomas J.G., Olson J.M. and Tapscott S.J. An efficient and robust statistical modeling approach to discover diferentially expressed genes using genomic expression profiles. Genome Research, 11, (2001), 1227-1236.
46. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B, 58(1), (1996), 267-288.
47. Tseng P. Convergence of block coordinate descent method for nondifferentiable maximization. Journal of Optimization Theory and Applications, 109, (2001), 473-492.
48. Tseng P. and Yun S. A coordinate descent gradient method for nonsmooth separable minimization. Mathematical Programming, 117, (2009), 387423.
49. Wang H. and Leng C. Unified lasso estimation via least squares approximation. Journal of the American Statistical Association, 102(479), (2007), 10391048.
50. Weston J., Elisseeff A., Scholkopf B. and Tipping M. Use of the zero-norm with linear models and kernel methods. Journal of Machine Learning Research, 3(Mar), (2003), 1439-1461.
51. Wright S.J., Nowak R.D. and Figueiredo M.A.T. Sparse reconstruction by separable approximation. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), (2008).
52. Wu T.T. and Lange K. Coordinate descent algorithms for lasso penalized regression. Annals of Applied Statistics, 2(1), (2008), 224-244.
53. Yuan M. and Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society Series B, 68(1), (2006), 49-67.
54. Zhang C.H. and Huang J. The sparsity and bias of the lasso selection in high-dimensional linear regression. Annals of Statistics, 36(4), (2008), 15671594.
55. Zhang T. Some sharp performance bounds for least squares regression with L1 regularization. Annals of Statistics, 37(5A), (2009), 2109-2144.
56. Zhang T. and Oles F. Text categorization based on regularized linear classifiers. Information Retrieval, 4(1), (2001), 5-31.
57. Zou H. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), (2006), 1418-1429.
58. Zou H. and Hastie T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B, 67(2), (2005), 301-320.
59. Zou H. and Li R. One-step sparse estimates in nonconcave penalized likelihood models. Annals of Statistics, 36(4), (2008), 1509-1533.