Efficient approximate k-fold and leave-one-out cross-validation for ridge regression

Biometrical Journal

Volumn 55, Issue 2, 2013, Pages 141-155

  Meijer, Rosa J ^a   Goeman, Jelle J ^a  

^a LEIDEN UNIVERSITY MEDICAL CENTER   (Netherlands)

Author keywords

Approximation method; Cross validation; Matrix Inversion Lemma; Ridge regression; Survival analysis

Indexed keywords

APPROXIMATION THEORY;

APPROXIMATION METHODS; BUILDING EVALUATIONS; CROSS VALIDATION; LEAVE ONE OUT; MATRIX INVERSION LEMMA; MATRIX INVERSIONS; MODEL EVALUATION; RESAMPLING METHOD; RIDGE REGRESSION; SURVIVAL ANALYSIS;

REGRESSION ANALYSIS;

EID: 84875002620     PISSN: 03233847     EISSN: 15214036     Source Type: Journal    
DOI: 10.1002/bimj.201200088     Document Type: Article

References (30)

1. Allen, D. (1974). The relationship between variable selection and data agumentation and a method for prediction. Technometrics 16, 125-127.
2. Arlot, S. and Celisse, A. (2010). A survey of cross-validation procedures for model selection. Statistics Surveys 4, 40-79.
3. Bartlett, M. (1951). An inverse matrix adjustment arising in discriminant analysis. Annals of Mathematical Statistics 22, 107-111.
4. Bøvelstad, H., Nygård, S., Størvold, H., Aldrin, M., Borgan, Ø., Frigessi, A. and Lingjærde, O. (2007). Predicting survival from microarray data-a comparative study. Bioinformatics 23, 2080-2087.
5. Cawley, G. and Talbot, N. (2004). Fast exact leave-one-out cross-validation of sparse least-squares support vector machines. Neural Networks 17, 1467-1475.
6. Cook, R. and Weisberg, S. (1992). Residuals and Influence in Regression. Chapman & Hall, New York.
7. Cox, D. (1972). Regression models and life-tables. Journal of the Royal Statistical Society. Series B (Methodological) 34, 187-220.
8. Desmedt, C., Piette, F., Loi, S., Wang, Y., d'Assignies, M.S., Bergh, J., Lidereau, R., Ellis, P., Harris, A.L., Klijn, J.G.M., Foekens, J.A., Cardoso, F., Piccart, M.J., Buyse, M. and Sotiriou, C. (2007). Strong time dependence of the 76-gene prognostic signature for node-negative breast cancer patients in the transbig multicenter independent validation series. Clinical Cancer Research 13, 3207-3214.
9. Golub, G., Heath, M. and Wahba, G. (1979). Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215-223.
10. Gui, J. and Li, H. (2005). Penalized Cox regression analysis in the high-dimensional and low-sample size settings, with applications to microarray gene expression data. Bioinformatics 21, 3001-3008.
11. Hager, W. (1989). Updating the inverse of a matrix. SIAM Review 31, 221-239.
12. Haibe-Kains, B., Desmedt, C., Sotiriou, C., and Bontempi, G. (2008). A comparative study of survival models for breast cancer prognostication based on microarray data: does a single gene beat them all? Bioinformatics 24, 2200-2208.
13. Huang, J. and Harrington, D. (2002). Penalized partial likelihood regression for right-censored data with bootstrap selection of the penalty parameter. Biometrics 58, 781-791.
14. Le Cessie, S. and Van Houwelingen, J. (1992). Ridge estimators in logistic regression. Journal of the Royal Statistical Society. Series C (Applied Statistics) 41, 191-201.
15. McCullagh, P. and Nelder, J.A. (1989). Generalized Linear Models. Chapman and Hall, New York.
16. Rosenwald, A., Wright, G., Chan, W.C., Connors, J.M., Campo, E., Fisher, R.I., Gascoyne, R.D., Muller-Hermelink, H.K., Smeland, E.B. and Staudt, L.M., (2002). The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. New England Journal of Medicine 346, 1937-1947.
17. Schumacher, M., Binder, H. and Gerds, T. (2007). Assessment of survival prediction models based on microarray data. Bioinformatics 23, 1768-1774.
18. Sherman, J. and Morrison, W. (1950). Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Annals of Mathematical Statistics 21, 124-127.
19. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society. Series B (Methodological) 36, 111-147.
20. Tapp, H. and Kemsley, E. (2008). Optimizing the efficiency of cross-validation in linear discriminant analysis through selective use of the Sherman-Morrison-Woodbury inversion formula. Journal of Chemometrics 22, 419-421.
21. Tran, M. (2009). Penalized maximum likelihood principle for choosing ridge parameter. Communications in Statistics-Simulation and Computation 38, 1610-1624.
22. van de Vijver, M.J., He, Y.D., van't Veer, L.J., Dai, H., Hart, A.A., Voskuil, D.W., Schreiber, G.J., Peterse, J.L., Roberts, C., Marton, M.J., Parrish, M., Atsma, D., Witteveen, A., Glas, A., Delahaye, L., van der Velde, T., Bartelink, H., Rodenhuis, S., Rutgers, E.T., Friend, S.H. and Bernards, R. (2002). A gene-expression signature as a predictor of survival in breast cancer. New England Journal of Medicine 347, 1999-2009.
23. Van Houwelingen, H., Bruinsma, T., Hart, A., Van't Veer, L., and Wessels, L. (2006). Cross-validated cox regression on microarray gene expression data. Statistics in medicine 25, 3201-3216.
24. Van Wieringen, W., Kun, D., Hampel, R., and Boulesteix, A. (2009). Survival prediction using gene expression data: a review and comparison. Computational Statistics & Data Analysis 53, 1590-1603.
25. Verweij, P. and Van Houwelingen, H. (1993). Cross-validation in survival analysis. Statistics in Medicine 12, 2305-2314.
26. Wang, H., Li, B., and Leng, C. (2009). Shrinkage tuning parameter selection with a diverging number of parameters. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71, 671-683.
27. Wang, Y.X., Klijn, J.G.M., Zhang, Y., Sieuwerts, A., Look, M.P., Yang, F., Talantov, D., Timmermans, M., Meijer-van Gelder, M.E., Yu, J., Jatkoe, T., Berns, E.M.J.J., Atkins, D. and Foekens, J.A. (2005). Gene-expression profiles to predict distant metastasis of lymph-node-negative primary breast cancer. The Lancet 365, 671-679.
28. Westerhuis, J. A., Hoefsloot, H. C. J., Smit, S., Vis, D. J., Smilde, A. K., van Velzen, E. J. J., van Duijnhoven, J. P. M. and van Dorsten, F. A. (2008). Assessment of PLSDA cross validation. Metabolomics 4, 81-89.
29. Woodbury, M. (1950). Inverting modified matrices. Memorandum Report 42, Statistical Research Group Princeton, Princeton University, Princeton, New Jersey.
30. Zhang, P. (1993). Model selection via multifold cross validation. The Annals of Statistics 21, 299-313.