Variable selection for generalized linear mixed models by L1-penalized estimation

Statistics and Computing

Volumn 24, Issue 2, 2014, Pages 137-154

  Groll, Andreas ^a   Tutz, Gerhard ^a  

^a UNIVERSITY OF MUNICH   (Germany)

Author keywords

Generalized linear mixed model; Gradient ascent; Lasso; Linear models; Penalty; Variable selection

Indexed keywords

EID: 84893918594     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11222-012-9359-z     Document Type: Article

References (75)

1. Akaike, H.: Information theory and the extension of the maximum likelihood principle. In: Second International Symposium on Information Theory, pp. 267-281 (1973).
2. Bates, D., Maechler, M.: lme4: linear mixed-effects models using S4 classes. R package version 0. 999375-34 (2010).
3. Bondell, H. D., Krishna, A., Ghosh, S. K.: Joint variable selection of fixed and random effects in linear mixed-effects models. Biometrics 66, 1069-1077 (2010).
4. Booth, J. G.: Bootstrap methods for generalized mixed models with applications to small area estimation. In: Seeber, G. U. H., Francis, B. J., Hatzinger, R., Steckel-Berger, G. (eds.) Statistical Modelling, vol. 104, pp. 43-51. Springer, New York (1996).
5. Booth, J. G., Hobert, J. P.: Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. J. R. Stat. Soc. B 61, 265-285 (1999).
6. Breiman, L.: Heuristics of instability and stabilization in model selection. Ann. Stat. 6, 2350-2383 (1996).
7. Breiman, L.: Arcing classifiers. Ann. Stat. 26, 801-849 (1998).
8. Breslow, N. E., Clayton, D. G.: Approximate inference in generalized linear mixed model. J. Am. Stat. Assoc. 88, 9-25 (1993).
9. Breslow, N. E., Lin, X.: Bias correction in generalized linear mixed models with a single component of dispersion. Biometrika 82, 81-91 (1995).
10. Broström, G.: glmmML: generalized linear models with clustering. R package version 0. 81-6 (2009).
11. Bühlmann, P., Hothorn, T.: Boosting algorithms: regularization, prediction and model fitting. Stat. Sci. 22, 477-522 (2007).
12. 2 loss: regression and classification. J. Am. Stat. Assoc. 98, 324-339 (2003).
13. Candes, E., Tao, T.: The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35, 2313-2351 (2007).
14. Chatterjee, A., Lahiri, S. N.: Bootstrapping lasso estimators. J. Am. Stat. Assoc. 106, 608-625 (2011).
15. Davison, A. C., Hinkley, D. V.: Bootstrap Methods and Their Application. Cambridge University Press, Cambridge (1997).
16. Efron, B.: The Jackknife, the Bootstrap and Other Resampling Plans. SIAM: CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 38. (1982).
17. Efron, B.: Estimating the error rate of a prediction rule: improvement on crossvalidation. J. Am. Stat. Assoc. 78, 316-331 (1983).
18. Efron, B.: How biased is the apparent error rate of a prediction rule? J. Am. Stat. Assoc. 81, 461-470 (1986).
19. Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993).
20. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32, 407-499 (2004).
21. Fahrmeir, L., Lang, S.: Bayesian inference for generalized additive mixed models based on Markov random field priors. Appl. Stat. 50, 201-220 (2001). doi: 10. 1111/1467-9876. 00229.
22. Fahrmeir, L., Tutz, G.: Multivariate Statistical Modelling Based on Generalized Linear Models, 2nd edn. Springer, New York (2001).
23. Fan, J., Li, R.: Variable selection via nonconcave penalize likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 1348-1360 (2001).
24. Freund, Y., Schapire, R. E.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148-156. Morgan Kaufmann, San Francisco (1996).
25. Friedman, J. H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 337-407 (2001).
26. Geissler, S.: The predictive sample reuse method with applications. J. Am. Stat. Assoc. 70, 320-328 (1975).
27. Genkin, A., Lewis, D., Madigan, D.: Large-scale Bayesian logistic regression for text categorization. Technometrics 49, 291-304 (2007).
28. 1 penalized estimation in the Cox proportional hazards model. Biom. J. 52, 70-84 (2010).
29. 1-penalized Estimation. R package version 1. 0. 1 (2011a).
30. Groll, A.: GMMBoost: Componentwise Likelihood-based Boosting Approaches to Generalized Mixed Models. R package version 1. 0. 2 (2011b).
31. Gui, J., Li, H. Z.: Penalized Cox regression analysis in the high-dimensional and low-sample size settings, with applications to microarray gene expression data. Bioinformatics 21, 3001-3008 (2005).
32. Hastie, T., Rosset, S., Tibshirani, R., Zhu, J.: The entire regularization path for the support vector machine. J. Mach. Learn. Res. 5, 1391-1415 (2004).
33. Ibrahim, J. G., Zhu, H., Garcia, R. I., Guo, R.: Fixed and random effects selection in mixed effects models. Biometrics 67, 495-503 (2011).
34. James, G. M., Radchenko, P.: A generalized Dantzig selector with shrinkage tuning. Biometrika 96(2), 323-337 (2009).
35. Kim, Y., Kim, J.: Gradient lasso for feature selection. In: Proceedings of the 21st International Conference on Machine Learning. ACM International Conference Proceeding Series, vol. 69, pp. 473-480 (2004).
36. Kneib, T., Hothorn, T., Tutz, G.: Variable selection and model choice in geoadditive regression. Biometrics 65, 626-634 (2009).
37. Lesaffre, E., Asefa, M., Verbeke, G.: Assessing the godness-of-fit of the laird and ware model-an example: the Jimma infant survival differential longitudinal study. Stat. Med. 18, 835-854 (1999).
38. Lin, X., Breslow, N. E.: Bias correction in generalized linear mixed models with multiple components of dispersion. J. Am. Stat. Assoc. 91, 1007-1016 (1996).
39. Littell, R., Milliken, G., Stroup, W., Wolfinger, R.: SAS System for Mixed Models. SAS Institute Inc., Cary (1996).
40. McCullagh, P.: Re-sampling and exchangeable arrays. Bernoulli 6, 303-322 (2000).
41. McCulloch, C. E., Searle, S. R., Neuhaus, J. M.: Generalized, Linear and Mixed Models, 2nd edn. Wiley, New York (2008).
42. Meier, L., van de Geer, S., Bühlmann, P.: The group lasso for logistic regression. J. R. Stat. Soc. B 70, 53-71 (2008).
43. Ni, X., Zhang, D., Zhang, H. H.: Variable selection for semiparametric mixed models in longitudinal studies. Biometrics 66, 79-88 (2010).
44. Osborne, M., Presnell, B., Turlach, B.: On the lasso and its dual. J. Comput. Graph. Stat. (2000).
45. 1-regularization path algorithm for generalized linear models. J. R. Stat. Soc. B 19, 659-677 (2007).
46. Picard, R., Cook, D.: Cross-validation of regression models. J. Am. Stat. Assoc. 79, 575-583 (1984).
47. Pinheiro, J. C., Bates, D. M.: Mixed-Effects Models in S and S-Plus. Springer, New York (2000).
48. Radchenko, P., James, G. M.: Variable inclusion and shrinkage algorithms. J. Am. Stat. Assoc. 103, 1304-1315 (2008).
49. Schall, R.: Estimation in generalised linear models with random effects. Biometrika 78, 719-727 (1991).
50. Schelldorfer, J.: lmmlasso: Linear mixed-effects models with Lasso. R package version 0. 1-2. (2011).
51. 1-penalization. Preprint, ETH Zurich, (2011). http://stat. ethz. ch/people/schell.
52. 1-penalization. Scand. J. Stat. 38(2), 197-214 (2011).
53. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461-464 (1978).
54. Segal, M. R.: Microarray gene expression data with linked survival phenotypes: diffuse large-b-cell lymphoma revisited. Biostatistics 7, 268-285 (2006).
55. Shang, J., Cavanaugh, J. E.: Bootstrap variants of the Akaike information criterion for mixed model selection. Comput. Stat. Data Anal. 52, 2004-2021 (2008).
56. Shevade, S. K., Keerthi, S. S.: A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics 19, 2246-2253 (2003).
57. Stone, M.: Cross-validatory choice and assessment of statistical predictions (with discussion). J. R. Stat. Soc. B 36, 111-147 (1974).
58. Stone, M.: Cross-validation: A review. Math. Oper. forsch. Stat. 9, 127-139 (1978).
59. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. B 58, 267-288 (1996).
60. Tibshirani, R.: The lasso method for variable selection in the Cox model. Stat. Med. 16, 385-395 (1997).
61. Tutz, G., Groll, A.: Generalized linear mixed models based on boosting. In: Kneib, T., Tutz, G. (eds.) Statistical Modelling and Regression Structures-Festschrift in the Honour of Ludwig Fahrmeir. Physica, Heidelberg (2010).
62. Tutz, G., Groll, A.: Likelihood-based boosting in binary and ordinal random effects models. J. Comput. Graph. Stat. (2012). doi: 10. 1080/10618600. 2012. 694769.
63. Tutz, G., Reithinger, F.: A boosting approach to flexible semiparametric mixed models. Stat. Med. 26, 2872-2900 (2007).
64. Venables, W. N., Ripley, B. D.: Modern Applied Statistics with S, 4th edn. Springer, New York (2002).
65. Vonesh, E. F.: A note on the use of Laplace's approximation for nonlinear mixed-effects models. Biometrika 83, 447-452 (1996).
66. Wang, D., Eskridge, K. M., Crossa, J.: Identifying QTLs and epistasis in structured plant populations using adaptive mixed lasso. J. Agric. Biol. Environ. Stat. 16, 170-184 (2010a).
67. Wang, S., Song, P. X., Zhu, J.: Doubly regularized REML for estimation and selection of fixed and random effects in linear mixed-effects models. Technical Report 89, The University of Michigan, (2010b).
68. Wolfinger, R. W.: Laplace's approximation for nonlinear mixed models. Biometrika 80, 791-795 (1994).
69. Wolfinger, R., O'Connell, M.: Generalized linear mixed models; a pseudolikelihood approach. J. Stat. Comput. Simul. 48, 233-243 (1993).
70. Wood, S. N.: Generalized Additive Models: An Introduction with R. Chapman & Hall, London (2006).
71. Yang, H.: Variable selection procedures for generalized linear mixed models in longitudinal data analysis. PhD thesis, North Carolina State University (2007).
72. Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. B 68, 49-67 (2006).
73. Zhao, P., Rocha, G., Yu, B.: The composite absolute penalties family for grouped and hierarchical variable selection. Ann. Stat. 37, 3468-3497 (2009).
74. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. B 67, 301-320 (2005).
75. Zou, H., Hastie, T.: The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 101, 1418-1429 (2006).