A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data

Journal of Clinical Epidemiology

Volumn 68, Issue 12, 2015, Pages 1406-1414

  Wynants, L ^a   Bouwmeester, W ^b,c   Moons, K G M ^b   Moerbeek, M ^d   Timmerman, D ^a,e   Van Huffel, S ^a   Van Calster, B ^a,f   Vergouwe, Y ^f  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^c PHARMERIT INTERNATIONAL   (Netherlands)

^d UTRECHT UNIVERSITY   (Netherlands)

^e UNIVERSITY HOSPITALS LEUVEN   (Belgium)

^f ERASMUS MEDICAL CENTER   (Netherlands)

Author keywords

Clustered data; Events per variable; Logistic model; Multicenter study; Prediction model; Simulation study

Indexed keywords

ARTICLE; CALIBRATION; CORRELATION COEFFICIENT; HUMAN; OVARY CANCER; PRACTICE GUIDELINE; PREDICTION; PREDICTIVE VALUE; PRIORITY JOURNAL; SAMPLE SIZE; SELECTION BIAS; SIMULATION; STATISTICAL ANALYSIS; CLINICAL TRIAL; CLUSTER ANALYSIS; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; FEMALE; INFORMATION PROCESSING; MULTICENTER STUDY; OVARIAN NEOPLASMS; REGRESSION ANALYSIS; STATISTICAL BIAS; STATISTICAL MODEL; STATISTICS; STATISTICS AND NUMERICAL DATA; VALIDATION STUDY;

BIAS (EPIDEMIOLOGY); CLUSTER ANALYSIS; COMPUTER SIMULATION; DATA COLLECTION; DECISION SUPPORT TECHNIQUES; FEMALE; HUMANS; LOGISTIC MODELS; MODELS, STATISTICAL; OVARIAN NEOPLASMS; REGRESSION ANALYSIS; SAMPLE SIZE; STATISTICS AS TOPIC;

EID: 84948711435     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2015.02.002     Document Type: Article

References (26)

1. E.W. Steyerberg Clinical prediction models: a practical approach to development, validation, and updating 2009 Springer US New York, NY
2. P. Peduzzi, J. Concato, E. Kemper, T.R. Holford, and A.R. Feinstein A simulation study of the number of events per variable in logistic regression analysis J Clin Epidemiol 49 1996 1373 1379
3. F.E. Harrell, K.L. Lee, and D.B. Mark Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors Stat Med 15 1996 361 387
4. E.W. Steyerberg, M.J. Eijkemans, and J.D. Habbema Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis J Clin Epidemiol 52 1999 935 942
5. D.S. Courvoisier, C. Combescure, T. Agoritsas, A. Gayet-Ageron, and T.V. Perneger Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure J Clin Epidemiol 64 2011 993 1000
6. E. Vittinghoff, and C.E. McCulloch Relaxing the rule of ten events per variable in logistic and Cox regression Am J Epidemiol 165 2007 710 718
7. S. Sprague, J.M. Matta, M. Bhandari, D. Dodgin, C.R. Clark, P. Kregor, and et al. Multicenter collaboration in observational research: improving generalizability and efficiency J Bone Joint Surg Am 91 Suppl 3 2009 80 86
8. S. Senn Some controversies in planning and analysing multi-centre trials Stat Med 17 1998 1753 1765 discussion 1799-1800
9. A.R. Localio, J.A. Berlin, T.R. Ten Have, and S.E. Kimmel Adjustments for center in multicenter studies: an overview Ann Intern Med 135 2001 112 123
10. W. Bouwmeester, J. Twisk, T. Kappen, W. Klei, K. Moons, and Y. Vergouwe Prediction models for clustered data: comparison of a random intercept and standard regression model BMC Med Res Methodol 13 2013 19
11. T.P.A. Debray, K.G. Moons, I. Ahmed, H. Koffijberg, and R.D. Riley A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis Stat Med 32 2013 3158 3180
12. T.A.B. Snijders, and R.J. Bosker T.A.B. Snijders, R.J. Bosker, Multilevel analysis: an introduction to basic and advanced multilevel modeling 2nd ed 2012 Sage London
13. C.J.M. Maas, and J.J. Hox Sufficient sample sizes for multilevel modeling Methodol Eur J Res Methods Behav Soc Sci 1 3 2005 86 92
14. R. Moineddin, F.I. Matheson, and R.H. Glazier A simulation study of sample size for multilevel logistic regression models BMC Med Res Methodol 7 2007 34
15. O. Paccagnella Sample size and accuracy of estimates in multilevel models Methodol Eur J Res Methods Behav Soc Sci 7 3 2011 111 120
16. G. Adams, M.C. Gulliford, O.C. Ukoumunne, S. Eldridge, S. Chinn, and M.J. Campbell Patterns of intra-cluster correlation from primary care research to inform study design and analysis J Clin Epidemiol 57 2004 785 794
17. A. Skrondal, and S. Rabe-Hesketh Prediction in multilevel generalized linear models J R Stat Soc Ser A Stat Soc 172 3 2009 659 687
18. F.E. Harrell F.E. Harrell Jr. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis 2001 Springer New York, NY
19. D.R. Cox Two further applications of a model for binary regression Biometrika 45 1958 562 565
20. R. Van Oirbeek, and E. Lesaffre Assessing the predictive ability of a multilevel binary regression model Comput Stat Data Anal 56 6 2012 1966 1980
21. R Development Core Team R: a language and environment for statistical computing 2011 R Foundation for Statistical Computing Vienna, Austria [October 7, 2014]. Available at http://www.R-project.org Accessed March 4, 2015
22. D. Bates, M. Maechler, and B. Bolker lme4: linear mixed-effects models using S4 classes. R package version 0.999375-42 2011 [October 7, 2014]. Available at http://CRAN.R-project.org/package=lme4 Accessed March 4, 2015
23. E.W. Steyerberg, M.J. Eijkemans, F.E. Harrell Jr., and J.D. Habbema Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets Stat Med 19 2000 1059 1079
24. D. Timmerman, L. Valentin, T.H. Bourne, W.P. Collins, H. Verrelst, and I. Vergote Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) group Ultrasound Obstetrics Gynecol 16 2000 500 505
25. J. Kaijser, T. Bourne, L. Valentin, A. Sayasneh, C. Van Holsbeke, I. Vergote, and et al. Improving strategies for diagnosing ovarian cancer: a summary of the International Ovarian Tumor Analysis (IOTA) studies Ultrasound Obstet Gynecol 41 2013 9
26. D. van Klaveren, E. Steyerberg, P. Perel, and Y. Vergouwe Assessing discriminative ability of risk models in clustered data BMC Med Res Methodol 14 2014 5