Separation in Logistic Regression: Causes, Consequences, and Control

American Journal of Epidemiology

Volumn 187, Issue 4, 2018, Pages 864-870

  Mansournia, Mohammad Ali ^a   Geroldinger, Angelika ^b   Greenland, Sander ^c,d   Heinze, Georg ^b  

^a TEHRAN UNIVERSITY OF MEDICAL SCIENCES   (Iran)

^b MEDICAL UNIVERSITY OF VIENNA   (Austria)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

logistic regression; maximum likelihood; penalized likelihood; separation; small samples; sparse data

Indexed keywords

BAYESIAN ANALYSIS; DATA SET; MAXIMUM LIKELIHOOD ANALYSIS; NUMERICAL METHOD; NUMERICAL MODEL; REGRESSION ANALYSIS; URINARY TRACT DISEASE;

ARTICLE; BAYES THEOREM; CAUSAL MODELING; CONTRACEPTIVE BEHAVIOR; HUMAN; LOGISTIC REGRESSION ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MAXIMUM LIKELIHOOD METHOD; SEPARATION TECHNIQUE; SOFTWARE; STATISTICAL MODEL; URINARY TRACT INFECTION; EPIDEMIOLOGY; SAMPLE SIZE; STATISTICAL ANALYSIS;

DATA INTERPRETATION, STATISTICAL; EPIDEMIOLOGIC RESEARCH DESIGN; HUMANS; LOGISTIC MODELS; SAMPLE SIZE;

EID: 85045447736     PISSN: 00029262     EISSN: 14766256     Source Type: Journal    
DOI: 10.1093/aje/kwx299     Document Type: Article

References (42)

1. Hirji KF, Tsiatis AA, Mehta CR. Median unbiased estimation for binary data. Am Stat. 1989;43(1):7-11.
2. Firth D. Bias reduction of maximum likelihood estimates. Biometrika. 1993;80(1):27-38.
3. Schaefer RL. Bias correction in maximum likelihood logistic regression. Stat Med. 1983;2(1):71-78.
4. Bull SB, Greenwood CM, Hauck WW. Jackknife bias reduction for polychotomous logistic regression. Stat Med. 1997;16(5):545-560.
5. Cordeiro GM, Cribari-Neto F. On bias reduction in exponential and non-exponential family regression models. Commun Stat Simul Comput. 1998;27(2):485-500.
6. Heinze G. A comparative investigation of methods for logistic regression with separated or nearly separated data. Stat Med. 2006;25(24):4216-4226.
7. Albert A, Anderson JA. On the existence of maximum likelihood estimates in logistic regression models. Biometrika. 1984;71(1):1-10.
8. Allison P. Convergence problems in logistic regression. In: Altman M, Gill G, McDonald MP, eds. Numerical Issues in Statistical Computing for the Social Scientist. Hoboken, NJ: John Wiley & Sons; 2004:238-252.
9. Foxman B, Marsh J, Gillespie B, et al. Condom use and first time urinary tract infection. Epidemiology. 1997;8(6):637-641.
10. Salama M, Andrukhova O, Hoda MA, et al. Concomitant endothelin-1 overexpression in lung transplant donors and recipients predicts primary graft dysfunction. Am J Transplant. 2010;10(3):628-636.
11. Potter DM. A permutation test for inference in logistic regression with small-and moderate-sized data sets. Stat Med. 2005;24(5):693-708.
12. Cole SR, Chu HT, Greenland S. Maximum likelihood, profile likelihood, and penalized likelihood: a primer. Am J Epidemiol. 2014;179(2):252-260.
13. Vaeth M. On the use of Wald's test in exponential-families. Int Stat Rev. 1985;53(2):199-214.
14. Greenland S, Mansournia MA, Altman DG. Sparse data bias: a problem hiding in plain sight. BMJ. 2016;352:i1981.
15. Agresti A. Categorical Data Analysis. 3rd ed. Hoboken, NJ: John Wiley & Sons; 2013.
16. Heinze G, Schemper M. A solution to the problem of separation in logistic regression. Stat Med. 2002;21(16): 2409-2419.
17. Heinze G, Puhr R. Bias-reduced and separation-proof conditional logistic regression with small or sparse data sets. Stat Med. 2010;29(7-8):770-777.
18. Lesaffre E, Lawson AB. Bayesian Biostatistics. Chichester, UK: John Wiley & Sons; 2012.
19. Kosmidis I, Firth D. Bias reduction in exponential family nonlinear models. Biometrika. 2009;96(4):793-804.
20. Rainey C, McCaskey K. Estimating Logit Models with Small Samples. carlislerainey. com/papers/small. pdf. Accessed July 7, 2017.
21. Puhr R, Heinze G, Nold M, et al. Firth's logistic regression with rare events: accurate effect estimates and predictions? Stat Med. 2017;36(14):2302-2317.
22. Rahman MS, Sultana M. Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data. BMC Med Res Methodol. 2017;17(1):33.
23. Heinze G. Comment on "Bias reduction in conditional logistic regression". Stat Med. 2011;30(12):1466-1467.
24. Heinze G. Comment on "A comparative study of the bias corrected estimates in logistic regression". Stat Methods Med Res. 2012;21(6):660-661.
25. Gelman A, Jakulin A, Pittau MG, et al. A weakly informative default prior distribution for logistic and other regression models. Ann Appl Stat. 2008;2(4):1360-1383.
26. Greenland S. Generalized conjugate priors for Bayesian analysis of risk and survival regressions. Biometrics. 2003;59(1):92-99.
27. Greenland S. Bayesian perspectives for epidemiological research. II. Regression analysis. Int J Epidemiol. 2007;36(1):195-202.
28. Greenland S, Mansournia MA. Penalization, bias reduction, and default priors in logistic and related categorical and survival regressions. Stat Med. 2015;34(23):3133-3143.
29. Sullivan SG, Greenland S. Bayesian regression in SAS software. Int J Epidemiol. 2013;42(1):308-317.
30. Discacciati A, Orsini N, Greenland S. Approximate Bayesian logistic regression via penalized likelihood by data augmentation. Stata J. 2015;15(3):712-736.
31. Le Cessie S, Van Houwelingen J. Ridge estimators in logistic regression. J R Stat Soc Ser C Appl Stat. 1992;41(1):191-201.
32. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol. 1996;58(1):267-288.
33. Steyerberg E. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York, NY: Springer-Verlag; 2008.
34. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1-22.
35. Greenland S, Schwartzbaum JA, Finkle WD. Problems due to small samples and sparse data in conditional logistic regression analysis. Am J Epidemiol. 2000;151(5):531-539.
36. Harrell F. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed. New York, NY: Springer-Verlag; 2015.
37. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning. 2nd ed. New York, NY: Springer-Verlag; 2009.
38. Shmueli G. To explain or to predict? Stat Sci. 2010;25(3): 289-310.
39. Greenland S, Maclure M, Schlesselman JJ, et al. Standardized regression-coefficients-a further critique and review of some alternatives. Epidemiology 1991;2(5):387-392.
40. Greenland S, Schlesselman JJ, Criqui MH. The fallacy of employing standardized regression coefficients and correlations as measures of effect. Am J Epidemiol. 1986; 123(2):203-208.
41. Heinze G, Schemper L. A solution to the problem of monotone likelihood in Cox regression. Biometrics. 2001;57(1):114-119.
42. Santos Silva JMC, Tenreyro S. Poisson: some convergence issues. Stata J. 2011;11(2):207-212.