Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction

Multivariate Behavioral Research

Volumn 52, Issue 5, 2017, Pages 661-670

  McNeish, Daniel ^a,b  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b ARIZONA STATE UNIVERSITY   (United States)

Author keywords

explanation; Kenward Roger; mixed model; Restricted maximum likelihood; tutorial

Indexed keywords

DOCUMENTATION; HUMAN; MAXIMUM LIKELIHOOD METHOD; MULTILEVEL ANALYSIS; SAMPLING BIAS; SCIENTIST; SIMULATION; REPRODUCIBILITY; SAMPLE SIZE; STATISTICAL BIAS; STATISTICAL MODEL;

BIAS; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL; MULTILEVEL ANALYSIS; REPRODUCIBILITY OF RESULTS; SAMPLE SIZE;

EID: 85024407861     PISSN: 00273171     EISSN: None     Source Type: Journal    
DOI: 10.1080/00273171.2017.1344538     Document Type: Article

References (28)

1. Bell, A., & Jones, K. (2015). Explaining fixed effects: Random effects modeling of time-series cross-sectional and panel data. Political Science Research and Methods, 3, 133–153. doi:10.1017/psrm.2014.7
2. Bell, B. A., Morgan, G. B., Schoeneberger, J. A., Kromrey, J. D., & Ferron, J. M. (2014). How low can you go?. Methodology, 10, 1–11. doi:10.1027/1614-2241/a000062
3. Browne, W. J., & Draper, D. (2006). A comparison of Bayesian and likelihood-based methods for fitting multilevel models. Bayesian Analysis, 1, 473–514. doi:10.1214/06-BA117
4. Efron, B., & Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika, 65, 457–482. doi:10.1093/biomet/65.3.457
5. Enders, C. K. (2010). Applied missing data analysis. New York: Guilford Press.
6. Ferron, J. M., Bell, B. A., Hess, M. R., Rendina-Gobioff, G., & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372–384. doi:10.3758/BRM.41.2.372
7. Goldstein, H. (1986). Multilevel mixed linear model analysis using iterative generalized least squares. Biometrika, 73, 43–56. doi:10.1093/biomet/73.1.43
8. Harville, D. A., & Jeske, D. R. (1992). Mean squared error of estimation or prediction under a general linear model. Journal of the American Statistical Association, 87, 724–731. doi:10.1080/01621459.1992.10475274
9. Hox, J. J., van de Schoot, R., & Matthijsse, S. (2012, July). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6, 87–93.
10. Huang, F. L. (2017). Using cluster bootstrapping to analyze nested data with a few clusters. Educational and Psychological Measurement. Advance online publication. doi:10.1177/0013164416678980.
11. Kackar, R. N., & Harville, D. A. (1981). Unbiasedness of two-stage estimation and prediction procedures for mixed linear models. Communications in Statistics-Theory and Methods, 10, 1249–1261. doi:10.1080/03610928108828108
12. Kackar, R. N., & Harville, D. A. (1984). Approximations for standard errors of estimators of fixed and random effects in mixed linear models. Journal of the American Statistical Association, 79, 853–862. doi:10.2307/2288715
13. Kenward, M. G., & Roger, J. H. (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics and Data Analysis, 53, 2583–2595. doi:10.1016/j.csda.2008.12.013
14. Kenward, M. G., & Roger, J. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997. doi:10.2307/2533558
15. Keselman, H. J., Algina, J., Kowalchuk, R. K., & Wolfinger, R. D. (1999). The analysis of repeated measurements: A comparison of mixed-model Satterthwaite F tests and a nonpooled adjusted degrees of freedom multivariate test. Communications in Statistics-Theory and Methods, 28, 2967–2999. doi:10.1080/03610929908832460
16. Maas, C. J., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 86–92. doi:10.1027/1614-2241.1.3.86
17. McNeish, D. (2016). On using Bayesian methods to address small sample problems. Structural Equation Modeling, 23, 750–773. doi:10.1080/10705511.2016.1186549
18. McNeish, D., & Stapleton, L. M. (2016a). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51, 495–518. doi:10.1080/00273171.2016.1167008
19. McNeish, D., & Stapleton, L. M. (2016b). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28, 295–314. doi:10.1007/s10648-014-9287-x
20. Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychological Methods, 17, 313–335. doi:10.1037/a0026802
21. Prasad, N. G. N., & Rao, J. N. (1990). The estimation of the mean squared error of small-area estimators. Journal of the American Statistical Association, 85, 163–171. doi:10.1080/01621459.1990.10475320
22. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
23. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110–114. doi:10.2307/3002019
24. Schaalje, G. B., McBride, J. B., & Fellingham, G. W. (2002). Adequacy of approximations to distributions of test statistics in complex mixed linear models. Journal of Agricultural, Biological, and Environmental Statistics, 7, 512–524. doi:10.1198/108571102726
25. Snijders, T. A., & Bosker, R. J. (1993). Standard errors and sample sizes for two-level research. Journal of Educational Statistics, 18, 237–259. doi:10.2307/1165134
26. Stapleton, L. M., Pituch, K. A., & Dion, E. (2015). Standardized effect size measures for mediation analysis in cluster-randomized trials. The Journal of Experimental Education, 83, 547–582. doi:10.1080/00220973.2014.919569
27. van de Schoot, R., Broere, J. J., Perryck, K. H., Zondervan-Zwijnenburg, M., & van Loey, N. E. (2015). Analyzing small data sets using Bayesian estimation: the case of posttraumatic stress symptoms following mechanical ventilation in burn survivors. European Journal of Psychotraumatology, 6. doi:10.3402/ejpt.v6.25216
28. van de Schoot, R., Kaplan, D., Denissen, J., Asendorpf, J. B., Neyer, F. J., & Aken, M. A. (2014). A gentle introduction to Bayesian analysis: Applications to developmental research. Child Development, 85, 842–860. doi:10.1111/cdev.12169