On the unnecessary ubiquity of hierarchical linear modeling

Psychological Methods

Volumn 22, Issue 1, 2017, Pages 114-140

  McNeish, Daniel ^a,b   Stapleton, Laura M ^a   Silverman, Rebecca D ^a  

^a UNIVERSITY OF MARYLAND   (United States)

^b UTRECHT UNIVERSITY   (Netherlands)

Author keywords

Cluster robust errors; Clustered data; GEE; HLM; Multilevel model

Indexed keywords

ERROR; HUMAN; HUMAN EXPERIMENT; PSYCHOLOGIST; PSYCHOLOGY; PSYCINFO; STATISTICAL MODEL; BEHAVIORAL SCIENCE; CLUSTER ANALYSIS; PSYCHOLOGICAL MODEL; SAMPLE SIZE;

BEHAVIORAL SCIENCES; CLUSTER ANALYSIS; HUMANS; LINEAR MODELS; MODELS, PSYCHOLOGICAL; MODELS, STATISTICAL; PSYCHOLOGY; SAMPLE SIZE;

EID: 85000936579     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000078     Document Type: Article

References (132)

1. Agerbo, E., Sterne, J. A., & Gunnell, D. J. (2007). Combining individual and ecological data to determine compositional and contextual socioeconomic risk factors for suicide. Social Science & Medicine, 64, 451-461. http://dx.doi.org/10.1016/j.socscimed.2006.08.043.
2. Agresti, A., Caffo, B., & Ohman-Strickland, P. (2004). Examples in which misspecification of a random effects distribution reduces efficiency, and possible remedies. Computational Statistics & Data Analysis, 47, 639-653. http://dx.doi.org/10.1016/j.csda.2003.12.009.
3. Allison, P. D. (2012, April). Handling missing data by maximum likelihood. Paper presented at the SAS Global Forum, Orlando, FL. Retrieved from http://www.statisticalhorizons.com/wp-content/uploads/ MissingDataByML.pdf.
4. Angrist, J. D., & Pischke, J. S. (2008). Mostly harmless econometrics: An empiricist's companion. Princeton, NJ: Princeton University Press.
5. Ballinger, G. A. (2004). Using generalized estimating equations for longitudinal data analysis. Organizational Research Methods, 7, 127-150. http://dx.doi.org/10.1177/1094428104263672.
6. Bates, D. (2009). Assessing the precision of estimates of variance components. Seewiesen, Austria: Presentation at the Max Planck Institute for Ornithology.
7. Bates, M. D., Castellano, K. E., Rabe-Hesketh, S., & Skrondal, A. (2014). Handling correlations between covariates and random slopes in multilevel models. Journal of Educational and Behavioral Statistics, 39, 524-549.
8. Bauer, D. J., & Sterba, S. K. (2011). Fitting multilevel models with ordinal outcomes: Performance of alternative specifications and methods of estimation. Psychological Methods, 16, 373-390. http://dx.doi.org/10 .1037/a0025813.
9. Begg, M. D., & Parides, M. K. (2003). Separation of individual-level and clusterlevel covariate effects in regression analysis of correlated data. Statistics in Medicine, 22, 2591-2602. http://dx.doi.org/10.1002/sim.1524.
10. Berkhof, J., & Kampen, J. K. (2004). Asymptotic effect of misspecification in the random part of the multilevel model. Journal of Educational and Behavioral Statistics, 29, 201-218. http://dx.doi.org/10.3102/ 10769986029002201.
11. Beunckens, C., Sotto, C., & Molenberghs, G. (2008). A simulation study comparing weighted estimating equations with multiple imputation based estimating equations for longitudinal binary data. Computational Statistics & Data Analysis, 52, 1533-1548. http://dx.doi.org/10.1016/j .csda.2007.04.020.
12. Breslow, N. E., & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association, 88, 9-25.
13. Burton, P., Gurrin, L., & Sly, P. (1998). Tutorial in biostatistics. Extending the simple linear regression model to account for correlated responses: An introduction to generalized estimating equations and multi-level mixed modeling. Statistics in Medicine, 17, 1261-1291. http://dx.doi.org/10.1002/ (SICI)1097-0258(19980615)17:111261:AID-SIM8463.0.CO;2-Z.
14. Cameron, A. C., Gelbach, J. B., & Miller, D. L. (2011). Robust inference with multiway clustering. Journal of Business & Economic Statistics, 29, 238-249. http://dx.doi.org/10.1198/jbes.2010.07136.
15. Cameron, A. C., & Miller, D. L. (2015). A practitioner's guide to clusterrobust inference. Journal of Human Resources, 50, 317-372.
16. Carlin, J. B., Wolfe, R., Brown, C. H., & Gelman, A. (2001). A case study on the choice, interpretation and checking of multilevel models for longitudinal binary outcomes. Biostatistics, 2, 397-416. http://dx.doi .org/10.1093/biostatistics/2.4.397.
17. Carpenter, J. R., Kenward, M. G., & Vansteelandt, S. (2006). A comparison of multiple imputation and doubly robust estimation for analyses with missing data. Journal of the Royal Statistical Society Series A, 169, 571-584. http://dx.doi.org/10.1111/j.1467-985X.2006.00407.x.
18. Chen, B., Yi, G., & Cook, R. J. (2010). Weighted generalized estimating functions for longitudinal response and covariate data that are missing at random. Journal of the American Statistical Association, 105, 336-353. http://dx.doi.org/10.1198/jasa.2010.tm08551.
19. Clarkson, D. B., & Zhan, Y. (2002). Using spherical-radial quadrature to fit generalized linear mixed effects models. Journal of Computational and Graphical Statistics, 11, 639-659. http://dx.doi.org/10.1198/ 106186002439.
20. Clayton, D., Spiegelhalter, D., Dunn, G., & Pickles, A. (1998). Analysis of longitudinal binary data from multiphase sampling. Journal of the Royal Statistical Society Series B. Methodological, 60, 71-87. http://dx.doi .org/10.1111/1467-9868.00109.
21. Dedrick, R. F., Ferron, J. M., Hess, M. R., Hogarty, K. Y., Kromrey, J. D., Lang, T. R. Lee, R. (2009). Multilevel modeling: A review of methodological issues and applications. Review of Educational Research, 79, 69-102. http://dx.doi.org/10.3102/0034654308325581.
22. Diaz, R. E. (2007). Comparison of PQL and Laplace 6 estimates of hierarchical linear models when comparing groups of small incident rates in cluster randomized trials. Computational Statistics & Data Analysis, 51, 2871-2888. http://dx.doi.org/10.1016/j.csda.2006.10.005.
23. Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
24. Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121-138. http://dx.doi.org/10.1037/1082-989X.12 .2.121.
25. Ferron, J., Dailey, R., & Yi, Q. (2002). Effects of misspecifying the first-level error structure in two-level models of change. Multivariate Behavioral Research, 37, 379-403. http://dx.doi.org/10.1207/ S15327906MBR3703_4.
26. Fitzmaurice, G. M. (1995). A caveat concerning independence estimating equations with multivariate binary data. Biometrics, 51, 309-317. http:// dx.doi.org/10.2307/2533336.
27. Fitzmaurice, G. M., Laird, N. M., & Rotnitzky, A. G. (1993). Regression models for discrete longitudinal responses. Statistical Science, 8, 284-299. http://dx.doi.org/10.1214/ss/1177010899.
28. Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2012). Applied longitudinal analysis. Hoboken, NJ: Wiley.
29. Gardiner, J. C., Luo, Z., & Roman, L. A. (2009). Fixed effects, random effects and GEE: What are the differences? Statistics in Medicine, 28, 221-239. http://dx.doi.org/10.1002/sim.3478.
30. Gebauer, J. E., Sedikides, C., & Neberich, W. (2012). Religiosity, social self-esteem, and psychological adjustment: On the cross-cultural specificity of the psychological benefits of religiosity. Psychological Science, 23, 158-160. http://dx.doi.org/10.1177/0956797611427045.
31. Ghisletta, P., & Spini, D. (2004). An introduction to generalized estimating equations and an application to assess selectivity effects in a longitudinal study on very old individuals. Journal of Educational and Behavioral Statistics, 29, 421-437. http://dx.doi.org/10.3102/10769986029004421.
32. Givens, G. H., & Hoeting, J. A. (2005). Computational statistics. New York: John Wiley & Sons.
33. Gosho, M. (2014). Criteria to select a working correlation structure for the generalized estimating equations method in SAS. Journal of Statistical Software, 57, 1-10. http://dx.doi.org/10.18637/jss.v057.c01.
34. Gosho, M., Hamada, C., & Yoshimura, I. (2011). Criterion for the selection of a working correlation structure in the generalized estimating equation approach for longitudinal balanced data. Communications in Statistics Theory and Methods, 40, 3839-3856. http://dx.doi.org/10.1080/ 03610926.2010.501938.
35. Gurka, M. J. (2006). Selecting the best linear mixed model under REML. The American Statistician, 60, 19-26. http://dx.doi.org/10.1198/ 000313006X90396.
36. Hanley, J. A., Negassa, A., Edwardes, M. D., & Forrester, J. E. (2003). Statistical analysis of correlated data using generalized estimating equations: An orientation. American Journal of Epidemiology, 157, 364-375. http://dx.doi.org/10.1093/aje/kwf215.
37. Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15, 234-249.
38. Hayes, A. F., & Cai, L. (2007). Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods, 39, 709-722. http://dx.doi .org/10.3758/BF03192961
39. Heeringa, S. G., West, B. T., & Berglund, P. A. (2010). Applied survey data analysis. Boca Raton, FL: Chapman and Hall/CRC. http://dx.doi .org/10.1201/9781420080674.
40. Hin, L. Y., Carey, V. J., & Wang, Y. G. (2007). Criteria for working-correlation structure selection in GEE. The American Statistician, 61, 360-364. http://dx.doi.org/10.1198/000313007X245122.
41. Hin, L. Y., & Wang, Y. G. (2009). Working-correlation-structure identification in generalized estimating equations. Statistics in Medicine, 28, 642-658.
42. Hoffmann, D. A., & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24, 623-641. http://dx.doi.org/10.1177/ 014920639802400504.
43. Horton, N. J., & Laird, N. M. (1999). Maximum likelihood analysis of generalized linear models with missing covariates. Statistical Methods in Medical Research, 8, 37-50. http://dx.doi.org/10.1191/ 096228099673120862.
44. Horton, N. J., & Laird, N. M. (2001). Maximum likelihood analysis of logistic regression models with incomplete covariate data and auxiliary information. Biometrics, 57, 34-42. http://dx.doi.org/10.1111/j.0006-341X.2001.00034.x.
45. Horton, N. J., & Lipsitz, S. R. (1999). Review of software to fit generalized estimating equation regression models. The American Statistician, 53, 160-169.
46. Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar, & M. Schader (Eds.), Classification, data analysis, and data highways (pp. 147-154). Berlin, Germany: Springer. http://dx.doi.org/ 10.1007/978-3-642-72087-1_17.
47. Hox, J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). Mahwah, NJ: Erlbaum.
48. Hubbard, A. E., Ahern, J., Fleischer, N. L., Van der Laan, M., Lippman, S. A., Jewell, N. Satariano, W. A. (2010). To GEE or not to GEE: Comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology, 21, 467-474. http://dx.doi.org/10.1097/EDE.0b013e3181caeb90.
49. Huber, P. J. (1967, June). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 221-233.
50. Huynh, M., Parker, J. D., Harper, S., Pamuk, E., & Schoendorf, K. C. (2005). Contextual effect of income inequality on birth outcomes. International Journal of Epidemiology, 34, 888-895. http://dx.doi.org/10 .1093/ije/dyi092.
51. Jacqmin-Gadda, H., Sibillot, S., Proust, C., Molina, J. M., & Thiébaut, R. (2007). Robustness of the linear mixed model to misspecified error distribution. Computational Statistics & Data Analysis, 51, 5142-5154. http://dx.doi.org/10.1016/j.csda.2006.05.021.
52. Job, V., Dweck, C. S., & Walton, G. M. (2010). Ego depletion-Is it all in your head? Implicit theories about willpower affect self-regulation. Psychological Science, 21, 1686-1693. http://dx.doi.org/10.1177/ 0956797610384745.
53. Joe, H. (2008). Accuracy of Laplace approximation for discrete response mixed models. Computational Statistics & Data Analysis, 52, 5066-5074. http://dx.doi.org/10.1016/j.csda.2008.05.002.
54. Kenward, M. G., & Molenberghs, G. (1998). Likelihood based frequentist inference when data are missing at random. Statistical Science, 13, 236-247. http://dx.doi.org/10.1214/ss/1028905886.
55. Keselman, H. J., Algina, J., Kowalchuk, R. K., & Wolfinger, R. D. (1998). A comparison of two approaches for selecting covariance structures in the analysis of repeated measurements. Communications in Statistics Simulation and Computation, 27, 591-604. http://dx.doi.org/10.1080/ 03610919808813497.
56. Kim, J. S., & Frees, E. W. (2007). Multilevel modeling with correlated effects. Psychometrika, 72, 505-533.
57. Kim, Y., Choi, Y. K., & Emery, S. (2013). Logistic regression with multiple random effects: A simulation study of estimation methods and statistical packages. The American Statistician, 67, 171-182. http://dx .doi.org/10.1080/00031305.2013.817357.
58. Kontos, S., Burchinal, M., Howes, C., Wisseh, S., & Galinsky, E. (2002). An eco-behavioral approach to examining the contextual effects of early childhood classrooms. Early Childhood Research Quarterly, 17, 239-258. http://dx.doi.org/10.1016/S0885-2006(02)00147-3.
59. Kovacevic, M. S., & Rai, S. N. (2003). A pseudo maximum likelihood approach to multilevel modelling of survey data. Communications in Statistics Theory and Methods, 32, 103-121. http://dx.doi.org/10.1081/ STA-120017802.
60. Kreft, I. G. G. (1996). Are multilevel techniques necessary? An overview, including simulation studies. Unpublished manuscript, California State University, Los Angeles, CA.
61. Kreft, I. G., de Leeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1-21. http://dx.doi.org/10.1207/s15327906mbr3001_1.
62. Laird, N. M., & Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963-974. http://dx.doi.org/10.2307/2529876.
63. LeBeau, B. (2013). Misspecification of the covariance matrix in the linear mixed model: A Monte Carlo simulation (Doctoral dissertation). University of Minnesota, St. Paul, MN.
64. Liang, K. Y., & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13-22. http://dx.doi.org/10 .1093/biomet/73.1.13.
65. Lin, T. I., & Lee, J. C. (2008). Estimation and prediction in linear mixed models with skew-normal random effects for longitudinal data. Statistics in Medicine, 27, 1490-1507.
66. Lipsitz, S. R., Ibrahim, J. G., & Zhao, L. P. (1999). A weighted estimating equation for missing covariate data with properties similar to maximum likelihood. Journal of the American Statistical Association, 94, 1147-1160. http://dx.doi.org/10.1080/01621459.1999.10473870.
67. Litière, S., Alonso, A., & Molenberghs, G. (2007). Type I and Type II error under random-effects misspecification in generalized linear mixed models. Biometrics, 63, 1038-1044. http://dx.doi.org/10.1111/j.1541-0420 .2007.00782.x.
68. Liu, L., & Yu, Z. (2008). A likelihood reformulation method in non-normal random effects models. Statistics in Medicine, 27, 3105-3124.
69. Lopez, R. B., Hofmann, W., Wagner, D. D., Kelley, W. M., & Heatherton, T. F. (2014). Neural predictors of giving in to temptation in daily life. Psychological Science, 25, 1337-1344.
70. Lu, B., Preisser, J. S., Qaqish, B. F., Suchindran, C., Bangdiwala, S. I., & Wolfson, M. (2007). A comparison of two bias-corrected covariance estimators for generalized estimating equations. Biometrics, 63, 935-941.
71. Lukočiene O., Varriale, R., & Vermunt, J. K. (2010). The simultaneous decision(s) about the number of lower-and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40, 247-283. http://dx.doi.org/10.1111/j.1467-9531.2010.01231.x.
72. Maas, C. J., & Hox, J. J. (2004). The influence of violations of assumptions on multilevel parameter estimates and their standard errors. Computational Statistics & Data Analysis, 46, 427-440. http://dx.doi.org/10 .1016/j.csda.2003.08.006.
73. Mancl, L. A., & DeRouen, T. A. (2001). A covariance estimator for GEE with improved small-sample properties. Biometrics, 57, 126-134. http:// dx.doi.org/10.1111/j.0006-341X.2001.00126.x.
74. Marschall, M. J. (2004). Citizen participation and the neighborhood context: A new look at the coproduction of local public goods. Political Research Quarterly, 57, 231-244. http://dx.doi.org/10.1177/ 106591290405700205.
75. McCullagh, P., & Nelder, J. A. (1989). Generalized linear models. London, UK: Chapman and Hall. http://dx.doi.org/10.1007/978-1-4899-3242-6.
76. McCulloch, C. E., & Searle, S. R. (2001). Generalized, linear, and mixed models. New York, NY: Wiley.
77. McNeish, D. M. (2014). Modeling sparsely clustered data: Design-based, model-based, and single-level methods. Psychological Methods, 19, 552-563. http://dx.doi.org/10.1037/met0000024.
78. McNeish, D. M., & Harring, J. R. (2015). Clustered data with small sample sizes: Comparing the performance of model-based and design-based approaches. Communications in Statistics. Simulation and Computation. Advance online publication. http://dx.doi.org/10.1080/03610918.2014.983648.
79. McNeish, D. M., & Stapleton, L. M. (2014). The effect of small sample size on two level model estimates: A review and illustration. Educational Psychology Review. Advance online publication. http://dx.doi.org/ 10.1007/s10648-014-9287-x.
80. McNeish, D. M., & Stapleton, L. M. (in press). Modeling clustered data with very few clusters. Multivariate Behavioral Research. http://dx.doi .org/10.1080/00273171.2016.1167008.
81. Molenberghs, G., & Verbeke, G. (2004). Meaningful statistical model formulations for repeated measures. Statistica Sinica, 14, 989-1020.
82. Molenberghs, G., & Verbeke, G. (2007). Likelihood ratio, score, and Wald tests in a constrained parameter space. The American Statistician, 61, 22-27. http://dx.doi.org/10.1198/000313007X171322.
83. Morel, J. G., Bokossa, M. C., & Neerchal, N. K. (2003). Small sample correction for the variance of GEE estimators. Biometrical Journal Biometrische Zeitschrift, 45, 395-409. http://dx.doi.org/10.1002/bimj .200390021.
84. Morrell, C. H. (1998). Likelihood ratio testing of variance components in the linear mixed-effects model using restricted maximum likelihood. Biometrics, 54, 1560-1568. http://dx.doi.org/10.2307/2533680.
85. Mroz, T. A., & Zayats, Y. V. (2008). Arbitrarily normalized coefficients, information sets, and false reports of "biases" in binary outcome models. The Review of Economics and Statistics, 90, 406-413. http://dx.doi.org/ 10.1162/rest.90.3.406.
86. Muthen, B. O., & Asparouhov, T. (2008). Growth mixture modeling: Analysis with non-Gaussian random effects. In G. Fitzmaurice, M. Davidian, G. Verbeke, & G. Molenberghs (Eds.), Longitudinal data analysis (pp. 143-165). Boca Raton, FL: Chapman & Hall/CRC.
87. Newton, S. D., & McCoach, D. B. (2015, April). Growth modeling: Akaike and Bayesian Information Criteria and sample size considerations. Paper presented at the annual meeting of the American Educational Research Association (AERA), SIG: Multilevel Modeling, Chicago, IL.
88. Nichols, A., and Schafer, M. (2007, July). Clustered standard errors in Stata, United Kingdom Stata Users' Group Meetings.
89. Pan, W. (2001). Akaike's information criterion in generalized estimating equations. Biometrics, 57, 120-125. http://dx.doi.org/10.1111/j.0006-341X.2001.00120.x.
90. Petersen, M. A. (2009). Estimating standard errors in finance panel data sets: Comparing approaches. Review of Financial Studies, 22, 435-480. http://dx.doi.org/10.1093/rfs/hhn053.
91. Petronis, K. R., & Anthony, J. C. (2003). A different kind of contextual effect: Geographical clustering of cocaine incidence in the USA. Journal of Epidemiology and Community Health, 57, 893-900. http://dx.doi.org/ 10.1136/jech.57.11.893.
92. Pfefferman, D., Skinner, C. J., Holmes, D. J., Goldstein, H., & Rasbash, J. (1998). Weighting for unequal selection probabilities in multilevel model. Journal of Royal Statistical Society, 60, 23-40.
93. Pinheiro, J. C., & Bates, D. M. (1995). Approximations to the loglikelihood function in the nonlinear mixed-effects model. Journal of Computational and Graphical Statistics, 4, 12-35.
94. Pinheiro, J. C., & Chao, E. C. (2006). Efficient Laplacian and adaptive Gaussian quadrature algorithms for multilevel generalized linear mixed models. Journal of Computational and Graphical Statistics, 15, 58-81. http://dx.doi.org/10.1198/106186006X96962.
95. Rabe-Hesketh, S., & Skrondal, A. (2006). Multilevel modelling of complex survey data. Journal of the Royal Statistical Society, 169, 805-827. http://dx.doi.org/10.1111/j.1467-985X.2006.00426.x.
96. Rao, J. N. K., Verret, F., & Hidiroglou, M. A. (2013). A weighted composite likelihood approach to inference for two-level models from survey data. Survey Methodology, 39, 263-282.
97. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Thousand Oaks, CA: Sage.
98. Raudenbush, S. W., Yang, M.-L., & Yosef, M. (2000). Maximum likelihood for generalized linear models with nested random effects via high-order, multivariate Laplace approximation. Journal of Computational and Graphical Statistics, 9, 141-157.
99. Recchia, A. (2010). R-squared measures for two-level hierarchical linear models using SAS. Journal of Statistical Software, 32, 1-9. http://dx.doi .org/10.18637/jss.v032.c02.
100. Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1995). Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. Journal of the American Statistical Association, 90, 106-121. http://dx.doi.org/10.1080/01621459.1995.10476493.
101. Rotnitzky, A., & Jewell, N. P. (1990). Hypothesis testing of regression parameters in semiparametric generalized linear models for cluster correlated data. Biometrika, 77, 485-497. http://dx.doi.org/10.1093/biomet/ 77.3.485.
102. Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581-592. http://dx.doi.org/10.1093/biomet/63.3.581.
103. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: Wiley. http://dx.doi.org/10.1002/9780470316696.
104. Ruiter, S., & Van Tubergen, F. (2009). Religious attendance in crossnational perspective: A multilevel analysis of 60.
105. countries. American Journal of Sociology, 115, 863-895. http://dx.doi.org/10.1086/603536.
106. Savalei, V. (2010). Expected versus observed information in SEM with incomplete normal and nonnormal data. Psychological Methods, 15, 352-367. http://dx.doi.org/10.1037/a0020143.
107. Savalei, V., & Kolenikov, S. (2008). Constrained versus unconstrained estimation in structural equation modeling. Psychological Methods, 13, 150-170. http://dx.doi.org/10.1037/1082-989X.13.2.150.
108. Scharfstein, D. O., Rotnitzky, A., & Robins, J. M. (1999). Adjusting for nonignorable drop-out using semiparametric nonresponse models. Journal of the American Statistical Association, 94, 1096-1120. http://dx .doi.org/10.1080/01621459.1999.10473862.
109. Sharpe, D. (2013). Why the resistance to statistical innovations? Bridging the communication gap. Psychological Methods, 18, 572-582. http://dx .doi.org/10.1037/a0034177.
110. Sherman, G. D., Haidt, J., & Clore, G. L. (2012). The faintest speck of dirt: Disgust enhances the detection of impurity. Psychological Science, 23, 1506-1514. http://dx.doi.org/10.1177/0956797612445318.
111. Shults, J., Ratcliffe, S. J., & Leonard, M. (2007). Improved generalized estimating equation analysis via xtqls for quasi-least squares in Stata. The Stata Journal, 7, 147-166.
112. Silberzahn, R., & Uhlmann, E. L. (2013). It pays to be Herr Kaiser: Germans with noble sounding surnames more often work as managers than as employees. Psychological Science. Advance online publication.
113. Snijders, T. A. B., & Bosker, R. J. (1994). Modeled variance in two-level models. Sociological Methods & Research, 22, 342-363. http://dx.doi .org/10.1177/0049124194022003004.
114. Snijders, T. A. B., & Bosker, R. J. (2011). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling (2nd ed.). London, UK: Sage. http://dx.doi.org/10.1007/978-3-642-04898-2_387.
115. Stiratelli, R., Laird, N., & Ware, J. H. (1984). Random-effects models for serial observations with binary response. Biometrics, 40, 961-971. http://dx.doi.org/10.2307/2531147.
116. Stram, D. O., & Lee, J. W. (1994). Variance components testing in the longitudinal mixed effects model. Biometrics, 50, 1171-1177. http://dx .doi.org/10.2307/2533455.
117. Twisk, J. W. (2004). Longitudinal data analysis. A comparison between generalized estimating equations and random coefficient analysis. European Journal of Epidemiology, 19, 769-776. http://dx.doi.org/10 .1023/B:EJEP.0000036572.00663.f2.
118. Verbeke, G., Fieuws, S., Molenberghs, G., & Davidian, M. (2014). The analysis of multivariate longitudinal data: A review. Statistical Methods in Medical Research, 23, 42-59. http://dx.doi.org/10.1177/ 0962280212445834.
119. Verbeke, G., & Lesaffre, E. (1997). The effect of misspecifying the random-effects distribution in linear mixed models for longitudinal data. Computational Statistics & Data Analysis, 23, 541-556. http://dx.doi .org/10.1016/S0167-9473(96)00047-3.
120. Verbeke, G., & Molenberghs, G. (2007). What can go wrong with the score test? The American Statistician, 61, 289-290. http://dx.doi.org/10.1198/ 000313007X243089.
121. Westgate, P. M. (2013). A bias correction for covariance estimators to improve inference with generalized estimating equations that use an unstructured correlation matrix. Statistics in Medicine, 32, 2850-2858. http://dx.doi.org/10.1002/sim.5709.
122. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817-838. http://dx.doi.org/10.2307/1912934.
123. White, H. (1984). Asymptotic theory for econometricians. San Diego, CA: Academic Press.
124. Whittaker, T. A., & Furlow, C. F. (2009). The comparison of model selection criteria when selecting among competing hierarchical linear models. Journal of Modern Applied Statistical Methods, 8, 173-193.
125. Wolfinger, R. (1993). Covariance structure selection in general mixed models. Communications in Statistics Simulation and Computation, 22, 1079-1106. http://dx.doi.org/10.1080/03610919308813143.
126. Wooldridge, J. (2003). Introductory econometrics: A modern approach. Boston, MA: Cengage.
127. Wu, S., Crespi, C. M., & Wong, W. K. (2012). Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials. Contemporary Clinical Trials, 33, 869-880. http://dx.doi.org/10.1016/j.cct.2012.05.004.
128. Zeger, S. L., & Liang, K. Y. (1986). Longitudinal data analysis for discrete and continuous outcomes. Biometrics, 42, 121-130. http://dx.doi.org/10 .2307/2531248.
129. Zeger, S. L., Liang, K. Y., & Albert, P. S. (1988). Models for longitudinal data: A generalized estimating equation approach. Biometrics, 44, 1049-1060. http://dx.doi.org/10.2307/2531734.
130. Zhou, X.-H., Perkins, A. J., & Hui, S. L. (1999). Comparisons of software packages for generalized linear multilevel models. The American Statistician, 53, 282-290.
131. Ziegler, J. C., Bertrand, D., Tóth, D., Csépe, V., Reis, A., Faísca, L., Blomert, L. (2010). Orthographic depth and its impact on universal predictors of reading: A cross-language investigation. Psychological Science, 21, 551-559. http://dx.doi.org/10.1177/0956797610363406
132. Zorn, C. J. (2001). Generalized estimating equation models for correlated data: A review with applications. American Journal of Political Science, 45, 470-490. http://dx.doi.org/10.2307/2669353