Alternatives to Multilevel Modeling for the Analysis of Clustered Data

Journal of Experimental Education

Volumn 84, Issue 1, 2016, Pages 175-196

  Huang, Francis L ^a  

^a UNIVERSITY OF MISSOURI   (United States)

Author keywords

Clustered data; DEFF adjusted standard errors; fixed effects model; multilevel modeling

Indexed keywords

EID: 84947036889     PISSN: 00220973     EISSN: 19400683     Source Type: Journal    
DOI: 10.1080/00220973.2014.952397     Document Type: Article

References (64)

1. Allison, P. D. (2005). Fixed effects regression methods for longitudinal data using SAS. Cary, NC: SAS Institute.
2. Allison, P. D. (2009). Fixed effects regression models. Thousand Oaks, CA: Sage.
3. American Institutes for Research, & Cohen, J. (2005, June). AM. Retrieved from http://www. am. air. org
4. Arceneaux, K., & Nickerson, D. (2009). Modeling certainty with clustered data: A comparison of methods. Political Analysis, 17, 177-190. doi:10. 10983/pan/mpp004
5. Astin, A. W., & Denson, N. (2009). Multi-campus studies of college impact: Which statistical method is appropriate? Research in Higher Education, 50, 354-367.
6. Bell, B., Morgan, G., Schoeneberger, J., Loudermilk, B., Kromrey, J., & Ferron, J. (2010, April). Dancing the sample size limbo with mixed models: How low can you go? Paper 197-2010 presented at the SAS Global Forum, Seattle, WA.
7. Bell, R., & McCaffrey, D. (2002). Bias reduction in standard errors for linear regression with multi-stage samples. Survey Methodology, 2, 169-181.
8. Bell-Ellison, B., & Kromrey, J. (2007, April). Alternatives for analysis of complex sample surveys: A comparison of SAS, SUDAAN, and AM. Paper presented at the first annual SAS Global Forum, Orlando, FL.
9. Bliese, P. D., & Hanges, P. J. (2004). Being both too liberal and too conservative: The perils of treating grouped data as though they were independent. Organizational Research Methods, 7, 400-417. doi:10. 1177/1094428104268542
10. Chaplin, D. (2003, December). Hierarchical linear models: Strengths and weaknesses. Paper presented at the annual meeting of the Association for Public Policy Analysis and Management, Washington, DC.
11. Claessens, A. (2012). Kindergarten child care experiences and child achievement and socioemotional skills. Early Childhood Research Quarterly, 27, 365-375. doi:10. 1016/j. ecresq. 2011. 12. 005
12. Clarke, P. (2008). When can group level clustering be ignored? Multilevel models versus single-level models with sparse data. Journal of Epidemiology and Community Health, 62, 752-758. doi:10. 1136/jech. 2007. 060798
13. Clarke, P., Crawford, C., Steele, F., & Vignoles, A. (2010). The choice between fixed and random effects models: Some considerations for educational research. IZA Discussion Paper Series 5287. Bonn, Germany: Institute for the Study of Labor.
14. Cohen, J., Cohen, P., West, S., & Aiken, L. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
15. Coyne, M. D., McCoach, D. B., Loftus, S., Zipoli, R., Ruby, M., Crevecoeur, Y. C., Kapp, S. (2010). Direct and extended vocabulary instruction in Kindergarten: Investigating transfer effects. Journal of Research on Educationet alffectiveness, 3, 93-120. doi:10. 1080/19345741003592410
16. Dee, T., & Jacob, B. (2012). Rational ignorance in education: A field experiment in student plagiarism. Journal of Human Resources, 47, 397-434.
17. Donner, A. (1998). Some aspects of the design and analysis of cluster randomization trials. Journal of the Royal Statistical Society: Series C, 47, 95-113.
18. Draper, D. (1995). Inference and hierarchical modeling in the social sciences. Journal of Educational and Behavioral Statistics, 20, 115-147.
19. Fan, X. (2001). Parental involvement and students' academic achievement: A growth modeling analysis. The Journal of Experimentet alducation, 70, 27-61.
20. Galbraith, S., Daniel, J. A., & Vissel, B. (2010). A study of clustered data and approaches to its analysis. The Journal of Neuroscience, 30, 10601-10608.
21. Green, D., & Vavreck, L. (2008). Analysis of cluster-randomized experiments: A comparison of alternative estimation approaches. Political Analysis, 138-152. doi:10. 1093/pan/mpm025
22. Guo, G., & Zhao, H. (2000). Multilevel modeling for binary data. Annual Review of Sociology, 26, 441-462.
23. Hahs-Vaughn, D. L. (2005). A primer for using and understanding weights with national datasets. The Journal of Experimentet alducation, 73, 221-248.
24. Hahs-Vaughn, D. L., McWayne, C. M., Bulotsky-Shearer, R. J., Wen, X., & Faria, A. M. (2011). Methodological considerations in using complex survey data. Evaluation Review, 35, 269-303.
25. Harden, J. J. (2011). A bootstrap method for conducting statistical inference with clustered data. State Politics & Policy Quarterly, 11, 223-246. doi:10. 1177/1532440011406233
26. Hedges, L. V., & Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educationet alvaluation and Policy Analysis, 29, 60-87.
27. Hofmann, D. A., & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24, 623-641.
28. Hosmer, D., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York, NY: Wiley.
29. Hox, J. J. (1998). Multilevel modeling: When and why. In I. Balderjahn, R. Mathar, & M. Schader (Eds.), Classification, data analysis, and data highways (pp. 147-154). New York, NY: Springer Verlag.
30. Kalton, G. (1983). Models in the practice of survey sampling. International Statistical Review/Revue Internationale de Statistique, 51, 175-188.
31. Kenny, D. A., & Judd, C. M. (1986). Consequences of violating the independence assumption in analysis of variance. Psychological Bulletin, 99, 422-431.
32. Kish, L. (1965). Survey sampling. New York, NY:Wiley.
33. Kish, L., & Frankel, M. R. (1974). Inference from complex samples. Journal of the Royal Statistical Society. Series B, 36, 1-37.
34. Kneipp, S. M., & Yarandi, H. N. (2002). Complex sampling designs and statistical issues in secondary analysis. Western Journal of Nursing Research, 24, 552-566.
35. Kreft, I. G., & De Leeuw, J. (1998). Introducing multilevel modeling. Newbury Park, CA: Sage.
36. Laplante, B., & Hébert, B. P. (2001). An introduction to the use of linear models with correlated data. Canadian Studies in Population, 28, 287-311.
37. Lee, E., Forthofer, R., & Lorimor, R. (1989). Analyzing complex survey data. Newbury Park, CA: Sage.
38. Leventhal, T., & Brooks-Gunn, J. (2000). The neighborhoods they live in: The effects of neighborhood residence on child and adolescent outcomes. Psychological Bulletin, 126, 309-337.
39. Luke, D. (2004). Multilevel modeling. Thousand Oaks, CA: Sage.
40. Lumley, T. (2004). Analysis of complex survey samples. Journal of Statistical Software, 9, 1-19.
41. Maas, C. J., & Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1, 86-92.
42. McCoach, D. B., & Adelson, J. L. (2010). Dealing with dependence (part I): Understanding the effects of clustered data. Gifted Child Quarterly, 54, 152-155.
43. McDermott, J. J., & Schukken, Y. H. (1994). A review of methods used to adjust for cluster effects in explanatory epidemiological studies of animal populations. Preventive Veterinary Medicine, 18, 155-173.
44. McNeish, D., & Stapleton, L. (2013, April). The effect of small number of clusters on two-level model estimates: A review and illustration. Paper presented at the annual American Educational Research Association Conference, San Francisco, CA.
45. Meijer, E., Busing, F. M. T. A., & Van der Leeden, R. (1998). Estimating bootstrap confidence intervals for two-level models. In J. J. Hox & E. D. De Leeuw (Eds.), Assumptions, robustness, and estimation methods in multivariate modeling (pp. 35-47). Amsterdam, Netherlands: TT Publicaties.
46. Möhring, K. (2012, September). The fixed effects approach as alternative to multilevel models for cross-national analyses. Paper presented at the 10th ESPAnet Conference, Edinburgh, Scotland.
47. Mukhopadhyay, P., An, A., Tobias, R., &Watts, D. (2008, March). Try, try again: Replication-based variance estimation methods for survey data analysis in SAS. Paper 367-2008 presented at the SAS Global Forum, San Antonio, TX.
48. Mundfrom, D., & Schultz, M. (2002). A Monte Carlo simulation comparing parameter estimates from multiple linear regression and hierarchical linear modeling. Multiple Linear Regression Viewpoints, 28, 18-21.
49. Murnane, R., & Willet, J. (2011). Methods matter. New York, NY: Oxford University Press.
50. Musca, S. C., Kamiejski, R., Nugier, A., Méot, A., Er-Rafiy, A., & Brauer, M. (2011). Data with hierarchical structure: Impact of intraclass correlation and sample size on Type-I error. Frontiers in Psychology, 2, 1-6. doi:10. 3389/fpsyg. 2011. 00074
51. Muthén, B., & Satorra, A. (1995). Complex sample data in structuret alquation modeling. Sociological Methodology, 25, 216-316.
52. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structuret alquation Modeling, 9, 599-620. doi:10. 1207/S15328007SEM0904 8
53. Newman, D., Newman, I., & Salzman, J. (2010). Comparing OLS and HLM models and the questions they answer: Potential concerns for Type VI errors. Multiple Linear Regression Viewpoints, 36, 1-8.
54. Primo, D. M., Jacobsmeier, M. L., & Milyo, J. (2007). Estimating the impact of state policies and institutions with mixed-level data. State Politics & Policy Quarterly, 7, 446-459.
55. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). London, UK: Sage.
56. Research Triangle Institute. (2010). SUDAAN language manual release 10. 0. Research Triangle Park, NC: Author.
57. Siller, A. B., & Tompkins, L. (2006, March). The Big Four: Analyzing complex sample survey data using SAS R, SPSS R, STATA R, and SUDAAN R. Paper presented at the 31st Annual SAS R Users Group International Conference, San Francisco, CA.
58. Silverstein, M., Augustyn, M., Cabral, H., & Zuckerman, B. (2006). Maternal depression and violence exposure: Double jeopardy for child school functioning. Pediatrics, 118, 792-800. doi:10. 1542/peds. 2005-1841
59. Snijders, T., & Bosker, R. (2012). Multilevel analysis (2nd ed.). Thousand Oaks, CA: Sage.
60. Stapleton, L. M. (2008). Variance estimation using replicationmethods in structuret alquationmodelingwith complex sample data. Structuret alquationModeling: AMultidisciplinary Journal, 15, 183-210. doi:10. 1080/10705510801922316
61. Sterba, S. K. (2009). Alternative model-based and design-based frameworks for inference from samples to populations: From polarization to integration. Multivariate Behavioral Research, 44, 711-740.
62. Steenbergen, M., & Jones, B. (2002). Modeling multilevel data structures. American Journal of Political Science, 46, 218-237.
63. Thomas, S., & Heck, R. (2001). Analysis of large-scale secondary data in higher education research: Potential perils associated with complex sampling designs. Research in Higher Education, 42, 517-540.
64. Thomas, S., Heck, R., & Bauer, K. (2005). Weighting and adjusting for design effects in secondary data analyses. New Directions for Institutional Research, 127, 51-72.