Three-level models for indirect effects in school- and class-randomized experiments in education

Journal of Experimental Education

Volumn 78, Issue 1, 2009, Pages 60-95

  Pituch, Keenan A ^a   Murphy, Daniel L ^a   Tate, Richard L ^b  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Experimental design; Indirect effect; Mediation; Moderation; Multi level model

Indexed keywords

EID: 73549114253     PISSN: 00220973     EISSN: None     Source Type: Journal    
DOI: 10.1080/00220970903224685     Document Type: Article

References (44)

1. Bauer, D. J., Preacher, K. J., & Gil, K. M. (2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142-163.
2. Boruch, R. F. (2002). The virtues of randomness. Stanford, CA: Hoover Institution. Retrieved October 8, 2006, from http://www.educationnext.org/20023/ 36.html
3. Boruch, R. F. (Ed.). (2005). Place randomized trials: Experimental test of public policy. The Annals of the American Academy of Political and Social Science, 599.
4. Chatterji, M. (2004). Evidence on "what works": An argument for extended-term mixed-methods (ETMM) evaluation designs. Educational Researcher, 33, 3-13.
5. Cook, T.D.,&Payne,M. R. (2002).Objections to objections to using random assignment in educational research. In R. F. Boruch & F. Mosteller (Eds.), Evidence Matters: Randomized Trials in Education Research (pp. 150-178). Washington DC: Brookings Institution Press.
6. Eisenhart,M.,&Towne, L. (2003). Contestation and change in national policy on "scientifically-based" education research. Educational Researcher, 32, 31-38.
7. Enders, C. K.,&Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevelmodels: A new look at an old issue. Psychological Methods, 12, 121-138.
8. Finn, J. D., & Achilles, C.M. (1990). Answers and questions about class size: A statewide experiment. American Educational Research Journal, 27, 557-577.
9. Institute of Education Sciences. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Institute of Education Sciences. Retrieved October 8, 2006, from http://www.excelgov.org/evidence.
10. Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602-619.
11. Kellam, S. G., & Langevin, D. J. (2003). A framework for understanding "evidence" in prevention research and programs. Prevention Science, 4, 137-153.
12. Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. T. Gilbert, S. T. Fiske, G. Lindzey (Eds.), The Handbook of Social Psychology (4th ed., pp. 233-265). New York: McGraw-Hill.
13. Kenny, D. A., Korchmaros, J. D., & Bolger, N. (2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115-128.
14. Krull, J. L., & MacKinnon, D. P. (1999). Multilevel mediation modeling in group-based intervention studies. Evaluation Review, 23, 144-158.
15. Krull, J. L., & MacKinnon, D. P. (2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249-277.
16. MacKinnon, D. P.,&Dwyer, J.H. (1993). Estimatingmediated effects in prevention studies. Evaluation Review, 17, 144-158.
17. MacKinnon, D. P., Fritz, M. S., Williams, J., & Lockwood, C. M. (2007). Distribution of the product confidence limits for the indirect effect: Program PRODCLIN. Behavior Research Methods, 39, 384-389.
18. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M.,West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83-104.
19. MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128.
20. MacKinnon, D. P., Lockwood, C. M., Brown, C. H., & Hoffman, J. M. (2007). The intermediate endpoint effect in logistic and probit regression. Clinical Trials, 4, 499-513.
21. Moerbeek, M. (2004). The consequence of ignoring a level of nesting in multilevel analysis. Multivariate Behavioral Research, 39, 129-149.
22. Moerbeek,M., vanBreukelen,G. J. P.,&Berger, M. (2000).Design issues for experiments inmultilevel populations. Journal of Educational and Behavioral Statistics, 25, 271-284.
23. Mosteller, F. (1995). The Tennesse study of class size in the early grades. The Future of Children: Critical Issues for Children and Youth, 5, 113-125.
24. Olson, D. R. (2004). The triumph of hope over experience in the search for "WhatWorks": A response to Slavin. Educational Researcher, 33(1), 24-26.
25. Pituch, K. A. (2004). Textbook presentations on supplemental hypothesis testing activities, nonnormality, and the concept of mediation. Understanding Statistics, 3, 135-150.
26. Pituch, K. A., & Stapleton, L. M. (2008). The performance of methods to test upper-level mediation in the presence of nonnormal data. Multivariate Behavioral Research, 43, 237-267.
27. Pituch, K. A., Stapleton, L. M., & Kang, J. Y. (2006). A comparison of single sample and bootstrap methods to assess mediation in cluster-randomized trials. Multivariate Behavioral Research, 41, 367-400.
28. Pituch, K. A., Whittaker, T. A., & Stapleton, L. M. (2005). A comparison of methods to test for mediation in multisite experiments. Multivariate Behavioral Research, 40, 1-23.
29. Plewis, I., & Hurry, J. (1998). A multilevel perspective on the design and analysis of intervention studies. Educational Research and Evaluation, 4, 13-26.
30. Preacher, K. J., Rucker, D. D., & Hayes, A. F. (2007). Addressing moderated mediation hypotheses: Theory, methods, and prescriptions. Multivariate Behavioral Research, 42, 185-227.
31. Rao, C. R. (1965). Linear Statistical Inference and Its Applications. New York: Wiley.
32. Rasbash, J., Charlton, C., Browne, W. J., Healy, M., & Cameron, B. (2005). MLwiN Version 2.02. Bristol, UK: Centre for Multilevel Modelling, University of Bristol.
33. Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster-randomized trials. Psychological Methods, 2, 173-185.
34. Raudenbush, S. W. (2003, November). Designing field trials of educational innovations. Paper presented at the conference Conceptualizing Scale-UP: Multidisciplinary Perspectives, Washington, DC.
35. Raudenbush, S. W., Bryk, A. S., Cheong, Y. K., Congdon Jr., R. T., & du Toit, M. (2004). HLM6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International.
36. Raudenbush, S. W., & Liu, X. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods, 5, 199-213.
37. Raudenbush, S. W., Liu, X.-F., Spybrook, J., Martinez, A., & Congdon, R. (2006). Optimal Design software for multi-level and longitudinal research (version 1.77) [computer software]. Available at http://sitemaker.umich.edu/ group-based
38. Raudenbush, S.W., Martinez, A., & Spybrook, J. (2007). Strategies for improving precision in grouprandomized experiments. Educational Evaluation and Policy Analysis, 29, 5-29.
39. Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15-21.
40. Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural equation models. In S. Leinhardt (Ed.), Sociological Methodology 1982 (pp. 290-312). Washington, DC: American Sociological Association.
41. Tate, R. L. (2004). Interpreting hierarchical linear and hierarchical generalized linear with models with slopes as outcomes. The Journal of Experimental Education, 73, 71-95.
42. Tate, R. L., & Pituch, K. A. (2007). Multivariate hierarchical linear modeling in randomized field experiments. Journal of Experimental Education, 75, 317-337.
43. Van Landeghem, G., De Fraine, B., & Van Damme, J. (2005). The consequence of ignoring a level of nesting in multilevel analysis: A comment. Multivariate Behavioral Research, 40, 423-434.
44. What Works Clearinghouse (2005). WWC study review standards. Institute of Education Sciences. Retrieved October 8, 2006, from http://w-w-c.org/