Analyzing average and conditional effects with multigroup multilevel structural equation models

Frontiers in Psychology

Volumn 5, Issue APR, 2014, Pages

  Mayer, Axel ^a,b   Nagengast, Benjamin ^c   Fletcher, John ^d   Steyer, Rolf ^a  

^a FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

^b GHENT UNIVERSITY   (Belgium)

^c UNIVERSITY OF TÜBINGEN   (Germany)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Average effects; Conditional effects; Multilevel analysis of covariance; Multilevel structural equation modeling; Quasi experimental designs

Indexed keywords

EID: 84899680091     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.00304     Document Type: Article

References (79)

1. Aiken, L. S., and West, S. G. (1996). Multiple Regression: Testing and Interpreting Interactions. Thousand Oaks, CA: Sage.
2. Baumert, J., Becker, M., Neumann, M., and Nikolova, R. (2009). Frühübergang in ein grundständiges Gymnasium-übergang in ein privilegiertes Entwicklungsmilieu? [ Early transition into the academic track of secondary schooling-transition into a privileged learning environment?]. Z. Erzieh. 12, 189-215. doi: 10.1007/s11618-009-0072-4.
3. Becker, M., Neumann, M., Tetzner, J., Böse, S., Knoppick, H., Maaz, K., et al. (2014). Is early ability grouping good for high-achieving students' psychosocial development? Effects of the transition into academically selective schools. J. Educ. Psychol. doi: 10.1037/a0035425.
4. Carroll, R. J., Ruppert, D., Stefanski, L. A., and Crainiceanu, C. M. (2010). Measurement Error in Nonlinear Models: A Modern Perspective. Boca Raton, FL: Chapman and Hall/CRC.
5. Chen, X. (2006). The adjustment of random baseline measurements in treatment effect estimation. J. Stat. Plan. Infer. 136, 4161-4175. doi: 10.1016/j.jspi.2005.08.046.
6. Cook, T. D., Steiner, P. M., and Pohl, S. (2009). How bias reduction is affected by covariate choice, unreliability, and mode of data analysis: results from two types of within-study comparisons. Multivariate Behav. Res. 44, 828-847. doi: 10.1080/00273170903333673.
7. Crager, M. R. (1987). Analysis of covariance in parallel-group clinical trials with pretreatment baselines. Biometrics 43, 895-901. doi: 10.2307/2531543.
8. Croon, M. A., and van Veldhoven, M. J. P. M. (2007). Predicting group-level outcome variables from variables measured at the individual level: a latent variable multilevel model. Psychol. Methods 12, 45-57. doi: 10.1037/1082-989X.12.1.45.
9. Culpepper, S. A., and Aguinis, H. (2011). Using analysis of covariance (ANCOVA) with fallible covariates. Psychol. Methods 16, 166-178. doi: 10.1037/a0023355.
10. Dawid, A. P. (2011). Fundamentals of Statistical Causality. Research report, University of Sheffield.
11. Degracie, J. S., and Fuller, W. A. (1972). Estimation of the slope and analysis of covariance when the concomitant variable is measured with error. J. Am. Stat. Assoc. 67, 930-937. doi: 10.1080/01621459.1972.10481321.
12. Enders, C. K., and Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychol. Methods 12, 121-138. doi: 10.1037/1082-989X.12.2.121.
13. Gatsonis, C., and Sampson, A. R. (1989). Multiple correlation: exact power and sample size calculations. Psychol. Bull. 106, 516-524. doi: 10.1037/0033-2909.106.3.516.
14. Gitelman, A. I. (2005). Estimating causal effects from multilevel group-allocation data. J. Educ. Behav. Stat. 30, 397-412. doi: 10.3102/10769986030004397.
15. Grilli, L., and Rampichini, C. (2011). The role of sample cluster means in multilevel models: a view on endogeneity and measurement error issues. Methodology 7, 121-133. doi: 10.1027/1614-2241/a000030.
16. Harker, R., and Tymms, P. (2004). The effects of student composition on school outcomes. School Eff. School Improv. 15, 177-199. doi: 10.1076/sesi.15.2.177.30432.
17. Hong, G., and Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: a case study of causal inferrence for multilevel observational data. J. Am. Stat. Assoc. 101, 901-910. doi: 10.1198/016214506000000447.
18. Hong, G., and Raudenbush, S. W. (2008). Causal inference for time-varying instructional treatments. J. Educ. Behav. Stat. 33, 333-362. doi: 10.3102/1076998607307355.
19. Hutchison, D. (2007). When is a compositional effect not a compositional effect? Qual. Quan. 41, 219-232. doi: 10.1007/s11135-007-9094-2.
20. Jak, S., Oort, F. J., and Dolan, C. V. (2013). A test for cluster bias: detecting violations of measurement invariance across clusters in multilevel data. Struct. Eq. Model. Multidisc. J. 20, 265-282. doi: 10.1080/10705511.2013.769392.
21. Kreft, I. G., de Leeuw, J., and Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behav. Res. 30, 1-21. doi: 10.1207/s15327906mbr3001_1.
22. Kröhne, U. (2009). Estimation of Average Causal Effects in Quasi-Experimental Designs: Non-Linear Constraints in Structural Equation Models. Ph.D. thesis, Friedrich-Schiller-University Jena, Germany.
23. Lehmann, R. (2010). "Längeres gemeinsames Lernen. Erschwert oder erleichtert es den übergang in weiterführende Schularten?," in übergänge im Schulwesen, eds S. Lin-Klitzing, D.-S. Di Fuccia, and G. Müller-Frerich (Bad Heilbrunn: Klinkhardt), 127-139.
24. Lehmann, R., Gänsfuß, R., and Peek, R. (1999). Aspekte der Lernausgangslage und der Lernentwicklung von Schülerinnen und Schülern an Hamburger Schulen-Klassenstufe 7. Hamburg: Behörde für Schule, Jugend und Berufsbildung, Amt für Schule.
25. Lehmann, R. H., and Lenkeit, J. (2008). Element. Erhebung zum Lese-und Mathematikverständnis. Entwicklungen in den Jahrgangsstufen 4 bis 6 in Berlin: Abschlussbericht über die Untersuchungen 2003, 2004 und 2005 an Berliner Grundschulen und grundständigen Gymnasien. Berlin: Humboldt Universität zu Berlin.
26. Lehmann, R. H., Peek, R., and Gänsfuß, R. (1997). Aspekte der Lernausgangslage von Schülerinnen und Schülern der fünften Klassen an Hamburger Schulen. Bericht über die Untersuchung im September 1996. Hamburg: Behörde für Schule, Jugend und Berufsbildung, Amt für Schule.
27. Leite, W. L., and Zuo, Y. (2011). Modeling latent interactions at level 2 in multilevel structural equation models: an evaluation of mean-centered and residual-centered unconstrained approaches. Struct. Eq. Model. Multidisc. J. 18, 449-464. doi: 10.1080/10705511.2011.582400.
28. Lockwood, J. R., and McCaffrey, D. F. (2014). Correcting for test score measurement error in ANCOVA models for estimating treatment effects. J. Educ. Behav. Stat. 39, 22-52. doi: 10.3102/1076998613509405.
29. Lüdtke, O., Marsh, H. W., Robitzsch, A., and Trautwein, U. (2011). AŒ2 taxonomy of multilevel latent contextual models: accuracy-bias trade-offs in full and partial error correction models. Psychol. Methods 16, 444-467. doi: 10.1037/a0024376.
30. Lüdtke, O., Marsh, H. W., Robitzsch, A., Trautwein, U., Asparouhov, T., and Muthén, B. O. (2008). The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies. Psychol. Methods 13, 203-229. doi: 10.1037/a0012869.
31. Marsh, H. W., Lüdtke, O., Nagengast, B., Trautwein, U., Morin, A. J. S., Abduljabbar, A. S., et al. (2012). Classroom climate and contextual effects: conceptual and methodological issues in the evaluation of group-level effects. Educ. Psychol. 47, 106-124. doi: 10.1080/00461520.2012.670488.
32. Marsh, H. W., Lüdtke, O., Robitzsch, A., Trautwein, U., Asparouhov, T., Muthén, B., et al. (2009). Doubly-latent models of school contextual effects: integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behav. Res. 44, 764-802. doi: 10.1080/00273170903333665.
33. Mehta, P. D., and Neale, M. C. (2005). People are variables too: multilevel structural equation modeling. Psychol. Methods 10, 259-284. doi: 10.1080/00273170903333665.
34. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika 58, 525-543. doi: 10.1007/BF02294825.
35. Mullis, I. V. S., Kennedy, A. M., Martin, M. O., and Sainsbury, M. (2006). PIRLS 2006 Assessment Framework and Specifications. Boston, MA: TIMSS&PIRLS International Study Center, Lynch School of Education, Boston College.
36. Murray, D. M. (1998). Design and Analysis of Group-Randomized Trials. New York, NY: Oxford University Press.
37. Murray, D. M. (2001). Statistical models appropriate for designs often used in group-randomized trials. Stat. Med. 20, 1373-1385. doi: 10.1002/sim.675.
38. Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations. Psychometrika 54, 557-585. doi: 10.1007/BF02296397.
39. Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociol. Methods Res. 22, 378-398.
40. Muthén, B. O. (1998-2004). Mplus Technical Appendices. Los Angeles, CA: Muthén&Muthén.
41. Muthén, B. O. (2002). Beyond SEM: general latent variable modeling. Behaviormetrika 29, 81-117. doi: 10.2333/bhmk.29.81.
42. Muthén, B. O., Khoo, S.-T., and Gustafsson, J.-E. (1997). Multilevel Latent Variable Modeling in Multiple Populations. Technical report, Graduate School of Education&Information Studies, University of California, Los Angeles.
43. Muthén, L. K., and Muthén, B. O. (1998-2012). Mplus User's Guide, 7th Edn. Los Angeles, CA: Muthén&Muthén.
44. Nagengast, B. (2006). Standard Errors of ACE Estimates: Comparing Adjusted Group Means Against the Adjusted Grand Mean. A Simulation Study. Unpublished Diploma thesis, University of Jena, Jena, Thuringia, Germany.
45. Nagengast, B. (2009). Causal Inference in Multilevel Models. Ph.D. thesis, Friedrich-Schiller-Universität Jena, Thüringen, Germany.
46. Nagengast, B., Trautwein, U., Kelava, A., and Lüdtke, O. (2013). Synergistic effects of expectancy and value on homework engagement: the case for a within-person perspective. Multivariate Behav. Res. 48, 428-460. doi: 10.1080/00273171.2013.775060.
47. Pearl, J. (1993). "Aspects of graphical models connected with causality," in Proceedings of the 49th Session of the International Statistical Institute. Tome LV, Book1 (Florence, Italy).
48. Pearl, J. (2009). Causality: Models, Reasoning, and Inference, 2nd Edn. Cambridge, UK: Cambridge University Press. doi: 10.1017/CBO9780511803161.
49. Pinheiro, J. C., and Bates, D. M. (2000). Mixed-Effects Models in S and S-PLUS. New York, NY: Springer. doi: 10.1007/978-1-4419-0318-1.
50. Pituch, K. A. (2001). Using multilevel modeling in large-scale planned variation educational experiments: improving understanding of intervention effects. J. Exp. Educ. 69, 347-373. doi: 10.1080/00220970109599492.
51. Plewis, I., and Hurry, J. (1998). A multilevel perspective on the design and analysis of intervention studies. Educ. Res. Eval. 4, 13-26. doi: 10.1076/edre.4.1.13.13014.
52. Rabe-Hesketh, S., Skrondal, A., and Pickles, A. (2004). Generalized multilevel structural equation modeling. Psychometrika 69, 167-190. doi: 10.1007/BF02295939.
53. Rabe-Hesketh, S., Skrondal, A., and Zheng, X. (2012). "Generalized multilevel structural equation modeling," in Handbook of Structural Equation Modeling, ed R. Hoyle (New York, NY: Guilford Press), 512-531.
54. Raudenbush, S. W. (1993). A crossed random effects model for unbalanced data with applications in cross-sectional and longitudinal research. J. Educ. Behav. Stat. 18, 321-349. doi: 10.3102/10769986018004321.
55. Raudenbush, S. W., and Bryk, A. S., (eds.). (2002). Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd Edn. Thousand Oaks, CA: Sage.
56. Raudenbush, S. W., and Willms, J. D. (1995). The estimation of school effects. J. Educ. Behav. Stat. 20, 307-335. doi: 10.3102/10769986020004307.
57. Robins, J. M. (1999). Association, causation, and marginal structural models. Synthese 121, 151-179. doi: 10.1023/A:1005285815569.
58. Rogosa, D. (1980). Comparing nonparallel regression lines. Psychol. Bull. 88, 307-321. doi: 10.1037/0033-2909.88.2.307.
59. Rosenbaum, P. R. (2002). Covariance adjustment in randomized experiments and observational studies. Stat. Sci. 17, 286-304. doi: 10.1214/ss/1042727942.
60. Rosenbaum, P. R., and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41-55. doi: 10.1093/biomet/70.1.41.
61. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Edu. Psychol. 66, 688-701. doi: 10.1037/h0037350.
62. Sampson, A. R. (1974). A tale of two regressions. J. Am. Stat. Assoc. 69, 682-689. doi: 10.1080/01621459.1974.10480189.
63. Schafer, J. L., and Kang, J. D. Y. (2008). Average causal effects from observational studies: a practical guide and simulated example. Psychol. Methods 13, 279-313. doi: 10.1080/01621459.1974.10480189.
64. Schermelleh-Engel, K., Kerwer, M., and Klein, A. G. (2014). Evaluation of model fit in nonlinear multilevel structural equation modeling. Front. Psychol. 5:181. doi: 10.3389/fpsyg.2014.00181.
65. Schweig, J. (2013). Cross-level measurement invariance in school and classroom environment surveys. Educ. Eval. Policy Anal. doi: 10.3102/0162373713509880. [Epub ahead of print].
66. Seltzer, M. (2004). "The use of hierarchical models in analyzing data from experiments and quasi-experiments conducted in field settings," in The SAGE Handbook of Quantitative Methodology for the Social Sciences, ed D. Kaplan (Thousand Oaks, CA: Sage).
67. Senn, S., Graf, E., and Caputo, A. (2007). Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment or exposure. Stat. Med. 26, 5529-5544. doi: 10.1002/sim.3133.
68. Shadish, W. R., Cook, T. D., and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
69. Shieh, G. (2006). Exact interval estimation, power calculation, and sample size determination in normal correlational analysis. Psychometrika 71, 529-540. doi: 10.1007/s11336-04-1221-6.
70. Shin, Y., and Raudenbush, S. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. J. Educ. Behav. Stat. 35, 26-53. doi: 10.3102/1076998609345252.
71. Skrondal, A., and Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models. Boca Raton, FL: Chapman&Hall. doi: 10.1201/9780203489437.
72. Snijders, T. A., and Bosker, R. J. (2012). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London: Sage.
73. Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate?: causal inference in the face of interference. J. Am. Stat. Assoc. 101, 1398-1407. doi: 10.1198/016214506000000636.
74. Steiger, J. H., and Fouladi, R. T. (1992). R2: a computer program for interval estimation, power calculations, sample size estimation, and hypothesis testing in multiple regression. Behav. Res. Methods Instrum. Comput. 24, 580-582. doi: 10.3758/BF03203611.
75. Steyer, R., Gabler, S., von Davier, A. A., and Nachtigall, C. (2000). Causal regression models II: unconfoundedness and causal unbiasedness. Methods Psychol. Res. Online 5, 55-87. Available online at: http://www.dgps.de/fachgruppen/methoden/mpr-online/issue11/art4/steyerCRII.pdf.
76. Steyer, R., Mayer, A., and Fiege, C. (2014). "Causal inference on total, direct, and indirect effects," in Encyclopedia of Quality of Life and Well-Being Research, ed A. C. Michalos (Dodrecht, Netherlands: Springer), 606-631. doi: 10.1007/978-94-007-0753-5_295.
77. Steyer, R., and Partchev, I. (2008). EffectLite for Mplus: a Program for the Uni-and Multivariate Analysis of Unconditional, Conditional and Average Mean Differences Between Groups. [Computer Software and Manual]. Available online at: www.statlite.com (Retrieved May 5, 2008).
78. Ukoumunne, C. C., Gulliford, M. C., Chinn, S., Sterne, J. A. C., and Burney, P. G. J. (1999). Methods for evaluating area-wide and organisation based interventions in health and health care: a systematic review. Health Technol. Assess. 3, 1-99.
79. VanderWeele, T. J. (2008). Ignorability and stability assumptions in neighborhood effects research. Stat. Med. 27, 1934-1943. doi: 10.1002/sim.3139.