A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods

Frontiers in Psychology

Volumn 5, Issue APR, 2014, Pages

  Koch, Tobias ^a   Schultze, Martin ^a   Eid, Michael ^a   Geiser, Christian ^b  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

^b UTAH STATE UNIVERSITY   (United States)

Author keywords

Longitudinal modeling; Method effects; MTMM modeling; Multilevel structural equation modeling; Multirater data; Rater bias; Simulation study

Indexed keywords

EID: 84899697394     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2014.00311     Document Type: Article

References (96)

1. Bandalos, D. L. (2002). The effects of item parceling on goodness-of-fit and parameter estimate bias in structural equation modeling. Struct. Equ. Model. 9, 78-102. doi: 10.1207/S15328007SEM0901_5.
2. Bandalos, D. L. (2006). "The use of monte carlo studies in structural equation modeling research," in Structural Equation Modeling: A Second Course, eds G. R. Hancock and R. O. Mueller (Greenwich, CT: Information Age Publishing), 385-426.
3. Bentler, P., and Chou, C.-P. (1987). Practical issues in structural modeling. Sociol. Methods Res. 17, 78-117. doi: 10.1177/0049124187016001004.
4. Boker, S., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., et al. (2011). OpenMx: an open source extended structural equation modeling framework. Psychometrika 76, 306-317. doi: 10.1007/s11336-010-9200-6.
5. Bollen, K. A. (1989). Structural equations with latent variables. Wiley series in probability and mathematical statistics. Applied probability and statistics section. New York, NY: John Wiley&Sons, pp 0271-6356.
6. Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annu. Rev. Psychol. 53, 605-634. doi: 10.1146/annurev.psych.53.100901.135239.
7. Bollen, K. A., and Curran, P. J. (2006). Latent Curve Models: A Structural Equation Perspective. New York, NY: Wiley.
8. Boomsma, A. (1982). The robustness of LISREL against small sample sizes in factor analysis models., pages 149-173. Amsterdam: North-Holland.
9. Burns, G. L., and Haynes, S. N. (2006). Clinical Psychology: Construct Validation with Multiple Sources of Information and Multiple Settings (Washington, DC: APA Publications), 401-418.
10. Byrne, B. M., Shavelson, R. J., and Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychol. Bull. 105, 456-466. doi: 10.1037/0033-2909.105.3.456.
11. Campbell, D. T., and Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 56, 81-105. doi: 10.1037/h0046016.
12. Carretero-Dios, H., Eid, M., and Ruch, W. (2011). Analyzing multitrait-mulitmethod data with multilevel confirmatory factor analysis: an application to the validation of the State-Trait Cheerfulness Inventory. J. Res. Pers. 45, 153-164. doi: 10.1016/j.jrp.2010.12.007.
13. Courvoisier, D. S., Nussbeck, F. W., Eid, M., Geiser, C., and Cole, D. A. (2008). Analyzing the convergent and discriminant validity of states and traits: development and applications of multimethod latent state-trait models. Psychol. Assess. 20, 270-280. doi: 10.1037/a0012812.
14. Crayen, C. (2008). Eine Monte-Carlo Simulationsstudie zur Untersuchung der Anwendbarkeit von Strukturgleichungsmodellen für Multitrait-Multimethod-Multioccasion Daten. [A simulation study for investigating the applicability of structural equation models for multitrait-multimethod-multioccasion data], (Unpublished diploma thesis.) Freie Universität Berlin, Berlin: Germany.
15. Dumenci, L. (2000). Multitrait-Multimethod Analysis. San Diego, CA: Academic Press, 583-611.
16. Duncan, S. C., Duncan, T. E., and Strycker, L. A. (2006). Alcohol use from ages 9 to 16: a cohort-sequential latent growth model. Drug Alcohol Depend. 81, 71-81. doi: 10.1016/j.drugalcdep.2005.06.001.
17. Eid, M. (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika 65, 241-261. doi: 10.1007/BF02294377.
18. Eid, M., and Diener, E. (2006). Introduction: The Need for Multimethod Measurement in Psychology. Wahington, DC: American Psychological Association, 3-8.
19. Eid, M., and Hoffmann, L. (1998). Measuring variability and change with an item response model for polytomous variables. J. Educ. Behav. Stat. 23, 193-215. doi: 10.2307/1165244.
20. Eid, M., Lischetzke, T., and Nussbeck, F. W. (2006). "Structural equation models for multitrait-multimethod data," in Handbook of Multimethod Measurement in Psychology, eds M. Eid and E. Diener (Washington, DC: American Psychological Association), 283-299.
21. Eid, M., Lischetzke, T., Nussbeck, F. W., and Trierweiler, L. I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: a multiple-indicator CT-C(M-1) model. Psychol. Methods 8, 38-60. doi: 10.1037/1082-989X.8.1.38.
22. Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., and Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: different models for different types of methods. Psychol. Methods 13, 230-253. doi: 10.1037/a0013219.
23. Enders, C. K., and Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Struct. Equa. Model. 8, 430-457. doi: 10.1207/S15328007SEM0803_5.
24. Erhardt, E. (2013). Corcounts: Generate Correlated Count Random Variables. R package version 1.4 [online]. Available online at: http://cran.r-project.org/web/packages/corcounts/index.html (Accessed March, 2014).
25. Fan, X., Thompson, B., and Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Struct. Equ. Model. Multidisc. J. 6, 56-83. doi: 10.1080/10705519909540119.
26. Geiser, C. (2009). Multitrait-Multimethod-Multioccasion Modeling. München, Germany: Akademischer Verlag München (AVM).
27. Geiser, C., Eid, M., Nussbeck, F., Courvoisier, D. S., and Cole, D. A. (2010). Multitrait-multimethod change modelling. Adv. Stat. Anal. 94, 185-201. doi: 10.1007/s10182-010-0127-0.
28. Geiser, C., Eid, M., and Nussbeck, F. W. (2008). On the meaning of the latent variables in the CT-C(M-1) model: a comment on Maydeu-Olivares and Coffman (2006). Psychol. Methods 13, 49-57. doi: 10.1037/1082-989X.13.1.49.
29. Geiser, C., Keller, B. T., Lockhart, G., Eid, M., Cole, D. A., and Koch, T. (2014). Distinguishing state variability from trait change in longitudinal data: the role of measurement (non) invariance in latent state-trait analyses. Behav. Res. Methods, 1-32. doi: 10.3758/s13428-014-0457-z. Available online at: http://link.springer.com/article/10.3758/s13428-014-0457-z.
30. Geiser, C., Koch, T., and Eid, M. (in press). Data-generating mechanisms versus constructively-defined latent variables in multitrait-multimethod analyses: a comment on Castro-Schilo, Widaman, and Grimm (2013). Struct. Equ. Modeling.
31. Gerbing, D. W., and Anderson, J. C. (1985). The effects of sampling error and model characteristics on parameter estimation for maximum likelihood confirmatory factor analysis. Multivariate Behav. Res. 20, 255-271. doi: 10.1207/s15327906mbr2003_2.
32. Grimm, K. J., Pianta, R. C., and Konold, T. (2009). Longitudinal multitrait-multimethod models for developmental research. Multivariate Behav. Res. 44, 233-258. doi: 10.1080/00273170902794230.
33. Hallquist, M. (2011). MplusAutomation: Automating Mplus Model Estimation and Interpretation. R package version 0.5 [online]. Available online at: http://cran.r-project.org/web/packages/MplusAutomation/index.html (Accessed March, 2014).
34. Hancock, G. R., Kuo, W.-L., and Lawrence, F. R. (2001). An illustration of second-order latent growth models. Struct. Equ. Model. 8, 470-489. doi: 10.1207/S15328007SEM0803_7.
35. Heck, R. H., Thomas, S. L., and Tabata, L. N. (2013). Multilevel and Longitudinal Modeling with IBM Spss. New York, NY: Routledge.
36. Hertzog, C., and Nesselroade, J. R. (1987). Beyond autoregressive models: some implications of the trait-state distinction for the structural modeling of developmental change. Child Dev. 58, 93-109. doi: 10.2307/1130294.
37. Hox, J. J. (2010). Multilevel Analysis Techniques and Applications, 2nd Edn. New York, NY: Routledge.
38. Hox, J. J., and Maas, C. J. (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Struct. Equ. Model. 8, 157-174. doi: 10.1207/S15328007SEM0802_1.
39. Hox, J., van de Schoot, R., and Matthijsse, S. (2012). How few countries will do? Comparative survey analysis from a bayesian perspective. Surv. Res. Methods 6, 87-93. Available online at: http://mplus.fss.uu.nl/files/2012/07/hox-schoot-2012-how-few-countries-will-do.pdf.
40. Jackson, D. L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: a monte carlo investigation. Struct. Equ. Model. 8, 205-223. doi: 10.1207/S15328007SEM0802_3.
41. Jöreskog, K. G. (1979a). Statistical Models and Methods for Analysis of Longitudinal Data. Cambridge, MA: Abt Books, 129-169.
42. Jöreskog, K. G. (1979b). "Statistical estimation of structural models in longitudinal-developmental investigations," in Longitudinal Research in the Study of Behavior and Development, eds J. R. Nesselroade and P. B. Baltes (New York, NY: Academic Press), 303-351.
43. Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Struct. Equ. Model. 8, 325-352. doi: 10.1207/S15328007SEM0803_1.
44. Kenny, D. A., and Kashy, D. A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychol. Bull. 112, 165-172. doi: 10.1037/0033-2909.112.1.165.
45. Kenny, D. A., and Zautra, A. (2001). "Trait-state models for longitudinal data," in New Methods for the Analysis of Change. Decade of behavior, eds L. M. Collins and A. G. Sayer (Washington, DC: American Psychological Association), 243-263. doi: 10.1037/10409-008.
46. Khoo, S.-T., West, S. G., Wu, W., and Kwok, O.-M. (2006). "Longitudinal methods." in Handbook of Psychological Measurement: A Multimethod Perspective, eds M. Eid and E. Diener (Washington, DC: American Psychological Association), 301-317.
47. Koch, T. (2013). Multilevel Structural Equation Modeling of Multitraitmultimethod-Multioccasion Data, Unpublished doctoral thesis, Freie Universität Berlin, Berlin. Available online at: http://www.diss.fu-berlin.de/diss/receive/FUDISS_thesis_000000094645. (Accessed March, 2014).
48. Koch, T, Eid, M., and Lochner, K. (in press). "Multitrait-Multimethod-Analysis: the psychometric foundation of CFA-MTMM models," in The Wiley Handbook of Psychometric Testing. (London: John Wiley&Sons).
49. Little, T. D., Schnabel, K. U., and Baumert, J. (2000). Modeling Longitudinal and Multilevel Data. Mahwah, NJ: Lawrence Erlbaum Associates.
50. Maas, C. J. M., and Hox, J. J. (2005). Sufficient sample sizes for multilevel modeling. Methodology 1, 86-92. doi: 10.1027/1614-2241.1.3.86.
51. MacKinnon, D. P., Warsi, G., and Dwyer, J. H. (1995). A simulation study of mediated effect measures. Multivariate Behav. Res. 30, 41-62. doi: 10.1207/s15327906mbr3001_3.
52. Marsh, H. W. (1989). Confirmatory factor analyses of multitrait-multimethod data: Many problems and a few solutions. Appl. Psychol. Measur. 13, 335-361. doi: 10.1177/014662168901300402.
53. Marsh, H. W. (1993). Multitrait-multimethod analyses: inferring each trait-method combination with multiple indicators. Appl. Measur. Edu. 6, 49-81. doi: 10.1207/s15324818ame0601_4.
54. Marsh, H. W., and Grayson, D. (1994). Longitudinal confirmatory factor analysis: common, time-specific, item-specific, and residual-error components of variance. Struct. Equ. Model. 1, 116-145. doi: 10.1080/10705519409539968.
55. Marsh, H. W., and Grayson, D. (1995). Latent Variable Models of Multitrait-Multimethod Data. Thousand Oaks: Sage Publications, Inc, 177-198.
56. Marsh, H. W., Hau, K.-T., Balla, J. R., and Grayson, D. (1998). Is more ever too much? The number of indicators per factor in confirmatory factor analysis. Multivariate Behav. Res. 33, 181-220. doi: 10.1207/s15327906mbr3302_1.
57. Marsh, H. W., and Hocevar, D. (1988). A new, more powerful approach to multitrait-multimethod analyses: application of second-order confirmatory factor analysis. J. Appl. Psychol. 73, 107-117. doi: 10.1037/0021-9010.73.1.107.
58. McArdle, J. J. (1988). "Dynamic but structural equation modeling of repeated measures data," in Handbook of Multivariate Experimental Psychology, eds R. B. Cattell and J. Nesselroade (New York, NY: Plenum Press), 561-614.
59. McArdle, J. J., and Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Dev. 58, 110-133. doi: 10.2307/1130295.
60. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika 58, 525-543. doi: 10.1007/BF02294825.
61. Meredith, W., and Tisak, J. (1990). Latent curve analysis. Psychometrika 55, 107-122. doi: 10.1007/BF02294746.
62. Meuleman, B., and Billiet, J. (2009). A monte carlo sample size study: how many countries are needed for accurate multilevel sem? Surv. Res. Methods 3, 45-58. Available online at: http://hdl.handle.net/1854/LU-1041001.
63. Millsap, R. E. (2012). Statistical Approaches to Measurement Invariance. New York, NY: Routledge.
64. Muthén, L. K., and Muthén, B. O. (1998-2010). Mplus User's Guide, 6th Edn Los Angeles, CA: Muthén&Muthén.
65. Nussbeck, F. W., Eid, M., and Lischetzke, T. (2006). Analysing multitrait-multimethod data with structural equation models for ordinal variables applying the WLSMV estimator: what sample size is needed for valid results? Br. J. Math. Stat. Psychol. 59, 195-213. doi: 10.1348/000711005X67490.
66. Pham, G., Koch, T., Helmke, A., Schrader, F.-W., Helmke, T., and Eid, M. (2012). Do teachers know how their teaching is perceived by their pupils? Procedia Soc. Behav. 46, 3368-3374. doi: 10.1016/j.sbspro.2012.06.068.
67. Pohl, S., and Steyer, R. (2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behav. Res. 45, 45-72. doi: 10.1080/00273170903504729.
68. Pohl, S., Steyer, R., and Kraus, K. (2008). Modelling method effects as individual causal effects. J. R. Stat. Soc. Ser. A, 171, 41-63. doi: 10.1111/j.1467-985X.2007.00517.x.
69. Raykov, T. (2000). On sensitivity of structural equation modeling to latent relation misspecifications. Struct. Equ. Model. 7, 596-607. doi: 10.1207/S15328007SEM0704_4.
70. Rabe-Hesketh, S., and Skrondal, A. (2004). Generalized multilevel structural equation modeling. Psychometrica 69, 167-190. doi: 10.1007/BF02295939.
71. R Core Team. (2014). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing. Available online at: http://www.R-project.org/. (Accessed March, 2014).
72. Ryu, E. (2014). Model fit evaluation in multilevel structural equation models. Front. Psychol. 5:81. doi: 10.3389/fpsyg.2014.00081.
73. Ryu, E., and West, S. G. (2009). Level-specific evaluation of model fit in multilevel structural equation modeling. Struct. Equ. Model. 16, 583-601. doi: 10.1080/10705510903203466.
74. Satorra, A., and Muthen, B. (1995). Complex sample data in structural equation modeling. Sociol. Methodol. 25, 267-316. doi: 10.2307/271070.
75. Schermelleh-Engel, K., Kerwer, M., and Klein, A. G. (2014). Evaluation of model fit in nonlinear multilevel structural equation modeling. Front. Psychol. 5:181. doi: 10.3389/fpsyg.2014.00181.
76. Schultze, M. (2012). Evaluating What the Crowd Says. A Longitudinal Structural Equation Model for Exchangeable and Structurally Different Methods for Evaluating Interventions. Diploma thesis, Freie Universität Berlin.
77. Singer, J. D., and Willett, J. B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford: Oxford university press. doi: 10.1093/acprof:oso/9780195152968.001.0001.
78. Snijders, T. A. B., and Bosker, R. J. (2011). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London: Sage.
79. Stapleton, L. M. (2002). The incorporation of sample weights into multilevel structural equation models. Struct. Equ. Model. 9, 475-502. doi: 10.1207/S15328007SEM0904_2.
80. Steele, F. (2008). Multilevel models for longitudinal data. J. R. Stat. Soc. Ser. A 171, 5-19. doi: 10.1111/j.1467-985X.2007.00509.x.
81. Steyer, R. (1988). Experiment, Regression und Kausalität: Die Logische Struktur Kausaler Regressionsmodelle [Experiment, Regression and Causality: The Logical Structure of Causal Regression Models]. Unpublished thesis, Universität Trier.
82. Steyer, R. (1989). Models of classical psychometric test theory as stochastic measurement models: Representation, uniqueness, meaningfulness, identifiability, and testability. Methodika 3, 25-60.
83. Steyer, R. (1995). Das MTMM-Modell ist nicht identifiziert [The MTMM model is not identified]. Newsletter der Fachgruppe Methoden, 3, 5.
84. Steyer, R. (2005). Analyzing individual and average causal effects via structural equation models. Methodology 1:39. doi: 10.1027/1614-1881.1.1.39.
85. Steyer, R., and Eid, M. (2001). Messen und Testen. [Measurement and testing.], 2nd Edn. Heidelberg: Springer. doi: 10.1007/978-3-642-56924-1.
86. Steyer, R., Eid, M., and Schwenkmezger, P. (1997). Modeling true intraindividual change: true change as a latent variable. Methods Psychol. Res. 2, 21-33.
87. Steyer, R., Ferring, D., and Schmitt, M. J. (1992). States and traits in psychological assessment. Eur. J. Psychol. Assess. 8, 79-98.
88. Steyer, R., Partchev, I., and Shanahan, M. J. (2000). Modeling True Intraindividual Change in Structural Equation Models: The Case of Poverty and Children's Psychosocial Adjustment. Mahwah, NJ: Lawrence Erlbaum, 109-126.
89. Steyer, R., Schmitt, M., and Eid, M. (1999). Latent state-trait theory and research in personality and individual differences. Eur. J. Pers. 13, 389-408.
90. Suppes, P., and Zinnes, J. L. (1963). Basic Measurement Theory, Vol. 1. Wiley, New York, 1-76.
91. Tisak, J., and Tisak, M. S. (2000). Permanency and ephemerality of psychological measures with application to organizational commitment. Psychol. Methods 5, 175-198. doi: 10.1037/1082-989X.5.2.175.
92. Van De Schoot, R., Kluytmans, A., Tummers, L., Lugtig, P., Hox, J. J., and Muthén, B. (2013). Facing off with scylla and charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Front. Psychol. 4:770. doi: 10.3389/fpsyg.2013.00770.
93. Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Appl. Psychol. Measur. 9, 1-26. doi: 10.1177/014662168500900101.
94. Widaman, K. F., and Reise, S. P. (1997). Exploring the Measurement Invariance of Psychological Instruments: Applications in the Substance Use Domain. Washington, DC: American Psychological Association, 281-324.
95. Wothke, W. (1995). "Covariance components analysis of the multitrait-multimethod matrix," in Personality Research, Methods, and Theory: A Festschrift Honoring Donald W. Fiske, ed P. E. Shrout (Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.), 125-144.
96. Zimmerman, D. W. (1975). Probability spaces, Hilbert spaces, and the axioms of test theory. Psychometrika 40, 395-412. doi: 10.1007/BF02291765.