A Multilevel CFA-MTMM Model for Nested Structurally Different Methods

Journal of Educational and Behavioral Statistics

Volumn 40, Issue 5, 2015, Pages 477-510

  Koch, Tobias ^a   Schultze, Martin ^b   Burrus, Jeremy ^c   Roberts, Richard D ^c   Eid, Michael ^b  

^a LEUPHANA UNIVERSITY OF LÜNEBURG   (Germany)

^b FREIE UNIVERSITÄT BERLIN   (Germany)

^c Professional Examination Service   (United States)

Author keywords

MTMM analysis; multilevel structural equation modeling; structurally different and interchangeable methods

Indexed keywords

EID: 84945303884     PISSN: 10769986     EISSN: 19351054     Source Type: Journal    
DOI: 10.3102/1076998615606109     Document Type: Article

References (56)

1. T.A.Brown (2012). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
2. E.B.BrownlieK.LazareJ.Beitchman (2012). Validating a self-report screen for adhd in early adulthood using childhood parent and teacher ratings. Journal of Attention Disorders, 16, 467–477.
3. D.T.CampbellD.W.Fiske (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
4. H.Carretero-DiosM.EidW.Ruch (2011). Analyzing multitrait-mulitmethod data with multilevel confirmatory factor analysis: An application to the validation of the state-trait cheerfulness inventory. Journal of Research in Personality, 45, 153–164.
5. D.S.CourvoisierF.W.NussbeckM.EidC.GeiserD.A.Cole (2008). Analyzing the convergent and discriminant validity of states and traits: Development and applications of multimethod latent state-trait models. Psychological Assessment, 20, 270–280.
6. E.DanayM.Ziegler (2011). Is there really a single factor of personality? A multirater approach to the apex of personality. Journal of Research in Personality, 45, 560–567.
7. L.Dumenci (2000). Multitrait-multimethod analysis. In S.D.BrownH.E.Tinsley (Eds.), Handbook of applied multivariate statistics and mathematerial modeling (pp. 583–611). San Diego, CA: Academic Press.
8. B.DunlopT.LiS.KornsteinE.FriedmanA.RothschildR.Pedersen…M.Keller (2010). Correlation between patient and clinician assessments of depression severity in the PREVENT study. Psychiatry Research, 177, 177–183.
9. M.Eid (2000). A multitrait-multimethod model with minimal assumptions. Psychometrika, 65, 241–261.
10. M.EidT.LischetzkeF.W.NussbeckL.I.Trierweiler (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8, 38–60.
11. M.EidF.W.NussbeckC.GeiserD.A.ColeM.GollwitzerT.Lischetzke (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253.
12. M.EidT.LischetzkeF.W.Nussbeck (2006). Structural equation models for multitrait-multimethod data. In M.EidE.Diener (Eds.), Handbook of multimethod measurement in psychology (pp. 283–299). Washington, DC: American Psychological Association.
13. C.Geiser (2009). Multitrait-multimethod-multioccasion modeling. München, Germany: Akademischer Verlag München.
14. C.Geiser (2012). Data analysis with Mplus. New York, NY: Guilford Press.
15. C.GeiserG.L.BurnsM.Servera (2014). Testing for measurement invariance and latent mean differences across methods: Interesting incremental information from multitrait-multimethod studies. Frontiers in Psychology, 5, 1216.
16. C.GeiserM.EidF.W.Nussbeck (2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13, 49–57.
17. C.GeiserM.EidF.W.NussbeckD.S.CourvoisierD.A.Cole (2010). Analyzing true change in longitudinal multitrait-multimethod studies: Application of a multimethod change model to depression and anxiety in children. Developmental Psychology, 46, 29–45.
18. C.GeiserM.EidS.G.WestT.LischetzkeF.W.Nussbeck (2012). A comparison of method effects in two confirmatory factor models for structurally different methods. Structural Equation Modeling, 19, 409–436.
19. C.GeiserT.KochM.Eid (2014). Data-generating mechanisms versus constructively defined latent variables in multitrait-multimethod analysis: A comment on Castro-Schilo, Widaman, and Grimm (2013). Structural Equation Modeling: A Multidisciplinary Journal, 21, 509–523.
20. M.Hallquist (2011). MplusAutomation: Automating Mplus model estimation and interpretation. Retrieved fromhttp://cran.r-project.org/web/packages/MplusAutomation/index.html
21. K.M.Hannum (2007). Measurement equivalence of 360-assessment data: Are different raters rating the same constructs?International Journal of Selection and Assessment, 15, 293–301.
22. J.J.Hox (2010). Multilevel analysis techniques and applications (2nd ed.). New York, NY: Routledge.
23. J.J.HoxC.J.Maas (2001). The accuracy of multilevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling, 8, 157–174.
24. J.J.HoxR.van de SchootS.Matthijsse (2012). How few countries will do? comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6, 87–93.
25. R.H.Hoyle (2014). Handbook of structural equation modeling. New York, NY: Guilford Publications.
26. P.J.Huber (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Berkeley, CA: University of California Press.
27. V.HöflingK.Schermelleh-EngelH.Moosbrugger (2009). Analyzing multitrait-multimethod data. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 5, 99–111.
28. M.W.Julian (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling, 8, 325–352.
29. D.A.Kenny (1976). An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix. Journal of Experimental Social Psychology, 12, 247–252.
30. D.A.KennyD.A.Kashy (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112, 165–172.
31. T.Koch (2013). Multilevel structural equation modeling of multitrait-multimethod-multioccasion data(Doctoral dissertation). Freie Universität Berlin, Germany.
32. T.KochM.EidK.Lochner (in press). Multitrait-multimethod-analysis: The psychometric foundation of CFA-MTMM models. In P.IrwingT.BoothD.Hughes (Eds.), The Wiley handbook of psychometric testing. London, England: John Wiley.
33. T.KochM.SchultzeM.EidC.Geiser (2014). A longitudinal multilevel CFA-MTMM model for interchangeable and structurally different methods. Frontiers in Psychology, 5, 311.
34. T.KochM.SchultzeJ.MinjeongF.NussbeckA.-K.PraetoriusM.Eid (2014). A multilevel CFA-MTMM model for structurally different and non-independent interchangeable methods. Manuscript submitted for publication.
35. T.D.LittleT.D.JorgensenK.M.LangE.W.G.Moore (2014). On the joys of missing data. Journal of Pediatric Psychology, 39, 151–162.
36. O.LüdtkeH.MarshA.RobitzschU.TrautweinT.AsparouhovB.Muthén (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229.
37. H.W.Marsh (1993). Multitrait-multimethod analyses: Inferring each trait-method combination with multiple indicators. Applied Measurement in Education, 6, 49–81.
38. H.W.MarshM.Bailey (1991). Confirmatory factor analyses of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15, 47–70.
39. H.W.MarshD.Grayson (1995). Latent variable models of multitrait-multimethod data. In R.H.Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 177–198). Thousand Oaks, CA: Sage.
40. H.W.MarshD.Hocevar (1988). A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis. Journal of Applied Psychology, 73, 107–117.
41. B.O.Muthén (1994). Multilevel covariance structure analysis. Sociological Methods & Research, 22, 376–398.
42. B.MuthénT.Asparouhov (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313–335.
43. L.K.MuthénB.O.Muthén (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.
44. L.K.MuthénB.O.Muthén (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599–620.
45. S.PohlR.Steyer (2010). Modeling common traits and method effects in multitrait-multimethod analysis. Multivariate Behavioral Research, 45, 45–72.
46. S.PohlR.SteyerK.Kraus (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society: Series A (Statistics in Society), 171, 41–63.
47. A.SatorraP.M.Bentler (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A.von EyeC.Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Thousand Oaks, CA: Sage.
48. A.SatorraP.M.Bentler (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514.
49. K.Schermelleh-EngelN.KeithH.MoosbruggerV.Hodapp (2004). Decomposing person and occasion-specific effects: An extension of latent state-trait (LST) theory to hierarchical LST models. Psychological Methods, 9, 198.
50. M.SchultzeT.KochM.Eid (2015). The effects of non-independent rater sets in multilevel-multitrait-multimethod models. Structural Equation Modeling, 22, 439–448.
51. T.A.B.SnijdersR.J.Bosker (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London, England: Sage.
52. R.UherA.FarmerW.MaierM.RietschelJ.HauserA.Marusic…K.J.Aitchison (2008). Measuring depression: Comparison and integration of three scales in the GENDEP study. Psychological Medicine, 38, 289–300.
53. H.White (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817–838.
54. K.F.Widaman (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1–26.
55. A.WillifordP.J.FiteJ.L.Cooley (2015). Student–teacher congruence in reported rates of physical and relational victimization among elementary-school-age children: The moderating role of gender and age. Journal of School Violence, 14, 177–195.
56. W.Wothke (1995). Covariance components analysis of the multitrait-multimethod matrix. In P.E.ShroutS.T.Fiske (Eds.), Personality research, methods, and theory: A festschrift honoring Donald W. Fiske (pp. 125–144). Hillsdale, NJ: Lawrence Erlbaum.