The Problem with Having Two Watches: Assessment of Fit When RMSEA and CFI Disagree

Multivariate Behavioral Research

Volumn 51, Issue 2-3, 2016, Pages 220-239

  Lai, Keke ^a   Green, Samuel B ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b ARIZONA STATE UNIVERSITY   (United States)

Author keywords

comparative fit index; fit indices; root mean square error of approximation; Structural equation modeling

Indexed keywords

DECISION MAKING; MODEL; COMPUTER SIMULATION; HUMAN; PSYCHOLOGIC TEST; STATISTICAL ANALYSIS; STATISTICAL MODEL;

COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; HUMANS; MODELS, STATISTICAL; PSYCHOLOGICAL TESTS;

EID: 84961637017     PISSN: 00273171     EISSN: None     Source Type: Journal    
DOI: 10.1080/00273171.2015.1134306     Document Type: Article

References (65)

1. P.M.Bentler, (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238–246. doi: 10.1037/0033-2909.107.2.238
2. P.M.Bentler, (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.
3. P.M.Bentler,, & D.G.Bonett, (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606. doi: 10.1037/0033-2909.88.3.588
4. K.A.Bollen, (1989). Structural equations with latent variables. New York, NY: John Wiley & Sons.
5. M.W.Browne,, & R.Cudeck, (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230–258. doi: 10.1177/0049124192021002005
6. M.W.Browne,, R.C.MacCallum,, C.-T.Kim,, B.L.Andersen,, & R.Glaser, (2002). When fit indices and residuals are incompatible. Psychological Methods, 7, 403–421. doi: 10.1037/1082-989X.7.4.403
7. J.Cohen, (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
8. R.Cudeck,, & M.W.Browne, (1992). Constructing a covariance matrix that yields a specified minimizer and a specified minimum discrepancy function value. Psychometrika, 57, 357–369. doi: 10.1007/BF02295424
9. P.J.Curran,, K.A.Bollen,, F.Chen,, P.Paxton,, & J.B.Kirby, (2003). Finite sampling properties of the point estimates and confidence intervals of the RMSEA. Sociological Methods & Research Methods, 32, 208–252. doi: 10.1177/0049124103256130
10. P.Dudgeon, (2004). A note on extending Steiger's (1998) multiple sample RMSEA adjustment to other noncentrality parameter-based statistics. Structural Equation Modeling, 11, 305–319. doi: 10.1207/s15328007sem1103_1
11. X.Fan,, & S.A.Sivo, (2005). Sensitivity of fit indices to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural equation modeling, 12, 343–367. doi:10.1207/s15328007sem1203_1
12. X.Fan,, & S.A.Sivo, (2007). Sensitivity of fit indices to model misspecification and model type. Multivariate Behavioral Research, 42, 509–529. doi: 10.1080/00273170701382864
13. S.J.Finney,, & C.DiStefano, (2013). Nonnormal and categorical data in structural equation modeling. In G.R.Hancock & R.O.Mueller (Eds.). Structural equation modeling: A second course (2nd ed., pp. 439–492). Greenwich, CT: Information Age Publishing.
14. B.F.Green, (1977). Parameter sensitivity in multivariate methods. Multivariate Behavioral Research, 12, 263–288. doi: 10.1207/s15327906mbr1203_1
15. G.R.Hancock, (2006). Power analysis in covariance structure modeling. In G.R.Hancock & R.O.Mueller (Eds.), Structural equation modeling: A second course (pp. 69–115). Greenwich, CT: Information Age Publishing.
16. G.R.Hancock,, & R.O.Mueller, (2011). The reliability paradox in assessing structural relations within covariance structure models. Educational and Psychological Measurement, 71, 306–324. doi: 10.1177/0013164410384856
17. M.Heene,, S.Hilbert,, C.Draxler,, M.Ziegler,, & M.Buhner, (2011). Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices. Psychological Methods, 16, 319–336. doi: 10.1037/a0024917
18. L.Hu,, & P.M.Bentler, (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453. doi: 10.1037/1082-989X.3.4.424
19. L.Hu,, & P.M.Bentler, (1999). Cutoff criteria in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi: 10.1080/10705519909540118
20. D.L.Jackson,, J.J.A.Gillaspy,, & R.Purc-Stephenson, (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23. doi: 10.1037/a0014694
21. R.A.Johnson,, & D.W.Wichern, (2007). Applied multivariate statistical analysis. Upper Saddle River, NJ: Pearson.
22. K.G.Jöreskog,, & D.Sörbom, (1981). LISREL V: Analysis of linear structural relationships by maximum likelihood and least squares methods. Chicago, IL: International Educational Services.
23. K.G.Jöreskog,, & D.Sörbom, (2006). LISREL 8.8 for Windows. Lincolnwood, IL: Scientific Software International.
24. R.E.Kass,, & A.E.Raftery, (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. doi: 10.1080/01621459.1995.10476572
25. K.Kelley,, & K.Lai, (2012). MBESS [R package version 3.3.3]. Retrieved from http://CRAN.R-project.org/package=MBESS
26. D.A.Kenny,, & D.B.McCoach, (2003). Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling, 10, 333–351. doi: 10.1207/S15328007SEM1003_1
27. K.H.Kim, (2005). The relation among fit indexes, power, and sample size in structural equation modeling. Structural Equation Modeling, 12, 368–390. doi: 10.1207/s15328007sem1203_2
28. K.Lai,, & K.Kelley, (2011). Accuracy in parameter estimation for targeted effects in structural equation modeling: Sample size planning for narrow confidence intervals. Psychological Methods, 16, 127–148. doi: 10.1037/a0021764
29. R.C.MacCallum,, M.W.Browne,, & L.Cai, (2006). Testing differences between nested covariance structure models: Power analysis and null hypotheses. Psychological Methods, 11, 19–35. doi: 10.1037/1082-989X.11.1.19
30. R.C.MacCallum,, M.W.Browne,, & H.M.Sugawara, (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149. doi: 10.1037/1082-989X.1.2.130
31. R.C.MacCallum,, & S.Hong, (1997). Power analysis in covariance structure modeling using GFI and AGFI. Multivariate Behavioral Research, 32, 193–210. doi: 10.1207/s15327906mbr3202_5
32. D.Markland, (2005). The golden rule is that there are no golden rules: A commentary on Paul Barrett's recommendations for reporting model fit in structural equation modeling. Personality and Individual Differences, 42, 851–858. doi: 10.1016/j.paid.2006.09.023
33. H.W.Marsh, (1998). The equal correlation baseline model: Comment and constructive alternatives. Structural Equation Modeling, 5, 78–86. doi: 10.1080/10705519809540090
34. H.W.Marsh,, K.-T.Hau,, & D.Grayson, (2005). Goodness of fit evaluation in structural equation modeling. In A.Maydeu-Olivares & J.McArdle (Eds.), Contemporary psychometrics: A festschrift to Roderick P. McDonald (pp. 275--340). Mahwah, NJ: Lawrence Erlbaum Associates.
35. H.W.Marsh,, K.-T.Hau,, & Z.Wen, (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Structural Equation Modeling, 11, 391–410. doi: 10.1207/s15328007sem1103_2
36. S.E.Maxwell, (2000). Sample size and multiple regression. Psychological Methods, 5, 434–458. doi: 10.1037/1082-989X.5.4.434
37. R.P.McDonald,, & M.H.R.Ho, (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82. doi: 10.1037/1082-989X.7.1.64
38. R.P.McDonald,, & H.W.Marsh, (1990). Choosing a multivariate model: Noncentrality and goodness-of-fit. Psychological Bulletin, 107, 247–255. doi: 10.1037/0033-2909.107.2.247
39. T.Micceri, (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166. doi: 10.1037/0033-2909.105.1.156
40. J.Miles,, & M.Shevlin, (2007). A time and a place for incremental fit indices. Personality and Individual Differences, 42, 869–874. doi: 10.1016/j.paid.2006.09.022
41. R.E.Millsap, (2007). Structural equation modeling made difficult. Personality and Individual Differences, 42, 875–881. doi: 10.1016/j.paid.2006.09.021
42. M.Moshagen, (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling, 19, 86–98. doi: 10.1080/10705511.2012.634724
43. R.O.Mueller,, & G.R.Hancock, (2010). Structural equation modeling. In G.R.Hancock & R.O.Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 371–384). New York, NY: Routledge.
44. L.K.Muthén,, & B.O.Muthén, (2012). Mplus 7 user's guide. Los Angeles, CA: Muthén & Muthén.
45. J.M.Nelson,, & G.L.Canivez, (2012). Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample. Psychological Assessment, 24, 129–140. doi: 10.1037/a0024878
46. R Core Team (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
47. N.S.Raju,, R.Bilgic,, J.E.Edwards,, & P.F.Fleer, (1999). Accuracy of population validity and cross-validity estimation: An empirical comparison of formula-based, traditional empirical, and equal weights procedures. Applied Psychological Measurement, 23, 99–115. doi: 10.1177/01466219922031220
48. C.R.Reynolds,, & R.W.Kamphaus, (2003). Reynolds Intellectual Assessment Scales. Lutz, FL: Psychological Assessment Resources.
49. E.E.Rigdon, (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling, 3, 369–379. doi: 10.1080/10705519609540052
50. E.E.Rigdon, (1998a). The equal correlation baseline model: A reply to Marsh. Structural Equation Modeling, 5, 87–94. doi: 10.1080/10705519809540091
51. E.E.Rigdon, (1998b). The equal correlation baseline model for comparative fit assessment in structural equation modeling. Structural Equation Modeling, 5, 63–77. doi: 10.1080/10705519809540089
52. A.Satorra,, & P.M.Bentler, (1988). Scaling corrections for chi-square statistics in covariance structure analysis. Proceedings of the American Statistical Association (pp. 308--313). Alexandria, VA: American Statistical Association.
53. A.Satorra,, & W.E.Saris, (1985). Power of the likelihood ratio test in covariance structure analysis. Psychometrika, 50, 83–90. doi: 10.1007/BF02294150
54. P.H.Schonemann, (1981). Power as a function of communality in factor analysis. Bulletin of the Psychonomic Society, 17, 57–60. doi: 10.3758/BF03333667
55. M.E.Sobel,, & G.W.Bohrnstedt, (1985). Use of null models in evaluating the fit of covariance structure models. In N.B.Tuma (Ed.), Sociological methodology 1985 (pp. 152–178). San Francisco, CA: Plenum Press.
56. J.H.Steiger, (1998). A note on multiple sample extensions of the RMSEA fit index. Structural Equation Modeling, 5, 411–419. doi: 10.1080/10705519809540115
57. J.H.Steiger,, & J.C.Lind, (1980, May). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA.
58. J.H.Steiger,, A.Shapiro,, & M.M.Browne, (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50, 253–264. doi: 10.1007/BF02294104
59. L.R.Tucker,, & C.Lewis, (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. doi: 10.1007/BF02291170
60. W.N.Venables,, & B.D.Ripley, (2002). Modern applied statistics with S (4th ed.). New York, NY: Springer.
61. K.F.Widaman,, & J.S.Thompson, (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8, 16–37. doi: /10.1037/1082-989X.8.1.16
62. W.Wu,, S.G.West,, & A.B.Taylor, (2009). Growth curve modeling: Evaluating model fit and model selection. Psychological Methods, 14, 183–201. doi: 10.1037/a0015858
63. K.-H.Yuan, (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40, 115–148. doi: 10.1207/s15327906mbr4001_5
64. K.-H.Yuan,, & P.M.Bentler, (2007). Robust procedures in structural equation modeling. In S.-Y.Lee (Ed.), Handbook of latent variable and related models (pp. 367–397). Burlington, UK: Elsevier.
65. K.-H.Yuan,, & W.Chan, (2005). On nonequivalence of several procedures of structural equation modeling. Psychometrika, 70, 791–798. doi: 10.1007/s11336-001-0930-9