Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data

Practical Assessment, Research and Evaluation

Volumn 12, Issue 3, 2007, Pages

  Wu, Amery D ^a   Li, Zhen ^a   Zumbo, Bruno D ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84870938926     PISSN: None     EISSN: 15317714     Source Type: Journal    
DOI: None     Document Type: Article

References (42)

1. Ackerman, T. (1992). A didactic explanation of item bias, item impact, and item validity from a multi-dimensional perspective. Journal of Educational Measurement, 29, 67-91.
2. Bandalos, D. L. (2002). The effects of item parcelling on goodness-of-fit and parameter estimate bias in structural equation modeling. Structural Equation Modeling, 9, 78-102.
3. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
4. Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16, 201-13.
5. Byrne, B. M., Shavelson, R. J., & Muthén, B. O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial MI. Psychological Bulletin, 105, 456-466.
6. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing MI. Structural Equation Modeling, 9, 235-55.
7. Cronbach, L. J. (1947). "Test reliability": Its meaning and determination. Psychometrika, 12, 1-16.
8. Deshon, R. P. (2004). Measures are not invariant across groups with error variance homogeneity. Psychology Science, 46, 137-49.
9. Fan, X., & Sivo, S. A. (2005). Sensitivity of fit indexes to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural Equation Models, 12, 343-67.
10. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to MI in aging research. Experimental Aging Research, 18, 117-144.
11. Hu, L., & Bentler, P. (1998). Fit Indices in covariance structure modeling: Sensitivity to underparameterized Model Misspecification, Psychological Methods, 3, 424-53.
12. Jöreskog, K. G., & Sörbom, D. (1999). LISREL 8 user's reference guide. Chicago: Scientific Software International.
13. Kelloway, E. K. (1995). Structural equation modeling in perspective. Journal of Organizational Behavior, 16, 215-24.
14. Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53-76.
15. Lubke, G. H., & Dolan, C. V. (2003). Can unequal residual variance across groups mask difference in residual means in the common factor model? Structural Equation Modeling, 10, 175-192.
16. Macnab, D. (2000a). Forces for changes in mathematics in education: The case of TIMSS. Education Policy Analysis Archives, 8(15).
17. Macnab, D. (2000b). Raising standards in mathematics education: Values, vision, and TIMSS. Educational Studies in Mathematics, 42, 61-81.
18. Marsh, H. W. (1994). Confirmatory factor models of factorial invariance: A multi-faceted approach. Structural Equation Modeling, 1, 5-34.
19. Marsh, H. W., Hau, K., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cut-off values for fit indexes and dangers in overgeneralization Hu and Bentler's (1999) findings. Structural Equation Modeling, 11(3), 320-341.
20. McArdle, J. J. (1998). Contemporary statistical models of test bias. In J. J. McArdle & R. W. Woodcock (Eds.), Human abilities in theory and practice (pp. 157-195). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
21. McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6, 97-103.
22. McGaw, B., & Jöreskog, K. G. (1971). Factorial invariance of ability measures in groups differing in intelligence and socio-economic status. British Journal of Mathematics and Statistical Psychology, 24, 154-168.
23. Mellenbergh, G. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-43.
24. Meredith, W. (1993). MI, factor analysis and factorial invariance. Psychometrika, 58, 525-43.
25. Meredith, W. & Millsap, R. E. (1992). On the misuse of manifest variables in the detection of measurement invariance. Psychometrika, 57(2), 289-311.
26. Millsap, R. E. (1998). Group differences in regression intercept: Implication for factorial invariance. Multivariate Behavioral Research, 33(3), 403-424.
27. National Institute on Educational Governance, Finance, Policymaking, and Management, (1998). What the Third International Mathematics and Science Study (TIMSS) means for systemic school Improvement. (Report No. GFI-98-9501). Washington, DC: U.S. Government Printing Office.
28. Rozeboom, W.W. (1966). Foundations of the theory of prediction. Homewood, IL: Dorsey.
29. Schermelleh-Engel, K., Moonsbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Test of significance measures. Methods of Psychological Research Online, 8, 23-74.
30. Schmidt, W. H., McKnight, C. C., Cogan, L. S., Jakwerth, P. M., & Houang, R. T. (1999). Facing the consequences: Using TIMSS for a closer look at U. S. mathematics and science education. Netherlands: Kluwer Academic Publishers.
31. Shealy, R. & Stout, W. F. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
32. Steenkamp, J. E., & Baumgartner, H. (1998). Assessing MI in cross-national consumer research. Journal of Consumer Research, 25, 78-90.
33. Steiger, J. H. (1989). EzPATH: Causal modeling. Evanston, IL: SYSTAT.
34. The International Association for the Evaluation of Educational Achievement; The International Study Center, Lynch School of Education (2001). TIMSS 1999 International Mathematics Report (Chap. 1). Retrieved March 5, 2006 from the TIMSS 1999 Web site: http://timss.bc.edu/timss1999i/math_achievement_report.html
35. The International Association for the Evaluation of Educational Achievement; The International Study Center, Lynch School of Education. (2004). TIMSS 2003 International Mathematics Report (Chap. 1). Retrieved March 5, 2006 from the TIMSS 2003 Web site: http://timss.bc.edu/PDF/t03_download/T03_M.Chap1.pdf
36. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
37. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the MI literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
38. Widaman, K. F., & Reise, S. P. (1997). Exploring the MI of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association.
39. Zumbo, B. D. (2007). Validity: Foundational issues and statistical methodology. In C. R. Rao and S. Sinharay (Eds.), Handbook of statistics, Vol. 26: Psychometrics (pp. 45-79). Elsevier Science B.V.: The Netherlands.
40. Zumbo, B. D. (2003). Does Item-Level DIF Manifest Itself in Scale-Level Analyses?: Implications for Translating Language Tests. Language Testing, 20, 136-147
41. Zumbo, B. D., & Koh, K. H. (2005). Manifestation of Differences in Item-Level Characteristics in Scale-Level Measurement Invariance Tests of Multi-Group Confirmatory Factor Analyses. Journal of Modern Applied Statistical Methods, 4, 275-282.
42. Zumbo, B. D., Sireci, S. G., & Hambleton, R. K. (2003, April). Re-visiting exploratory methods for construct comparability and MI: Is there something to be gained from the ways of old? Paper presented at the Annual Meeting of the National Council for Measurement in Education (NCME), Chicago, Illinois