Measurement invariance conventions and reporting: The state of the art and future directions for psychological research

Developmental Review

Volumn 41, Issue , 2016, Pages 71-90

  Putnick, Diane L ^a   Bornstein, Marc H ^a  

^a EUNICE KENNEDY SHRIVER NATIONAL INSTITUTE OF CHILD HEALTH AND HUMAN DEVELOPMENT   (United States)

Author keywords

Comparative psychology; Confirmatory factor analysis; Measurement invariance; Multiple group analysis; Structural equation modeling

Indexed keywords

FACTORIAL ANALYSIS; HUMAN; PSYCHOLOGY; SAMPLE SIZE; SCIENTIST; STRUCTURAL EQUATION MODELING; SYSTEMATIC REVIEW;

EID: 84977609212     PISSN: 02732297     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dr.2016.06.004     Document Type: Review

References (72)

1. Barrett, P., Structural equation modelling: adjudging model fit. Personality and Individual Differences 42 (2007), 815–824, 10.1016/j.paid.2006.09.018.
2. Bauer, D.J., Hussong, A.M., Psychometric approaches for developing commensurate measures across independent studies: traditional and new models. Psychological Methods 14 (2009), 101–125, 10.1037/a0015583.
3. Bentler, P.M., On tests and indices for evaluating structural models. Personality and Individual Differences 42 (2007), 825–829, 10.1016/j.paid.2006.09.024.
4. Bornstein, M.H., Form and function: implications for studies of culture and human development. Culture & Psychology 1 (1995), 123–137, 10.1177/1354067X9511009.
5. Byrne, B.M., Shavelson, R.J., Muthén, B., Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin 105 (1989), 456–466, 10.1037/0033-2909.105.3.456.
6. Chen, F., Curran, P.J., Bollen, K.A., Kirby, J., Paxton, P., An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research 36 (2008), 462–494, 10.1177/0049124108314720.
7. Chen, F.F., Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 14 (2007), 464–504, 10.1080/10705510701301834.
8. Chen, F.F., What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology 95 (2008), 1005–1018, 10.1037/a0013193.
9. Cheung, G.W., Lau, R.S., A direct comparison approach for testing measurement invariance. Organizational Research Methods 15 (2012), 167–198, 10.1177/1094428111421987.
10. Cheung, G.W., Rensvold, R.B., Cross-cultural comparisons using non-invariant measurement items. Applied Behavioral Science Review 6 (1998), 93–110, 10.1016/S1068-8595(99)80006-3.
11. Cheung, G.W., Rensvold, R.B., Testing factorial invariance across groups: a reconceptualization and proposed new method. Journal of Management 25 (1999), 1–27, 10.1177/014920639902500101.
12. Cheung, G.W., Rensvold, R.B., Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 9 (2002), 233–255, 10.1207/S15328007SEM0902_5.
13. Cohen, J., Statistical power analysis for the behavioral sciences. 2nd ed., 1988, Erlbaum, Hillsdale, NJ.
14. Curran, P.J., Hussong, A.M., Integrative data analysis: the simultaneous analysis of multiple data sets. Psychological Methods 14 (2009), 81–100, 10.1037/a0015914.
15. Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., Billiet, J., Measurement equivalence in cross-national research. Annual Review of Sociology 40 (2014), 55–75, 10.1146/annurev-soc-071913-043137.
16. Fan, X., Sivo, S.A., Using ΔGoodness-of-Fit indexes in assessing mean structure invariance. Structural Equation Modeling 16 (2009), 54–69, 10.1080/10705510802561311.
17. French, B.F., Finch, W.H., Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling 13 (2006), 378–402, 10.1207/s15328007sem1303_3.
18. French, B.F., Finch, W.H., Multigroup confirmatory factor analysis: locating the invariant referent sets. Structural Equation Modeling 15 (2008), 96–113, 10.1080/10705510701758349.
19. Glanville, J.L., Wildhagen, T., The measurement of school engagement: assessing dimensionality and measurement invariance across race and ethnicity. Educational and Psychological Measurement 67 (2007), 1019–1041, 10.1177/0013164406299126.
20. Guenole, N., Brown, A., The consequences of ignoring measurement invariance for path coefficients in structural equation models. Frontiers in Psychology, 5, 2014, 980, 10.3389/fpsyg.2014.00980.
21. Herzog, W., Boomsma, A., Reinecke, S., The model-size effect on traditional and modified tests of covariance structures. Structural Equation Modeling 14 (2007), 361–390, 10.1080/10705510701301602.
22. Hong, S., Malik, M.L., Lee, M.K., Testing configural, metric, scalar, and latent mean invariance across genders in sociotropy and autonomy using a non-Western sample. Educational and Psychological Measurement 63 (2003), 636–654, 10.1177/0013164403251332.
23. Johnson, E.C., Meade, A.W., DuVernet, A.M., The role of referent indicators in tests of measurement invariance. Structural Equation Modeling 16 (2009), 642–657, 10.1080/10705510903206014.
24. Jung, E., Yoon, M., Comparisons of three empirical methods for partial factorial invariance: forward, backward, and factor-ratio tests. Structural Equation Modeling: A Multidisciplinary Journal, 2016, 10.1080/10705511.2015.1138092.
25. Kenny, D.A., Kaniskan, B., McCoach, D.B., The performance of RMSEA in models with small degrees of freedom. Sociological Methods and Research 44 (2015), 486–507, 10.1177/0049124114543236.
26. Kline, R.B., Principles and practice of structural equation modeling. 4th ed., 2015, Guilford Press, New York.
27. Little, T.D., Mean and covariance structure (MACS) analyses of cross-cultural data: practical and theoretical issues. Multivariate Behavioral Research 32 (1997), 53–76, 10.1207/s15327906mbr3201_3.
28. Little, T.D., On the comparability of constructs in cross-cultural research: a critique of Cheung and Rensvold. Journal of Cross-Cultural Psychology 31 (2000), 213–219, 10.1177/0022022100031002004.
29. Little, T.D., Longitudinal structural equation modeling. 2013, Guilford Press, New York.
30. Marsh, H.W., Hocevar, D., Application of confirmatory factor analysis to the study of self-concept: first-and higher order factor models and their invariance across groups. Psychological Bulletin 97 (1985), 562–582, 10.1037/0033-2909.97.3.562.
31. McDonald, R.P., An index of goodness-of-fit based on noncentrality. Journal of Classification 6 (1989), 97–103.
32. Meade, A.W., Johnson, E.C., Braddy, P.W., Power and sensitivity of alternative fit indices in tests of measurement invariance. The Journal of Applied Psychology 93 (2008), 568–592, 10.1037/0021-9010.93.3.568.
33. Meade, A.W., Lautenschlager, G.J., A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance. Organizational Research Methods 7 (2004), 361–388, 10.1177/1094428104268027.
34. Meredith, W., Notes on factorial invariance. Psychometrika 29 (1964), 177–185.
35. Meredith, W., Measurement invariance, factor analysis and factorial invariance. Psychometrika 58 (1993), 525–543.
36. Meuleman, B., When are item intercept differences substantively relevant in measurement invariance testing?. Salzborn, S., Davidov, E., Reinecke, J., (eds.) Methods, theories, and empirical applications in the social sciences: Festschrift for Peter Schmidt, 2012, Springer, Wiesbaden, Germany, 97–104.
37. Milfont, T.L., Fischer, R., Testing measurement invariance across groups: applications in cross-cultural research. International Journal of Psychological Research 3 (2010), 111–121 http://dx.doi.org/10.21500/20112084.857.
38. Millsap, R.E., Four unresolved problems in studies of factorial invariance. Maydeu-Olivares, A., McArdle, J.J., (eds.) Contemporary psychometrics: A festschrift for Roderick P. McDonald Multivariate applications book series, 2005, Erlbaum, Mahwah, NJ, 153–171.
39. Millsap, R.E., Statistical approaches to measurement invariance. 2011, Routledge, New York.
40. Millsap, R.E., Kwok, O.M., Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods 9 (2004), 93–115, 10.1037/1082-989X.9.1.93.
41. Molenaar, D., Dolan, C.V., Wicherts, J.M., van der Maas, H.L., Modeling differentiation of cognitive abilities within the higher-order factor model using moderated factor analysis. Intelligence 38 (2010), 611–624, 10.1016/j.intell.2010.09.002.
42. Muthén, B., Asparouhov, T., Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. http://www.statmodel.com/download/webnotes/CatMGLong.pdf, 2002 Accessed 27.06.16.
43. Muthén, L.K., Muthén, B.O., Mplus user's guide. 6th ed., 2010, Muthén & Muthén, Los Angeles, CA.
44. Nolte, S., Elsworth, G.R., Sinclair, A.J., Osborne, R.H., Tests of measurement invariance failed to support the application of the “then-test”. Journal of Clinical Epidemiology 62 (2009), 1173–1180, 10.1016/j.jclinepi.2009.01.021.
45. Nye, C.D., Drasgow, F., Effect size indices for analyses of measurement equivalence: understanding the practical importance of differences between groups. The Journal of Applied Psychology 96 (2011), 966–980, 10.1037/a0022955.
46. Oberski, D.L., Evaluating sensitivity of parameters of interest to measurement invariance in latent variable models. Political Analysis 22 (2014), 45–60, 10.1093/pan/mpt014.
47. Ployhart, R.E., Oswald, F.L., Applications of mean and covariance structure analysis: integrating correlational and experimental approaches. Organizational Research Methods 7 (2004), 27–65, 10.1177/1094428103259554.
48. Raju, N.S., Laffitte, L.J., Byrne, B.M., Measurement equivalence: a comparison of methods based on confirmatory factor analysis and item response theory. The Journal of Applied Psychology 87 (2002), 517–529, 10.1037/0021-9010.87.3.517.
49. Reise, S.P., Widaman, K.F., Pugh, R.H., Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychological Bulletin 114 (1993), 552–566, 10.1037/0033-2909.114.3.552.
50. Rensvold, R.B., Cheung, G.W., Testing measurement models for factorial invariance: a systematic approach. Educational and Psychological Measurement 58 (1998), 1017–1034, 10.1177/0013164498058006010.
51. Rensvold, R.B., Cheung, G.W., Testing for metric invariance using structural equation models: solving the standardization problem. Schriesheim, C.A., Neider, L.L., (eds.) Research in management Equivalence of measurement, Vol. 1, 2001, Information Age, Greenwich, CT, 21–50.
52. Rosenthal, R., The file drawer problem and tolerance for null results. Psychological Bulletin 86 (1979), 638–641, 10.1037/0033-2909.86.3.638.
53. Rutkowski, L., Svetina, D., Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement 74 (2014), 31–57, 10.1177/0013164413498257.
54. Schmitt, N., Golubovich, J., Leong, F.T., Impact of measurement invariance on construct correlations, mean differences, and relations with external correlates: an illustrative example using Big Five and RIASEC measures. Assessment 18 (2011), 412–427, 10.1177/1073191110373223.
55. Schmitt, N., Kuljanin, G., Measurement invariance: review of practice and implications. Human Resource Management Review 18 (2008), 210–222, 10.1016/j.hrmr.2008.03.003.
56. Senese, P.V., Bornstein, M.H., Haynes, O.M., Rossi, G., Venuti, P.A., Cross-cultural comparison of mothers’ beliefs about their parenting very young children. Infant Behavior and Development 35 (2012), 479–488, 10.1016/j.infbeh.2012.02.006.
57. Stark, S., Chernyshenko, O.S., Drasgow, F., Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. The Journal of Applied Psychology 91 (2006), 1292–1306, 10.1037/0021-9010.91.6.1292.
58. Steenkamp, J.E.M., Baumgartner, H., Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25 (1998), 78–90 http://dx.doi.org/10.1086/209528.
59. Steinmetz, H., Analyzing observed composite differences across multiple groups: Is partial measurement invariance enough?. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences 9 (2013), 1–12, 10.1027/1614-2241/a000049.
60. Struening, E.L., Cohen, J., Factorial invariance and other psychometric characteristics of five opinions about mental illness factors. Educational and Psychological Measurement 23 (1963), 289–298, 10.1177/001316446302300206.
61. Tay, L., Meade, A.W., Cao, M., An overview and practical guide to IRT measurement equivalence analysis. Organizational Research Methods 18 (2015), 3–46, 10.1177/1094428114553062.
62. van de Schoot, R., Lugtig, P., Hox, J., A checklist for testing measurement invariance. European Journal of Developmental Psychology 9 (2012), 486–492, 10.1080/17405629.2012.686740.
63. Vandenberg, R.J., Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods 5 (2002), 139–158, 10.1177/1094428102005002001.
64. Vandenberg, R.J., Lance, C.E., A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods 2 (2000), 4–69, 10.1177/109442810031002.
65. Wang, M., Summers, J.A., Little, T., Turnbull, A., Poston, D., Mannan, H., Perspectives of fathers and mothers of children in early intervention programmes in assessing family quality of life. Journal of Intellectual Disability Research 50 (2006), 977–988, 10.1111/j.1365-2788.2006.00932.x.
66. Wang, T., Merkle, E.C., Zeileis, A., Score-based tests of measurement invariance: use in practice. Frontiers in Psychology, 5, 2014, 10.3389/fpsyg.2014.00438.
67. Widaman, K.F., Ferrer, E., Conger, R.D., Factorial invariance within longitudinal structural equation models: measuring the same construct across time. Child Development Perspectives 4 (2010), 10–18, 10.1111/j.1750-8606.2009.00110.x.
68. Widaman, K.F., Grimm, K.J., Advanced psychometrics: confirmatory factor analysis, item response theory, and the study of measurement invariance. Reis, H.T., Judd, C.M., (eds.) Handbook of research methods in social and personality psychology, 2nd ed., 2014, Cambridge University Press, New York, 534–570.
69. Widaman, K.F., Reise, S.P., Exploring the measurement invariance of psychological instruments: applications in the substance use domain. Bryant, K.J., Windle, M.E., West, S.G., (eds.) The science of prevention: Methodological advances from alcohol and substance abuse research, 1997, American Psychological Association, Washington, DC, 281–324.
70. Willse, J.T., Goodman, J.T., Comparison of multiple-indicators, multiple-causes–and item response theory-based analyses of subgroup differences. Educational and Psychological Measurement 68 (2008), 587–602, 10.1177/0013164407312601.
71. Yoon, M., Kim, E.S., A comparison of sequential and nonsequential specification searches in testing factorial invariance. Behavior Research Methods 46 (2014), 1199–1206, 10.3758/s13428-013-0430-2.
72. Yoon, M., Millsap, R.E., Detecting violations of factorial invariance using data-based specification searches: a Monte Carlo study. Structural Equation Modeling 14 (2007), 435–463, 10.1080/10705510701301677.