Methodological issues in using structural equation models for testing differential item functioning

Cross-Cultural Analysis: Methods and Applications

Volumn , Issue , 2012, Pages 55-84

  Lee, Jaehoon ^a   Little, Todd D ^a   Preacher, Kristopher J ^a  

^a UNIVERSITY OF KANSAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84921365393     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203882924     Document Type: Chapter

References (87)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Bagozzi, R. P. (1994). The effects of arousal on the organization of positive and negative affect and cognitions: Application to attitude theory. Structural Equation Modeling, 1, 222-252.
3. Birnbaum. A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
4. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
5. Brannick, M. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16, 201-213.
6. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
7. Byrne, B. M., & Stewart, S. M. (2006). The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13, 287-321.
8. Camilli, G., & Shepard, L. A. (1994). Measurement methods for the social sciences series: Methods for identifying biased test items (Vol. 4). Thousand Oaks, CA: Sage.
9. Chan, D. (2000). Detection of differential item functioning on the Kirton Adaptation-Innovation Inventory using multiple-group mean and covariance structure analyses. Multivariate Behavioral Research, 35, 169-199.
10. Chen, C., & Anthony, J. C. (2003). Possible age-associated bias in reporting of clinical features of drug dependence: Epidemiological evidence on adolescent-onset marijuana use. Addiction, 98, 71-82.
11. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504.
12. Cheung, G. W., & Rensvold, R. B. (1999). Testing factorial invariance across groups: A reconceptualization and proposed new method. Journal of Management, 25, 1-27.
13. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
14. Christensen, H., MacKinnon, A. J., Korten, A., & Jorm, A. F. (2001). The "common cause hypothesis" of cognitive aging: Evidence for not only a common factor but also specific associations of age with vision and grip strength in a cross-sectional analysis. Psychology and Aging, 16, 588-599.
15. de Beuckelaer, A., Lievens, F., & Swinnen, G. (2007). Measurement equivalence in the conduct of a global organizational survey across countries in six cultural regions. Journal of Occupational and Organizational Psychology, 80, 575-600.
16. Dholakia, N., Firat, A. F., & Bagozzi, R. (1980). The de-Americanization of marketing thought: In search of a universal basis. In C. Lamb & P. Dunne (Eds.), Theoretical developments in marketing (pp. 25-29). Chicago, IL: American Marketing Association.
17. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel Haenszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
18. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355-368.
19. Drasgow, F. (1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are central issues. Psychological Bulletin, 95, 134-135.
20. Ferrando, P. J. (1996). Calibration of invariant item parameters in a continuous item response model using the extended LISREL measurement submodel. Multivariate Behavioral Research, 31, 419-439.
21. Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST and the IRT likelihood ratio test. Applied Psychological Measurement, 29, 278-295.
22. Fleishman, J. A. (2005). Using MIMIC models to assess the influence of differential item functioning. Retrieved from http://outcomes. cancer. gov/conference/irt/fleishman. pdf.
23. Fleishman, J. A., & Lawrence, W. F. (2003). Demographic variation in SF-12 scores: True differences or differential item functioning. Medical Care, 41, 75-86.
24. Fleishman, J. A., Spector, W. D., & Altman, B. M. (2002). Impact of differential item functioning on age and gender differences in functional disability. Journal of Gerontology: Social Sciences, 57, 275-283.
25. Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466-491.
26. French, B. F., & Finch, H. (2006). Confirmatory factor analytic procedures for determination of measurement invariance. Structural Equation Modeling, 13, 378-402.
27. Gallo, J. J., Anthony, J. C., & Muthen, B. O. (1994). Age differences in the symptoms of depression: A latent trait analysis. Journal of Gerontology: Psychological Sciences, 49, 251-264.
28. Gelin, M. N. (2005). Type I error rates of the DIF MIMIC approach using Joreskog's covariance matrix with ML and WLS estimation. Unpublished doctoral dissertation, University of British Columbia, Canada.
29. Goldstein, H., & Wood, R. (1989). Five decades of item response modeling. British Journal of Mathematical and Statistical Psychology, 42, 139-167.
30. Gonzalez-Roma, V., Tomas, I., Ferreres, D., & Hernandez, A. (2005). Do items that measure self-perceived physical appearance function differentially across gender groups of adolescents? An application of the MACS model. Structural Equation Modeling, 12, 157-171.
31. Grayson, D. A., Mackinnon, A., Jorm, A. F., Creasey, H., & Broe, G. A. (2000). Item bias in the Center for Epidemiological Studies Depression Scale: Effects of physical disorders and disability in an elderly community sample. Journal of Gerontology: Psychological Sciences, 55B, 273-282.
32. Grayson, D. A., & Marsh, H. W. (1994). Identification with deficient rank loading matrices in Confirmatory Factor Analysis: Multitrait-multimethod models. Psychometrika, 59, 121-134.
33. Gregorich, S. E. (2006). Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44, 78-94.
34. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
35. Hernandez, A., & Gonzalez-Roma, V. (2003). Evaluating the multiple-group Mean and Covariance Structure analysis model for the detection of Differential Item Functioning in polytomous ordered items. Psichotema, 15, 322-327.
36. Jones, R. N. (2006). Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination: Detecting differential item functioning using MIMIC modeling. Medical Care, 44, 124-133.
37. Joreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426.
38. Joreskog, K. G., & Goldberger, A. S. (1975). Estimation of a model with multiple indicators and multiple causes of a single latent variable. Journal of the American Statistical Association, 10, 631-639.
39. Kamata, A., & Bauer, D. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15, 136-153.
40. Katsuya, T. (2007). Cross-cultural validity of self-construal scales: An investigation of differential item functioning using multigroup mean and covariance structure. Japanese Journal of Behaviormetrics, 34, 79-89.
41. Kelloway, E. K. (1995). Structural equation modeling in perspective. Journal of Organizational Behavior, 16, 215-224.
42. Lee, J. (2009). Type I error and power of the MACS CFA for DIF detection: Methodological issues and resolutions. Unpublished doctoral dissertation, University of Kansas.
43. Lee, J. Type I Error and power of the MIMIC technique for DIF detection: Problem of biased anchor set and a recommended procedure. Manuscript in preparation.
44. Leone, L., van der Zee, K., van Oudenhoven, J. P., Perugini, M., & Ercolani, A. P. (2005). The cross-cultural generalizability and validity of the Multicultural Personality Questionnaire. Personality and Individual Differences, 38, 1449-1462.
45. Liang, J. (2001). Assessing cross-cultural comparability in mental health among older adults. Journal of Mental Health and Aging, 7, 21-30.
46. Little, T. D. (1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53-76.
47. Little, T. D. (in press). Longitudinal structural equation modeling: Individual-difference panel models. New York: Guilford press.
48. Little, T. D., Slegers, D. W., & Card, N. A. (2006). A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models. Structural Equation Modeling, 13, 59-72.
49. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison Wesley.
50. Lubke, G. H., & Muthen, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11, 514-534.
51. MacIntosh, R., & Hashim, S. (2003). Variance estimation for converting MIMIC model parameters to IRT parameters in DIF analysis. Applied Psychological Measurement, 27, 372-379.
52. Mackinnon, A., Jorm, A. F., Christensen, H., Korten, A. E., Jacomb, P. A., & Rodgers, B. (1999). A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample. Personality and Individual Differences, 27, 405-416.
53. Marsh, H. W., Tracey, D. K., & Craven, R. G. (2006). Multidimensional self-concept structure for preadolescents with mild intellectual disability: A hybrid multigroup-MIMIC approach to factorial invariance and latent mean differences. Educational and Psychological Measurement, 66, 795-818.
54. McDonald, R. P. (1967). Nonlinear factor analysis. Psychometric Monographs, No. 15. New York, NY: Springer.
55. McDonald, R. P. (1999). Test theory: Unified treatment. Mahwah, NJ: Lawrence Erlbaum.
56. Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in test of measurement invariance. Journal of Applied Psychology, 93, 568-592.
57. Mellenbergh, G. J. (1994). A unidimensional latent trait model for continuous item responses. Multivariate Behavioral Research, 29, 223-237.
58. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.
59. Millsap, R. E. (2005). Four unresolved problems in studies of factorial invariance. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics (pp. 153-172). Mahwah, NJ: Lawrence Erlbaum.
60. Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39, 479-515.
61. Muthen, B. O. (1988). Some uses of structural equation modeling in validity studies: Extending IRT to external variables. In H. Wainder & H. Braun (Eds.), Test validity (pp. 213-238). Hillsdale, NJ: Lawrence Erlbaum.
62. Muthen, B. O., & Asparouhov, T. (2002). Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. Los Angeles, CA: University of California and Muthen & Muthen.
63. Muthen, B. O., & Christoffersson, A. (1981). Simultaneous factor analysis of dichotomous variables in several groups. Psychometrika, 46, 407-419.
64. Muthen, B. O., & Lehman, J. (1988). Multiple group IRT modeling: Applications to item bias analysis. Journal of Educational Statistics, 10, 133-142.
65. Navas-Ara, M. J., & Gomez-Benito, J. (2002). Effects of ability scale purification on identification of DIF. European Journal of Psychological Assessment, 18, 9-15.
66. Oishi, S. (2006). The concept of life satisfaction across cultures: An IRT analysis. Journal of Research in Personality, 40, 411-423.
67. Raju, N. S., Laffitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.
68. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research, Copenhagen. (Expanded edition, 1980. Chicago: The University Chicago Press).
69. Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory Factor Analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
70. Riordan, C. R., & Vandenberg, R. J. (1994). A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner? Journal of Management, 20, 643-671.
71. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monographs, No. 17. New York, NY: Springer.
72. Samejima, F. (1972). A general model for free-response data. Psychometrika Monographs, No. 18. New York, NY: Springer.
73. Satorra, A., & Bentler, P. M. (1988). Scaling corrections for chi-square statistics in covariance structure analysis. American Statistical Association 1988 proceedings of the business and economics section (pp. 308-313). Alexandria VA: American Statistical Association.
74. Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507-514.
75. Schaffer, B. S., & Riordan, C. M. (2003). A review of cross-cultural methodologies for organizational research: A best-practices approach. Organizational Research Methods, 6, 169-215.
76. Sorbom, D. (1974). A general method for studying differences in factor means and factor structure between groups. British Journal of Mathematical and Statistical Psychology, 27, 229-239.
77. Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1202-1306.
78. Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in crossnational consumer research. Journal of Consumer Research, 25, 78-90.
79. Takane, Y., & de Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.
80. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
81. van de Vijver, F., & Poortinga, Y. H. (1992). Testing in culturally heterogeneous populations: When are cultural loadings undesirable? European Journal of Psychological Assessment, 8, 17-24.
82. Wang, C. K. J., Liu, W. C., Biddle, S. J. H., & Spray, C. M. (2005). Cross-cultural validation of the Conceptions of the Nature of Athletic Ability Questionnaire Version 2. Personality and Individual Differences, 38, 1245-1256.
83. Welkenhuysen-Gybels, J., & van de Vijver, F. (2001). A comparison of methods for the evaluation of construct equivalence in a multigroup setting. Proceedings of the annual meeting of the American Statistical Association. Alexandria VA: American Statistical Association.
84. Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle, & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association.
85. Woods, C. M. (2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1-27.
86. Woods, C. M., Oltmanns, T. F., & Turkheimer, E. (2009). Illustration of MIMIC-Model DIF Testing with the Schedule for Nonadaptive and Adaptive Personality. Journal of Psychopathology and Behavioral Assessment, 31, 320-330.
87. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.