An Overview and Practical Guide to IRT Measurement Equivalence Analysis

Organizational Research Methods

Volumn 18, Issue 1, 2015, Pages 3-46

  Tay, Louis ^a   Meade, Adam W ^b   Cao, Mengyang ^c  

^a PURDUE UNIVERSITY   (United States)

^b NORTH CAROLINA STATE UNIVERSITY   (United States)

^c UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

construct validation procedures; invariance testing; item response theory; measurement models; quantitative research; reliability and validity

Indexed keywords

EID: 84912096006     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428114553062     Document Type: Article

References (95)

1. Andrich D.A rating formulation for ordered response categories.Psychometrika. 1978;43:561-573
2. Bock R. D.Estimating item parameters and latent ability when responses are scored in two or more nominal categories.Psychometrika. 1972;37:29-51
3. Bock R. D.A brief history of item response theory.Educational Measurement: Issues and Practice. 1997;16 (4): 21-33
4. Cai L.,Thissen D.,du Toit S. H. C.Lincolnwood, IL: Scientific Software International; 2011:
5. Candell G. L.,Drasgow F.An iterative procedure for linking metrics and assessing item bias in item response theory.Applied Psychological Measurement. 1988;12:253-260
6. Chang H.-H.,Mazzeo J.,Roussos L.Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure.Journal of Educational Measurement. 1996;33:333-353
7. Chen W.-H.,Thissen D.Local dependence indexes for item pairs using item response theory.Journal of Educational and Behavioral Statistics. 1997;22:265-289
8. Cheung G. W.,Rensvold R. W.Testing factorial invariance across groups: A reconceptualization and proposed new method.Journal of Management. 1999;25:1-27
9. Cheung G. W.,Rensvold R. W.Evaluating goodness-of-fit indexes for testing measurement invariance.Structural Equation Modeling. 2002;9:233-255
10. De Ayala R. J.,Sava-Bolesta M.Item parameter recovery for the nominal response model.Applied Psychological Measurement. 1999;23:3-19
11. DeMars C. E.Sample size and recovery of nominal response model item parameters.Applied Psychological Measurement. 2003;27:275-288
12. DeMars C. E.An analytic comparison of effect sizes for differential item functioning.Applied Measurement in Education. 2011;24:189-209
13. Drasgow F.Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables.Psychological Bulletin. 1984;95:134-135
14. Drasgow F.Study of the measurement bias of two standardized psychological tests.Journal of Applied Psychology. 1987;72:19-29
15. Drasgow F.,Chernyshenko O. S.,Stark S.75 years after Likert: Thurstone was right!.Industrial and Organizational Psychology: Perspectives on Science and Practice. 2010;3:465-476
16. Drasgow F.,Levine M. V.,Tsien S.,Williams B. A.,Mead A. D.Fitting polytomous item response models to multiple-choice tests.Applied Psychological Measurement. 1995;19:145-165
17. Drasgow F.,Lissak R. I.Modified parallel analysis: A procedure for examining the latent dimensionality of dichotomously scored item responses.Journal of Applied Psychology. 1983;68:363-367
18. Drasgow F.,Nye C. D.,Tay L.Handbook of workplace assessment: Evidence-based practices for selecting and developing organizational talent. Scott J. C.Reynolds D. H., ed. San Francisco, CA: John Wiley & Sons; 2010:27-60.
19. Drasgow F.,Parsons C. K.Application of unidimensional item response theory models to multidimensional data.Applied Psychological Measurement. 1983;7:189-199
20. Ellis B. B.Differential item functioning: Implications for test translations.Journal of Applied Psychology. 1989;74:912-921
21. Embretson S. E.,Reise S. P.Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum; 2000:
22. Fidalgo A. M.,Madeira J. M.Generalized Mantel-Haenszel methods for differential item functioning detection.Educational and Psychological Measurement. 2008;68:940-958
23. Finch W. H.,French B. F.Detection of crossing differential item functioning: A comparison of four methods.Educational and Psychological Measurement. 2007;67:565-582
24. Gonzalez A.,Padilla J.-L.,Hidalgo M. D.,Gomez-Benito J.,Benitez I.EASY-DIF: Software for analyzing differential item functioning using the Mantel-Haenszel and standardization procedures.Applied Psychological Measurement. 2011;35:483-484
25. Hernandez A.,Drasgow F.,Gonzalez-Roma V.Investigating the functioning of a middle category by means of a mixed-measurement model.Journal of Applied Psychology. 2004;89 (4): 687-699
26. Holland W. P.,Thayer D. T.Test validity. Wainer H.Braun H. I., ed. Hillsdale, NJ: Lawrence Erlbaum; 1988:129-145.
27. Horn J. L.A rationale and test for the number of factors in factor analysis.Psychometrika. 1965;30:330-334
28. Hulin C. L.,Lissak R. I.,Drasgow F.Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study.Applied Psychological Measurement. 1982;6:249-260
29. Kim S.-H.,Cohen A. S.,Park T.-H.Detection of differential item functioning in multiple groups.Journal of Educational Measurement. 1995;32:261-276
30. Langer M.A reexamination of Lord’s Wald test for differential item functioning using item response theory and modern error estimation. University of North Carolina, Chapel Hill: ; 2008:
31. Lord F. M.Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum; 1980:
32. Lord F. M.,Novick M. R.Statistical theories of mental test scores. Reading, MA: Addison-Wesley; 1968:
33. Masters G. N.A Rasch model for partial credit scoring.Psychometrika. 1982;50:69-82
34. Maydeu-Olivares A.,Joe H.Limited and full information estimation and testing in 2n contingency tables: A unified framework.Journal of the American Statistical Association. 2005;100:1009-1020
35. Maydeu-Olivares A.,Joe H.Limited information goodness-of-fit testing in multidimensional contingency tables.Psychometrika. 2006;71:713-732
36. Meade A. W.A taxonomy of effect size measures for differential functioning of items and scales.Journal of Applied Psychology. 2010;95:728-743
37. Meade A. W.,Lautenschlager G. J.A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance.Organizational Research Methods. 2004;7:361-388
38. Meade A. W.,Lautenschlager G. J.,Johnson E. C.A Monte Carlo examination of the sensitivity of the differential functioning of items and tests framework for tests of measurement invariance with Likert data.Applied Psychological Measurement. 2007;31:430-455
39. Meade A. W.,Wright N. A.Solving the measurement invariance anchor item problem in item response theory.Journal of Applied Psychology. 2012;97:1016-1031
40. Mencl J.,Tay L.,Schwoerer C. E.,Drasgow F.Evaluating quantitative and qualitative types of change: An analysis of the malleability of general and specific self-efficacy constructs and measures.Journal of Leadership & Organizational Studies. 2012;19:378-391
41. Millsap R. E.,Hartog S. B.Alpha, beta and gamma change in evaluation research: A structural equation approach.Journal of Applied Psychology. 1988;73:574-584
42. Muraki E.A generalized partial credit model: Application of an EM algorithm.Applied Psychological Measurement. 1992;16:159-176
43. Narayanan P.,Swaminathan H.Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning.Applied Psychological Measurement. 1994;18:315-328
44. Narayanan P.,Swaminathan H.Identification of items that show nonuniform DIF.Applied Psychological Measurement. 1996;20:257-274
45. Nye C. D.,Drasgow F.Assessing goodness of fit: Simple rules of thumb simply do not work.Organizational Research Methods. 2010;14:548-570
46. Olino T. M.,Yu L.,Klein D. N.,Rohde P.,Seeley J. R.,Pilkonis P. A.,Lewinsohn P. M.Measuring depression using item response theory: An examination of three measures of depressive symptomatology.International Journal of Methods in Psychiatric Research. 2012;21:76-85
47. Orlando M.,Thissen D.Likelihood-based item fit indices for dichotomous item response theory models.Applied Psychological Measurement. 2000;24:50-64
48. Orlando M.,Thissen D.Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models.Applied Psychological Measurement. 2003;27:289-298
49. Penfield R. D.Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures.Applied Measurement in Education. 2001;14:235-259
50. Penfield R. D.DIFAS 5.0: Differential item functioning analysis system.Applied Psychological Measurement. 2005;29:150-151
51. Penfield R. D.,Camilli G.Handbook of statistics. Sinharay S.Rao C. R., ed. New York, NY: Elsevier; 2007:125-167.
52. Potenza M. T.,Dorans N. J.DIF assessment for polytomously scored items: A framework for classification and evaluation.Applied Psychological Measurement. 1995;19:23-27
53. Raju N. S.The area between two item characteristic curves.Psychometrika. 1988;53:495-502
54. Raju N. S.,van der Linden W. J.,Fleer P. F.IRT-based internal measures of differential functioning of items and tests.Applied Psychological Measurement. 1995;19:353-368
55. Ramsay J. O.Kernel smoothing approaches to nonparametric item characteristic curve estimation.Psychometrika. 1991;56:611-630
56. Reckase M. D.Unifactor latent trait models applied to multifactor tests: Results and implications.Journal of Educational Statistics. 1979;4:207-230
57. Reckase M. D.Multidimensional item response theory. New York, NY: Springer; 2009:
58. Riordan C. M.,Vandenberg R. J.A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner?.Journal of Management. 1994;20:643-671
59. Rizopoulos D.ltm: An R package for latent variable modeling and item response theory analyses.Journal of Statistical Software. 2006;17:1-25
60. Robert C.,Lee W. C.,Chan K.-Y.An empirical analysis of measurement equivalence with the INDCOL measure of individualism and collectivism: Implications for valid cross-cultural inference.Personnel Psychology. 2006;59:65-99
61. Roberts J. S.,Donoghue J. R.,Laughlin J. E.A general item response theory model for unfolding unidimensional polytomous responses.Applied Psychological Measurement. 2000;24:3-32
62. Samejima F.Estimation of latent ability using a response pattern of graded scores.Psychometrika Monograph Supplement. 1969;34:100-114
63. Shealy R.,Stout W.A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF.Psychometrika. 1993;58:159-194
64. Sireci S. G.,Rios J. A.Decisions that make a difference in detecting differential item functioning.Educational Research and Evaluation. 2013;19:170-187
65. Stark S.MODFIT: A computer program for model-data fit. Urbana-Champaign, IL: University of Illinois at Urbana-Champaign; 2001:
66. Stark S.Urbana-Champaign, IL: Department of Psychology, University of Illinois at Urbana-Champaign; 2002:
67. Stark S.,Chernyshenko O. S.,Drasgow F.Examining the effects of differential item (functioning and differential) test functioning on selection decisions: When are statistically significant effects practically important?.Journal of Applied Psychology. 2004;89:497-508
68. Stark S.,Chernyshenko O. S.,Drasgow F.Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy.Journal of Applied Psychology. 2006;91:1292-1306
69. Stark S.,Chernyshenko O. S.,Drasgow F.,Williams B. A.Examining assumptions about item responding in personality assessment: Should ideal point models be considered for scale development and scoring?.Journal of Applied Psychology. 2006;91:25-39
70. Steinberg L.,Thissen D.Using effect sizes for research reporting: Examples using item response theory to analyze differential item functioning.Psychological Methods. 2006;11:402-415
71. Swaminathan H.,Rogers J. H.Detecting differential item functioning using logistic regression procedures.Journal of Educational Measurement. 1990;27:361-370
72. Takane Y.,De Leeuw J.On the relationship between item response theory and factor analysis of discretized variables.Psychometrika. 1987;52:393-408
73. Tay L.,Drasgow F.Adjusting the adjusted χ2/df ratio statistic for dichotomous item response theory analyses: Does the model fit?.Educational and Psychological Measurement. 2012a;72:510-528
74. Tay L.,Drasgow F.Theoretical and statistical issues in the assessment of construct dimensionality: Accounting for the item response process.Organizational Research Methods. 2012b;15:363-384
75. Tay L.,Drasgow F.,Rounds J.,Williams B.Fitting measurement models to vocational interest data: Are dominance models ideal?.Journal of Applied Psychology. 2009;94:1287-1304
76. Tay L.,Newman D. A.,Vermunt J. K.Using mixed-measurement item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence.Organizational Research Methods. 2011;14:147-176
77. Tay L.,Vermunt J. K.,Wang C.Assessing the item response theory with covariate (IRT-C) procedure for ascertaining DIF.International Journal of Testing. 2013;13:201-222
78. Tay L.,Woo S. E.,Vermunt J. K.A conceptual and methodological framework for psychometric isomorphism: Validation of multilevel construct measures.Organizational Research Methods. 2014;17:77-106
79. Thissen D.,Chen W.-H.,Bock R. D.Lincolnwood, IL: Scientific Software International, Inc; 2003:
80. Thissen D.,Steinberg L.,Gerrard M.Beyond group-mean differences: The concept of item bias.Psychological Bulletin. 1986;99:118-128
81. Thissen D.,Steinberg L.,Wainer H.Differential item functioning. Holland P. W.Wainer H., ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1993:67-113.
82. Thompson R. C.,Hunt J. G.Inside the black box of alpha, beta, and gamma change: Using a cognitive-processing model to assess attitude structure.Academy of Management Review. 1996;21:655-690
83. Vandenberg R. J.,Lance C. E.A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research.Organizational Research Methods. 2000;3:4-70
84. Vermunt J. K.,Magidson J.Technical guide for Latent GOLD 4.0: Basic and advanced. Belmont, MA: Statistical Innovations Inc; 2005:
85. Vermunt J. K.,Magidson J.Belmont, MA: Statistical Innovations Inc; 2008:
86. Vermunt J. K.,Magidson J.Belmont, MA: Statistical Innovations Inc; 2013:
87. Wang W.,Tay L.,Drasgow F.Detecting differential item functioning of polytomous items for an ideal point response process.Applied Psychological Measurement. 2013;37:316-335
88. Weng L.-J.,Cheng C.-P.Parallel analysis with unidimensional binary data.Educational and Psychological Measurement. 2005;65:697-716
89. Woods C. M.Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis.Multivariate Behavioral Research. 2009;44:1-27
90. Woods C. M.,Cai L.,Wang M.The Langer-Improved Wald test for DIF testing with multiple groups: Evaluation and comparison to two-group IRT.Educational and Psychological Measurement. 2012;:
91. Woods C. M.,Grimm K. J.Testing of nonuniform differential item functioning with multiple indicator multiple cause models.Applied Psychological Measurement. 2011;35:339-361
92. Woods C. M.,Oltmanns T. F.,Turkheimer E.Illustration of MIMIC-model DIF testing with the Schedule for Nonadaptive and Adaptive Personality.Journal of Psychopathology and Behavioral Assessment. 2009;31:320-330
93. Zimowski M. F.,Muraki E.,Mislevy R. J.,Bock R. D.Chicago, IL: Scientific Software International; 1996:
94. Zumbo B. D.Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going.Language Assessment Quarterly. 2007;4:223-233
95. Zwick R.,Thayer D. T.Evaluating the magnitude of differential item functioning in polytomous items.Journal of Educational and Behavioral Statistics. 1996;21:187-201