Using Mixed-Measurement Item Response Theory With Covariates (MM-IRT-C) to Ascertain Observed and Unobserved Measurement Equivalence

Organizational Research Methods

Volumn 14, Issue 1, 2011, Pages 147-176

  Tay, Louis ^a   Newman, Daniel A ^a   Vermunt, Jeroen K ^b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b TILBURG UNIVERSITY   (Netherlands)

Author keywords

differential item functioning; item response theory; latent class analysis; measurement invariance; profile analysis

Indexed keywords

EID: 78650138347     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428110366037     Document Type: Article

References (66)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.
2. Baker, F. (1995). EQUATE 2.1: Computer program for equating two metrics in item response theory. (Version 2.1). Madison: University of Wisconsin, Laboratory of Experimental Design.
3. Baron, R.M., & Kenny, D.A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
4. Bolt, D.M., Cohen, A.S., & Wollack, J.A. (2002). Item parameter estimation under conditions of test speededness: Application of a mixture Rasch model with ordinal constraints. Journal of Educational Measurement, 39, 331-348.
5. Bozdogan, H. (1993). Choosing the number of component clusters in the mixture-model using a new informational complexity criterion of the inverse-Fisher information matrix. In O. Opitz, B. Lausen & R. Klar (Eds.), Studies in classification, data analysis, and knowledge organization. Heidelberg, Germany: Springer-Verlag.
6. Candell, G.L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
7. Cohen, A.S., & Bolt, D.M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42, 133-148.
8. Collins, W.C., Raju, N.S., & Edwards, J.E. (2000). Assessing differential functioning in a satisfaction scale. Journal of Applied Psychology, 85, 451-461.
9. De Ayala, R.J., Kim, S.-H., Stapleton, L.M., & Dayton, C.M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2, 243-276.
10. Drasgow, F. (1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables. Psychological Bulletin, 95, 134-135.
11. Drasgow, F. (1987). Study of measurement bias of two standardized psychological tests. Journal of Applied Psychology, 72, 19-29.
12. Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662-680.
13. Eid, M., & Diener, E. (2001). Norms for experiencing emotions in different cultures: Inter- and intranational differences. Journal of Personality and Social Psychology, 81, 869-885.
14. Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.
15. Eid, M., & Zickar, M.J. (2007). Detecting response styles and faking in personality and organizational assessments by mixed Rasch models. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models (pp. 255-270). New York, NY: Springer.
16. Glas, C.A.W. (1998). Detection of differential item functioning using Lagrange Multiplier tests. Statistica Sinica, 8, 647-667.
17. Hambleton, R.K., & Kanjee, A. (1995). Increasing the validity of cross-cultural assessments: Use of improved methods for test adaptations. European Journal of Psychological Assessment, 11, 147-157.
18. Hernandez, A., Drasgow, F., & Gonzalez-Roma, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.
19. Hofstede, G.H. (1984). Culture's consequences: International differences in work-related values (abridged ed.). Beverly Hills, CA: Sage.
20. Hofstede, G.H. (2001). Culture's consequences: Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Thousand Oaks, CA: Sage.
21. Hulin, C.L., Drasgow, F., & Parsons, C.K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Dow Jones-Irwin.
22. Kim, S.-H., Cohen, A.S., & Park, T.-H. (1995). Detection of differential item functioning in multiple groups. Journal of Educational Measurement, 32, 261-276.
23. Kulas, J.T., & Finkelstein, L.M. (2007). Content and reliability of discrepancy-defined self-awareness in multisource feedback. Organizational Research Methods, 10, 502-522.
24. Landis, R.S., Beal, D.J., & Tesluk, P.E. (2000). A comparison of approaches to forming composite measures in structural equation models. Organizational Research Methods, 3, 186-207.
25. Li, F., Cohen, A.S., Kim, S.H., & Cho, S.J. (2009). Model selection methods for mixture dichotomous IRT models. Applied Psychological Measurement, 33, 353-373.
26. Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
27. Lubke, G.H., & Muthén, B. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21-39.
28. Magidson, J., & Vermunt, J.K. (2004). Latent class models. In D. Kaplan (Ed.), Handbook of quantitative methodology for the social sciences (pp. 175-198). Newbury Park, CA: SAGE.
29. Maij-de Meij, A.M., Kelderman, H., & van der Flier, H. (2005). Latent-trait latent-class anlaysis of self-disclosure in the work environment. Multivariate Behavioral Research, 40, 435-459.
30. Maij-de Meij, A.M., Kelderman, H., & van der Flier, H. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.
31. McShane, S.L. (1986). The multidimensionality of union participation. Journal of Occupational Psychology, 59, 177-187.
32. Mislevy, R.J., & Verhelst, N. (1990). Modeling item responses when different subjects employ different solution strategies. Psychometrika, 55, 195-215.
33. Muthén, B., & Muthén, L.K. (2007). Mplus Version 5.2 [Computer Program]. Los Angeles, CA: Muthén & Muthén.
34. Penfield, R.D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 2001, 235-259.
35. Raju, N.S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
36. Raju, N.S. (1999). DFIT5P: A Fortran Program for Calculating DIF/DTF [Computer software]. Chicago: Illinois Institute of Technology.
37. Raju, N.S., Laffitte, L.J., & Byrne, B.M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.
38. Raju, N.S., van der Linden, W.J., & Fleer, P.F. (1995). IRT-based internal measures of differential functioning of items and tests. Applied Psychological Measurement, 19, 353-368.
39. Reise, S.P., & Gomel, J.N. (1995). Modeling qualitative variation within latent trait dimensions: Application of mixed-measurement to personality assessment. Multivariate Behavioral Research, 30, 341-358.
40. Reise, S.P., & Waller, N.G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45-58.
41. Riordan, C.M., & Vandenberg, R.J. (1994). A central question in cross-cultural research: Do employees of different cultures interpret work-related measures in an equivalent manner? Journal of Management, 20, 643-671.
42. Robert, C., Lee, W.C., & Chan, K.-Y. (2006). An empirical analysis of measurement equivalence with the INDCOL measure of individualism and collectivism: Implications for valid cross-cultural inference. Personnel Psychology, 59, 65-99.
43. Rost, J. (1990). Rasch model in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 14, 271-282.
44. Rost, J. (1991). A logistic mixture distribution model for polychotomous item responses. British Journal of Mathematical and Statistical Psychology, 44, 75-92.
45. Rost, J., Carstensen, C., & von Davier, M. (1997). Applying the mixed Rasch model to personality questionnaires. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences. Munster, Germany: Waxman.
46. Schaffer, B.S., & Riordan, C.M. (2003). A review of cross-cultural methodologies for organizational research: A best-practices approach. Organizational Research Methods, 6, 169-215.
47. Schwarz, G.E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
48. Smit, J.A., Kelderman, H., & van der Flier, H. (1999). Collateral information and mixed Rasch models. Methods of Psychological Research, 4, 19-32.
49. Smit, J.A., Kelderman, H., & Van der Flier, H. (2000). The mixed Birnbaum model: Estimation using collateral information. Methods of Psychological Research, 5, 1-13.
50. Stark, S. (2002). ITERLINK: Iterative linking and pairwise DIF detection for the 3PL model using Lord's chisquare [Computer program]. Department of Psychology, University of Illinois at Urbana-Champaign.
51. Stark, S., Chernyshenko, O.S., Chan, K.-Y., Lee, W.C., & Drasgow, F. (2001). Effects of the testing situation on item responding: Cause for concern. Journal of Applied Psychology, 86, 943-953.
52. Stark, S., Chernyshenko, O.S., & Drasgow, F. (2004). Examining the effects of differential item (functioning and differential) test functioning on selection decisions: When are statistically significant effects practically important? Journal of Applied Psychology, 89, 497-508.
53. Stark, S., Chernyshenko, O.S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1292-1306.
54. Swaminathan, H., & Rogers, J.H. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
55. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-113). Hillsdale, NJ: Erlbaum.
56. Triandis, H., & Gelfand, M.J. (1998). Converging measurement of horizontal and vertical individualism and collectivism. Journal of Personality and Social Psychology, 74, 118-128.
57. Vandenberg, R.J. (2002). Toward a further understanding of and improvement in measurement invariance methods and procedures. Organizational Research Methods, 5, 139-158.
58. Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.
59. Vermunt, J.K., & Magidson, J. (2000). Latent GOLD 4.0 user manual. Belmont, MA: Statistical Innovations Inc.
60. Vermunt, J.K., & Magidson, J. (2005). Technical guide for Latent GOLD 4.0: Basic and advanced. Belmont, MA: Statistical Innovations Inc.
61. Vermunt, J.K., & Magidson, J. (2008). Latent GOLD 4.5 [Computer program]. Belmont, MA: Statistical Innovations Inc.
62. von Davier, M. (1997). Bootstrapping goodness-of-fit statistics for sparse categorical data. Results on a Monte Carlo study. Methods of Psychological Research Online, 2, 29-48.
63. Wang, M. (2007). Profiling retirees in the retirement transition and adjustment process: Examining the longitudinal change patterns of retirees' psychological well-being. Journal of Applied Psychology, 92, 455-474.
64. Woods, C.M. (2009). Evaluation of MIMIC-model methods for DIF testing with comparison to two-group analysis. Multivariate Behavioral Research, 44, 1-27.
65. Woods, C.M., Oltmanns, T.F., & Turkheimer, E. (2009). Illustration of MIMIC-model DIF testing with the Schedule for Nonadaptive and Adaptive Personality. Journal of Psychopathology and Behavioral Assessment, 31, 320-330.
66. Zickar, M.J., Gibby, R.E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 168-190.