Using Mixed-Model Item Response Theory to Analyze Organizational Survey Responses: An Illustration Using the Job Descriptive Index

Organizational Research Methods

Volumn 14, Issue 1, 2011, Pages 116-146

  Carter, Nathan T ^a   Dalal, Dev K ^a   Lake, Christopher J ^a   Lin, Bing C ^a,b   Zickar, Michael J ^a  

^a BOWLING GREEN STATE UNIVERSITY   (United States)

^b PORTLAND STATE UNIVERSITY   (United States)

Author keywords

invariance testing; item response theory; latent class analysis

Indexed keywords

EID: 78650118259     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428110363309     Document Type: Article

References (72)

1. Abrams, L.C., Cross, R., Lesser, E., & Levin, D.Z. (2003). Nurturing interpersonal trust in knowledge-sharing networks. Academy of Management Executive, 17, 64-71.
2. Adams, J.E., Carter, N.T., Wolford, K., Zickar, M.J., & Highhouse, S. (2009). The job descriptive index: A reliability generalization study. Presented at the 24th Annual Meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.
3. 3.
4. Andrich, D. (2005). The Rasch model explained. In S. Alagumalai, D. D. Durtis & N. Hungi (Eds.), Applied Rasch measurement: A book of exemplars. New York: Springer.
5. Balzer, W.K., Kihm, J.A., Smith, P.C., Irwin, J.L., Bachiochi, P.D., Robie, C.,et al.(1997). User's manual for the Job Descriptive Index (JDI; 1997 Revision) and the Job in General (JIG) scales. OH Bowling Green State University.
6. Bock, R.D. (1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
7. Bock, R., & Jones, L.V. (1968). The measurement and prediction of judgment and choice. San Francisco: Holden Day.
8. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. New York: Cambridge University Press.
9. Bowling, N.A., Hendricks, E.A., & Wagner, S.H. (2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business and Psychology, 23, 115-125.
10. Bozdogan, H. (1987). Model selection and Akaike's Information Criteria: The general theory and its analytic extensions. Psychometrika, 52, 345-370.
11. Chan, K.-Y., Drasgow, F., & Sawin, L.L. (1999). What is the shelf life of a test? The effect of time on the psychometrics of a cognitive ability test battery. Journal of Applied Psychology, 84, 610-619.
12. Chernyshenko, O.S., Stark, S., Chan, K.-Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523-562.
13. Clark, M.C., & Payne, R.L. (1997). The nature and structure of workers' trust in management. Journal of Organizational Behavior, 18, 205-224.
14. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
15. Collins, W.C., Raju, N.S., & Edwards, J.E. (2000). Assessing differential functioning in a satisfaction scale. Journal of Applied Psychology, 85, 451-461.
16. Connolly, J.J., & Viswesvaran, C. (2000). The role of affectivity in job satisfaction: A meta-analysis. Personality and Individual Differences, 29, 265-281.
17. Cooper-Hakim, A., & Viswesvaran, C. (2005). The construct of work commitment: Testing an integrative framework. Psychological Bulletin, 131, 241-259.
18. Cramér, H. (1946). Mathematical methods of statistics. Uppsala, Sweden: Almqvist & Wiksells.
19. Cruickshank, P.J. (1984). A stress arousal mood scale for low vocabulary subjects: A reworking of Mackay et al. (1978). British Journal of Psychology, 75, 89-94.
20. Detert, J.R., & Burris, E.R. (2007). Leadership behavior and employee voice: Is the door really open? Academy of Management Journal, 50, 869-884.
21. Dirks, K.T., & Ferrin, D.L. (2002). Trust in leadership: Meta-analytic findings and implications for research and practice. Journal of Applied Psychology, 87, 611-628.
22. Donovan, M.A., Drasgow, F., & Probst, T.M. (2000). Does computerizing paper-and-pencil job attitude scales make a difference? New IRT analyses offer insight. Journal of Applied Psychology, 85, 305-313.
23. Dubois, B., & Burns, J.A. (1975). An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 35, 869-884.
24. Eid, M., & Rauber, M. (2000). Detecting measurement invariance in organizational surveys. European Journal of Psychological Assessment, 16, 20-30.
25. Eid, M., & Zickar, M.J. (2007). Detecting response styles and faking in personality and organizational assessments by mixed Rasch models. In M. Von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.
26. Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
27. Goldberg, G. (1971). Response format in attitude scales. Unpublished manuscript, Northwestern University.
28. Hanisch, K.A. (1992). The job descriptive index revisited: Questions about the question mark. Journal of Applied Psychology, 77, 377-382.
29. Hart, R., & Hart, M. (2002). Statistical process control for health care. Pacific Grove, CA: Duxbury.
30. Hernandez, A., Drasgow, F., & Gonzalez-Roma, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.
31. Jablin, F.M. (1979). Superior-subordinate communication: The state of the art. Psychological Bulletin, 86, 1201-1222.
32. Judge, T.A., Heller, D., & Mount, M.K. (2002). Five-factor model of personality and job satisfaction: A meta-analysis. Journal of Applied Psychology, 87, 530-541.
33. Kaplan, K.J. (1972). On the ambivalence-indifference problem in attitude theory: A suggested modification of the semantic differential technique. Psychological Bulletin, 77, 361-372.
34. Krosnick, J.A., Holbrook, A.L., Berent, M.K., Carson, R.T., Hanemann, W.M., Kopp, R.J.,et al.(2002). The impact of "no opinion" response options on data quality: Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly, 66, 371-403.
35. Kutner, M.H., Nachtsheim, C.J., Neter, J., & Li, W. (2005). Applied linear statistical models (5th ed.). Boston: McGraw-Hill.
36. Lazarsfeld, P.E., & Henry, N.W. (1968). Latent structure analysis. Boston: Houghton Mifflin.
37. Levin, D.Z., Whitener, E.M., & Cross, R. (2006). Perceived trustworthiness of knowledge sources: The moderating impact of relationship length. Journal of Applied Psychology, 91, 1163-1171.
38. Maij-de Meij, A.M., Kelderman, H., & van der Flier, H. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.
39. Masters, G.N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
40. Maurer, T.J., Raju, N.S., & Collins, W.C. (1998). Peer and subordinate performance appraisal measurement equivalence. Journal of Applied Psychology, 83, 693-702.
41. Mayer, R.C., Davis, J.H., & Schoorman, F.D. (1995). An integrative model of organizational trust. Academy of Management Review, 20, 709-734.
42. Mellinger, G.D. (1956). Interpersonal trust as a factor in communication. Journal of Abnormal Social Psychology, 52, 304-309.
43. Menard, S. (2002). Applied logistic regression analysis (2nd ed.). Thousand Oaks, CA: Sage.
44. Mislevy, R., & Huang, C.W. (2007). Measurement models as narrative structures. In M. Von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.
45. Ostroff, C., & Kozlowski, S. (1992). Organizational socialization as a learning process: The role of information acquisition. Personnel Psychology, 45, 849-874.
46. Payne, R., & Clark, M. (2003). Dispositional and situational determinants of trust in two types of managers. International Journal of Human Resource Management, 14, 128-138.
47. Personnel Decisions International. (1997). Selection systems test scale manual. Unpublished document. Minneapolis, MN: Author.
48. Reise, S.P., & Gomel, J.N. (1995). Modeling qualitative variation within latent trait dimensions: Application of mixed-measurement to personality assessment. Multivariate Behavioral Research, 30, 341-358.
49. Reise, S.P., & Waller, N.G. (2003). How many IRT parameter does it take to model personality items? Psychological Methods, 8, 164-184.
50. Rost, J. (1991). A logistic mixture distribution for polychotomous item responses. British. Journal of Mathematical and Statistical Psychology, 44, 75-92.
51. Rost, J. (1997). Logistic mixture models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 449-463). New York: Springer.
52. Rost, J., & von Davier, M. (1994). A conditional item-fit index for Rasch models. Applied Psychological Measurement, 18, 171-182.
53. Rousseau, D.M., Sitkin, S.B., Burt, R.S., & Camerer, C. (1998). Not so different after all: A cross-discipline view of trust. Academy of Management Review, 23, 393-404.
54. Ryan, A.M., Horvath, M., Ployhart, R.E., Schmitt, N., & Slade, L. (2000). Hypothesizing differential item functioning in global employee opinion surveys. Personnel Psychology, 53, 531-562.
55. Schwarz, G.E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
56. Sheskin, D.J. (2004). Handbook of parametric and non-parametric statistical procedures (3rd ed.). Boca Raton, FL: Chapman & Hall/CRC.
57. Smit, A., Kelderman, H., & van der Flier, H. (2003). Latent trait latent class analysis of an Eysenck personality questionnaire. Methods of Psychological Research Online, 8, 23-50.
58. Smith, P.C., Kendall, L.M., & Hulin, C.L. (1969). The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Skokie, IL: Rand-McNally.
59. Tay, L., Drasgow, F., Rounds, J., & Williams, B. A. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal? Journal of Applied Psychology, 94, 1287-1304.
60. Thissen, D., Chen, W.-H., & Bock, R.D. (2003). Multilog (version 7) [Computer software]. Lincolnwood, IL: Scientific Software International.
61. Thissen, D. & Steinberg, L. (1986). A taxonomy of item response models. Psychometrika, 51, 567-577.
62. Vermunt, J.K., & Magidson, J. (2005). LatentGOLD (v4.5). Belmont, MA: Statistical Innovations, Inc.
63. von Davier, M. (2001). WINMIRA 2001: Windows mixed Rasch model analysis [Computer software and User manual]. Kiel, the Netherlands: Institute for Science Education.
64. von Davier, M., Rost, J., & Carstensen, C.H. (2007). Introduction: Extending the Rasch model. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.
65. Wang, M., & Russell, S.S. (2005). Measurement equivalence of the Job Descriptive Index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement, 65, 709-732.
66. White, L.A., Nord, R.D., Mael, F.A., & Young, M.C. (1993). The Assessment of Background and Life Experiences (ABLE). In T. Trent & J. H. Laurence (Eds.), Adaptability screening for the armed forces (pp. 101-162). Washington, DC: Office of Assistant secretary of Defense (Force Management and Personnel).
67. Wilde, G. (1970). Neurotische labiliteit gemeten volgens de vragenlijstmethode [Neurotic ability measured by the questionnaire method]. Amsterdam: Van Rossen.
68. Worthy, M. (1969). Note on scoring midpoint responses in extreme response style scores. Psychological Reports, 24, 189-190.
69. Zickar, M.J. (2001). Conquering the next frontier: Modeling personality data with item response theory. In B. Roberts & R. Hogan (Eds.), Applied personality psychology: The intersection of personality and I/O psychology (pp. 141-158). Washington, DC: American Psychological Association.
70. Zickar, M.J., & Broadfoot, A.A. (2009). The partial revival of a dead horse? Comparing classical test theory and item response theory. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 37-60). New York: Routledge.
71. Zickar, M.J., Gibby, R.E., & Robie, C. (2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory. Organizational Research Methods, 7, 168-190.
72. Zickar, M.J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551-563.