Normative scoring of multidimensional pairwise preference personality scales using IRT: Empirical comparisons with other formats

Human Performance

Volumn 22, Issue 2, 2009, Pages 105-127

  Chernyshenko, Oleksandr S ^a   Stark, Stephen ^b   Prewett, Matthew S ^b   Gray, Ashley A ^b   Stilson, Frederick R ^b   Tuttle, Matthew D ^b  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b UNIVERSITY OF SOUTH FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 70449553320     PISSN: 08959285     EISSN: None     Source Type: Journal    
DOI: 10.1080/08959280902743303     Document Type: Article

References (44)

1. Andrich, D. (1995). Hyperbolic cosine latent models for unfolding direct responses and pairwise preferences. Applied Psychological Measurement, 19, 269-290.
2. Böckenholt, U. (2004). Comparative judgments as an alternative to ratings: Identifying the scale origin. Psychological Methods, 9, 453-465.
3. Borman, W. C., Buck, D., Hanson, M. A., Motowidlo, S. J., Stark, S., & Drasgow, F. (2001). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86, 965-973.
4. Bowen, C., Martin, B. A., & Hunt, S. T. (2002). A comparison of ipsative and normative approaches for ability to control faking in personality questionnaires. International Journal of Organizational Analysis, 10, 240-259.
5. Butcher, J. N., Dahlstrom, W. G., Graham, I. R., Tellegen, A., & Kaemmer, B. (1989). MMPI-2: Minnesota Multiphasic Personality Inventory-2: Manual for administration and scoring. Minneapolis: University of Minnesota Press.
6. Chernyshenko, O. S. (2002). Applications of ideal point approaches to scale construction and scoring in personality measurement: The development of a six-faceted measure of conscientiousness (Doctoral dissertation, University of Illinois at Urbana-Champaign, 2002). Dissertation Abstracts International, 63, 55-56.
7. Chernyshenko, O. S., Stark, S. E., Drasgow, F. D., & Roberts, B. W. (2007). Constructing personality scales under the assumptions of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19, 88-106.
8. Christiansen, N. D., Burns, G. N., & Montgomery, G. E. (2005). Reconsidering forced-choice item formats for applicant personality assessment. Human Performance, 18, 267-307.
9. Drasgow, F., Lee, W.C., Stark, S., & Chernyshenko, O.S. (2001, Spring). Alternative methodologies for predicting attrition in the Army: The new AIM scales (Report to HUMMRO). Alexandria, VA: Human Resources Research Organization.
10. Edwards, A. L. (1954). Personal Preference Schedule. New York: Psychological Corporation.
11. Goldberg, L. R. (1992). The development of markers for the Big Five factor structure. Psychological Assessment, 4, 26-42.
12. Heggestad, E. D., Morrison, M., Reeve, C. L., & McCloy, R. A. (2006). Forced-choice assessments of personality for selection: Evaluating issues of normative assessment and faking resistance. Journal of Applied Psychology, 91, 9-24.
13. Hicks, L. E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74, 167-184.
14. Houston, J. S., Borman, W. C., Farmer, W. L., & Bearden, R. M. (2005). Development of the enlisted computer adaptive personality scales (ENCAPS), renamed Navy Computer Adaptive Personality Scales (Tech. Rep. No. 502). Navy Personnel Research, Studies, and Technology, Navy Personnel Command.
15. Jackson, D. N., Wrobleski, V. R., & Ashton, M. C. (2000). The impact of faking on employment tests: Does forced-choice offer a solution? Human Performance, 13, 371-388.
16. James, L. R. (1998). Measurement of personality via conditional reasoning. Organizational Research Methods, 1, 131-163.
17. Levine, M. V. & Rubin, D. F. (1979). Measuring the appropriateness of multiple choice test scores. Journal of Educational Statistics, 4, 269-290.
18. McCloy, R. A., Heggestad, E. D., & Reeve, C. L. (2005). A silk purse from the sow's ear: Retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222-248.
19. Meade, A. W. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organizational Psychology, 77, 531-552.
20. Maydeu-Olivares, A., & Böckenholt, U. (2005). Structural equation modeling of paired-comparison and ranking data. Psychological Methods, 10, 285-304.
21. Murray, H. (1943). Thematic Apperception Test manual. Cambridge, MA: Harvard University Press.
22. Oswald, F. L., Schmitt, N., Kim, B. H., Ramsay, L. J., & Gillespie, M. A. (2004). Developing a biodata measure and situational judgment inventory as predictors of college success. Journal of Applied Psychology, 89, 187-208.
23. Roberts, B. (2002). Study Behavior Questionnaire. Unpublished manuscript, Department of Psychology, University of Illinois at Urbana-Champaign.
24. Roberts, B., Chernyshenko, O. S., Stark, S., & Goldberg, L. (2005). The construct of conscientiousness: The convergence between lexical models and scales drawn from six major personality questionnaires. Personnel Psychology, 58, 103-139.
25. Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3-32.
26. Roberts, J. S., & Laughlin, J. E. (1996). A unidimensional item response model for unfolding responses from a graded disagree-agree response scale. Applied Psychological Measurement, 20, 231-255.
27. Roberts, J. S., Laughlin, J. E., & Wedell, D. H. (1999). Validity issues in the Likert and Thurstone approaches to attitude measurement. Educational and Psychological Measurement, 59, 211-233.
28. Sisson, E. D. (1948). Forced choice-the new Army rating. Personnel Psychology, 1, 365-381.
29. Stark, S. (2002a). An Excel program for scoring MDPP items. Unpublished manuscript, University of Illinois at Urbana-Champaign.
30. Stark, S. (2002b). A new IRT approach to test construction and scoring designed to reduce the effects of faking in personal- ity assessment. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign.
31. Stark, S. (2005). EAP scoring program for ZG model. Unpublished manuscript, University of Illinois at Urbana- Champaign.
32. Stark, S., & Chernyshenko, O. S. (2006, August). Streamlining the construction of unidimensional pairwise preference scales: Effects on scoring accuracy and test validity. Paper submitted to the American Psychological Association conference, New Orleans, LA.
33. Stark, S., & Chernyshenko, O.S. (2007) Adaptive testing with the multi-unidimensional pairwise preference model (MUPP). Manuscript submitted for publication.
34. Stark, S., Chernyshenko, O. S., & Drasgow, F. (2005a). Examining the recovery of normative scores with three- and fivedimensional nonadaptive multidimensional forced choice tests. Paper presented at the 47th annual conference of the International Military Testing Association, Singapore.
35. Stark, S., Chernyshenko, O.S., & Drasgow, F. (2005b). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: An application to the problem of faking in personality assessment. Applied Psychological Measurement, 29, 184-201.
36. Stark, S., Chernyshenko, O. S., Drasgow, F., & Williams, B. A. (2006). Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring? Journal of Applied Psychology, 91, 25-39.
37. Stark, S., & Drasgow, F. (1998, April). Application of an item response theory ideal point model to computer adaptive assessment of job performance. Paper presented at the 13th annual conference for the Society of Industrial and Organizational Psychologists, Dallas, TX.
38. Stark, S., & Drasgow, F. (2002). An EM approach to parameter estimation for the Zinnes and Griggs paired comparison IRT model. Applied Psychological Measurement, 26, 208-227.
39. Thissen, D., & Wainer, H. (Eds.). (2001). Test scoring. Mahwah, NJ: Erlbaum.
40. Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19, 175-199.
41. Vickers, R. R., Jr., Conway, T. L., & Hervig, L. K. (1990). Demonstration of replicable dimensions of health behaviors. Preventive Medicine, 19, 377-401.
42. White, L. A., & Young, M. C. (1998, August). Development and validation of the Assessment of Individual Motivation (AIM). Paper presented at the annual meeting of the American Psychological Association, San Francisco, CA.
43. Zickar, M. J., & Drasgow, F. (1996). Detecting faking on a personality instrument using appropriateness measurement. Applied Psychological Measurement, 20, 71-87.
44. Zinnes, J. L., & Griggs, R.A. (1974). Probabilistic, multidimensional unfolding analysis. Psychometrika, 39, 327-350.