Item Response Models for Forced-Choice Questionnaires: A Common Framework

Psychometrika

Volumn 81, Issue 1, 2016, Pages 135-160

  Brown, Anna ^a  

^a UNIVERSITY OF KENT   (United Kingdom)

Author keywords

Bradley Terry model; dominance model; forced choice; ideal point model; ipsative data; Thurstonian choice model; unfolding model

Indexed keywords

DECISION MAKING; FACTOR ANALYSIS; HUMAN; PSYCHOLOGICAL MODEL; PSYCHOMETRY; QUESTIONNAIRE; STATISTICAL MODEL;

CHOICE BEHAVIOR; FACTOR ANALYSIS, STATISTICAL; HUMANS; MODELS, PSYCHOLOGICAL; MODELS, STATISTICAL; PSYCHOMETRICS; SURVEYS AND QUESTIONNAIRES;

EID: 84959200418     PISSN: 00333123     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11336-014-9434-9     Document Type: Article

References (59)

1. Andersen, E. B. (1976). Paired comparisons with individual differences. Psychometrika, 41(2), 141–157.
2. Andrich, D. (1989). A probabilistic IRT model for unfolding preference data. Applied Psychological Measurement, 13, 193–296.
3. Andrich, D. (1995). Hyperbolic cosine latent trait models for unfolding direct-responses and pairwise preferences. Applied Psychological Measurement, 20, 269–290.
4. Bartram, D. (2007). Increasing validity with forced-choice criterion measurement formats. International Journal of Selection and Assessment, 15, 263–272.
5. Bennett, J. F., & Hays, W. L. (1960). Multidimensional unfolding: Determining the dimensionality of ranked preference data. Psychometrika, 25, 27–43.
6. Block, J. (1961). The Q-sort method in personality assessment and psychiatric research. Springfield, IL: Charles C. Thomas.
7. Böckenholt, U. (2004). Comparative judgments as an alternative to ratings: Identifying the scale origin. Psychological Methods, 9, 453–465.
8. Böckenholt, U. (2006). Thurstonian-based analyses: Past, present and future utilities. Psychometrika, 71(4), 615–629.
9. Bradley, R. A. (1953). Some statistical methods in taste testing and quality evaluation. Biometrics, 9, 22–38.
10. Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39, 324–345.
11. Brady, H. E. (1989). Factor and ideal point analysis for interpersonally incomparable data. Psychometrika, 54, 181–202.
12. Brown, A. (2009). Doing less but getting more: Improving forced-choice measures with IRT. Paper presented at the 24th annual conference of the Society for Industrial and Organizational Psychology, New Orleans, LA.
13. Brown, A. & Bartram, D. (2009–2011). OPQ32r Technical Manual. Surrey, UK: SHL Group.
14. Brown, A., & Maydeu-Olivares, A. (2010). Issues that should not be overlooked in the dominance versus ideal point controversy. Industrial and Organizational Psychology, 3, 489–493.
15. Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71, 460–502.
16. Brown, A., & Maydeu-Olivares, A. (2012). Fitting a Thurstonian IRT model to forced-choice data using Mplus. Behavior Research Methods, 44, 1135–1147.
17. Brown, A., & Maydeu-Olivares, A. (2013). How IRT can solve problems of ipsative data in forced-choice questionnaires. Psychological Methods, 18, 36–52.
18. Brown, A., & Maydeu-Olivares, A. (in press). Modeling forced-choice response formats. In P. Irwing, T. Booth, & D. Hughes (Eds.), The Wiley Handbook of Psychometric Testing. London: Wiley.
19. Chan, W. (2003). Analyzing ipsative data in psychological research. Behaviormetrika, 30, 99–121.
20. Cheung, M. W. L., & Chan, W. (2002). Reducing uniform response bias with ipsative measurement in multiple-group confirmatory factor analysis. Structural Equation Modeling, 9, 55–77.
21. Christiansen, N., Burns, G., & Montgomery, G. (2005). Reconsidering the use of forced-choice formats for applicant personality assessment. Human Performance, 18, 267–307.
22. Clemans, W. V. (1966). An analytical and empirical examination of some properties of ipsative measures. Psychometric Monographs, 14.
23. Coombs, C. H. (1950). Psychological scaling without a unit of measurement. Psychological Review, 57, 145–158.
24. Coombs, C. H. (1960). A theory of data. Psychological Review, 67, 143–159.
25. De Soete, G., & Carroll, J. D. (1983). A maximum likelihood method for fitting the wandering vector model. Psychometrika, 48, 553–566.
26. Drasgow, F., Chernyshenko, O. S., & Stark, S. (2009). Test theory and personality measurement. In J. N. Butcher (Ed.), Oxford handbook of personality assessment. London: Oxford University Press.
27. Drasgow, F., Chernyshenko, O. S., & Stark, S. (2010). 75 years after Likert: Thurstone was right!. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 465–476.
28. Huang, J., & Mead, A. D. (2014, July 7). Effect of personality item writing on psychometric properties of ideal-point and Likert scales. Psychological Assessment. Advance online publication. doi: http://dx.doi.org/10.1037/a0037273.
29. Jackson, D., Wroblewski, V., & Ashton, M. (2000). The impact of faking on employment tests: Does forced choice offer a solution? Human Performance, 13, 371–388.
30. Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. New York, NY: Wiley.
31. Luce, R. D. (1977). The choice axiom after twenty years. Journal of Mathematical Psychology, 15, 215–233.
32. Martin, B. A., Bowen, C.-C., & Hunt, S. T. (2002). How effective are people at faking on personality questionnaires? Personality and Individual Differences, 32, 247–256.
33. Maydeu-Olivares, A. (1999). Thurstonian modeling of ranking data via mean and covariance structure analysis. Psychometrika, 64, 325–340.
34. Maydeu-Olivares, A., & Böckenholt, U. (2005). Structural equation modeling of paired-comparison and ranking data. Psychological Methods, 10, 285–304.
35. Maydeu-Olivares, A., & Böckenholt, U. (2008). Modeling subjective health outcomes: Top 10 reasons to use Thurstone’s method. Medical Care, 46, 346–348.
36. Maydeu-Olivares, A., & Brown, A. (2010). Item response modeling of paired comparison and ranking data. Multivariate Behavioral Research, 45, 935–974.
37. McCloy, R., Heggestad, E., & Reeve, C. (2005). A silk purse from the sow’s ear: Retrieving normative information from multidimensional forced-choice items. Organizational Research Methods, 8, 222–248.
38. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
39. McFadden, D. (1973). Conditional logit analysis of qualitative choice behavior. In P. Zarembka (Ed.), Frontiers in Econometrics. New York: Academic Press.
40. McFadden, D. (1976). Quantal choice analysis: A survey. Annals of Economic and Social Measurement, 5, 363–390.
41. McFadden, D. (2001). Economic choices. The American Economic Review, 91(3), 351–378.
42. Meade, A. (2004). Psychometric problems and issues involved with creating and using ipsative measures for selection. Journal of Occupational and Organisational Psychology, 77, 531–552.
43. Muthén, L.K. & Muthén, B.O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
44. Roberts, J. S., Donoghue, J. R., & Laughlin, J. E. (2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24, 3–32.
45. Schwarz, N., Knäuper, B., Hippler, H. J., Noelle-Neumann, E., & Clark, L. (1991). Rating scales numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, 570–582.
46. Shepard, R. N. (1957). Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space. Psychometrika, 22, 325–345.
47. Stark, S., Chernyshenko, O., & Drasgow, F. (2005). An IRT approach to constructing and scoring pairwise preference items involving stimuli on different dimensions: The multi-unidimensional pairwise-preference model. Applied Psychological Measurement, 29, 184–203.
48. Stark, S., & Drasgow, F. (2002). An EM approach to parameter estimation for the Zinnes and Griggs paired comparison IRT model. Applied Psychological Measurement, 26, 208–227.
49. Takane, Y. (1987). Analysis of covariance structures and probabilistic binary choice data. Communication and Cognition, 20, 45–62.
50. Takane, Y. (1996). An item response model for multidimensional analysis of multiple choice data. Behaviormetrika, 23, 153–167.
51. Takane, Y., & De Leeuw, J. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393–408.
52. Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34, 273–286.
53. Thurstone, L. L. (1928). Attitudes can be measured. American Journal of Sociology, 33, 529–554.
54. Thurstone, L. L. (1929). The measurement of psychological value. In T. V. Smith & W. K. Wright (Eds.), Essays in philosophy by seventeen doctors of philosophy of the University of Chicago (pp. 157–174). Chicago: Open Court.
55. Thurstone, L. L. (1931). Rank order as a psychophysical method. Journal of Experimental Psychology, 14, 187–201.
56. Tsai, R. C., & Böckenholt, U. (2001). Maximum likelihood estimation of factor and ideal point models for paired comparison data. Journal of Mathematical Psychology, 45, 795–811.
57. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79(4), 281–299.
58. Vasilopoulos, N. L., Cucina, J. M., Dyomina, N. V., Morewitz, C. L., & Reilly, R. R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19, 175–199.
59. Zinnes, J. L., & Griggs, R. A. (1974). Probabilistic, multidimensional unfolding analysis. Psychometrika, 39, 327–350.