Comparing the validity of numerical judgements elicited by direct rating and point allocation: Insights from objectively verifiable perceptual tasks

European Journal of Operational Research

Volumn 228, Issue 1, 2013, Pages 148-157

  Bottomley, Paul A ^a   Doyle, John R ^a  

^a CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

Decision analysis; Direct rating; Methods of value elicitation; Objective tasks; Point allocation; Validity

Indexed keywords

BAR CHART; INDIVIDUAL LEVELS; NUMERICAL VALUES; OBJECTIVE TASKS; POINT ALLOCATION; PRIOR KNOWLEDGE; VALIDITY; VALUE ELICITATIONS;

DECISION THEORY; RATING;

NUMERICAL METHODS;

EID: 84875225224     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2013.01.005     Document Type: Article

References (42)

1. J.S. Armstrong, and F. Collopy Error measures for generalising about forecasting methods: empirical comparison International Journal of Forecasting 8 1992 69 80
2. K. Arrow Risk perception in psychology and economics Economic Inquiry 20 1982 1 9
3. F.H. Barron, and B.E. Barrett Decision quality using ranked attribute weights Management Science 42 1996 1515 1523
4. P.F. Bone, S. Sharma, and T.A. Shimp A bootstrap procedure for evaluating goodness-of-fit indices of structural equation and confirmatory factor models Journal of Marketing Research 26 1989 105 111
5. P.A. Bottomley, J.R. Doyle, and R.H. Green Testing the reliability of weight elicitation methods: direct rating versus point allocation Journal of Marketing Research 37 2000 508 513
6. G. Bous, P. Fortemps, F. Gilneur, and M. Pirlot ACUTA: a novel method for eliciting additive value functions on the basis of holistic preference statements European Journal of Operational Research 206 2010 435 444
7. W.S. Cleveland, and R. McGill Graphical perception: theory, experimentation, and application to the development of graphical methods Journal of the American Statistical Association 79 1984 531 554
8. E. Conde, and M. del la Paz Rivera Pérez A linear optimization problem to derive relative weights using an interval judgement matrix European Journal of Operational Research 201 2010 537 544
9. J.R. Doyle Elicitation and context effects in judgments: fixed sum versus fixed scale frames Management Science 45 1999 972 979
10. J.R. Doyle, R.H. Green, and P.A. Bottomley Judging relative importance. Direct rating and point allocation are not equivalent Organizational Behavior and Human Decision Processes 70 1997 65 72
11. B. Efron, and G. Wong A leisurely look at the bootstrap, the jack-knife, and cross-validation The American Statistician 37 1983 36 48
12. H. Gardiner Frames of Mind: The Theory of Multiple Intelligences 1983 Basic Books New York
13. A.L. Holbrook, M.C. Green, and J.A. Krosnick Telephone versus face-to-face interviewing of national probability samples with long questionnaires: comparisons of respondent satisficing and social desirability response bias Public Opinion Quarterly 67 2003 79 125
14. M.B. Holbrook, and R. Batra Accessing the role of emotions as mediators of consumer responses to advertising Journal of Consumer Research 14 1987 404 420
15. A. Ishizaka, D. Balkenborg, and T. Kaplan Does AHP help us make a choice? An experimental evaluation Journal of the Operational Research Society 62 2011 1801 1812
16. J. Jaccard, D. Brinberg, and L.J. Ackerman Assessing attribute importance. A comparison of six methods Journal of Consumer Research 12 1986 463 468
17. A. Jessop Minimally biased weight determination in personnel selection European Journal of Operational Research 153 2004 433 444
18. M. Kadziński, S. Greco, and R. Słowiński Selection of a representative value function in robust multiple criteria ranking and choice European Journal of Operational Research 217 2012 541 553
19. A.I. Kraut, A.D. Wolfson, and A. Rothenburg Some effects of position on opinion survey items Journal of Applied Psychology 60 1975 774 776
20. J.A. Krosnick, and D.F. Alwin A test of the form-resistant correlation hypothesis: ratings, rankings and the measurement of values Public Opinion Quarterly 52 1988 526 538
21. P. Linares Are inconsistent decisions better? An experiment with pairwise comparison European Journal of Operational Research 193 2009 492 498
22. N.H. Lurie, and C.H. Mason Visual representation: implications for decision making Journal of Marketing 71 2007 160 177
23. J.A. McCarty, and L.J. Shrum The measurement of personal values in survey research Public Opinion Quarterly 64 2000 271 298
24. A. Morton, and B. Fasolo Behavioural decision theory for multi-criteria decision analysis: a guided tour Journal of the Operational Research Society 60 2009 268 275
25. V.G. Morwitz, and G.J. Pitzsimons The mere measurement effect: why does measuring intentions change actual behaviour? Journal of Consumer Psychology 14 2004 64 74
26. R.M. Musal, R. Soyer, C. McCabe, and S.A. Kharroubi Estimating the population utility function: a parametric Bayesian approach European Journal of Operational Research 218 2012 538 547
27. M.T. Pham, J.B. Cohen, J. Pracejus, and D.G. Hughes Affect monitoring and the primacy of feelings in judgment Journal of Consumer Research 28 2001 167 188
28. M. Pöyhönen, H. Vrolijk, and R. Hämäläinen Behavioral and procedural consequences of structural variation in value trees European Journal of Operational Research 134 2001 216 227
29. T.R. Schori Note on attribute importance Perceptual and Motor Skills 80 1995 1129 1130
30. S. Siraj, L. Mikhailov, and J.A. Keane Preference elicitation from inconsistent judgments using multi-object optimization European Journal of Operational Research 220 2012 461 471
31. I. Simonson Choice based on reasons: the case of attraction and compromise effects Journal of Consumer Research 16 1989 158 174
32. P. Slovic The construction of preference American Psychologist 50 1995 364 371
33. P. Slovic, M.L. Finucane, E. Peters, and D.G. MacGregor The affect heuristic European Journal of Operational Research 177 2007 1333 1352
34. J. Srivastava, T. Connolly, and L.R. Beach Do ranks suffice? A comparison of alternative weighting approaches in value elicitation Organizational Behavior and Human Decision Processes 63 1995 112 116
35. S.S. Stevens Psychophysics: Introduction to its Perceptual, Neural and Social Prospects 1975 John Wiley & Sons New York
36. R. Tourangeau, L.J. Rips, and K. Rasinski The Psychology of Survey Response 2000 Cambridge University Press Cambridge, UK
37. K. Van Ittersum, J.M.E. Pennings, B. Wansink, and H.C.M. van Trijp The validity of attribute-importance measurement: a review Journal of Business Research 60 2007 1177 1190
38. D. von Winterfeldt, and W. Edwards Decision Analysis and Behavioural Research 1986 Cambridge University Press Cambridge
39. B. Wansink, and K. Van Ittersum Bottoms up! The influence of elongation on pouring and consumption volume Journal of Consumer Research 30 2003 455 463
40. M. Weber, and K. Borcherding Influences on weight judgments in multiattribute decision-making European Journal of Operational Research 67 1993 1 12
41. C.H. Yeh, and Y.H. Chang Modeling subjective evaluation for fuzzy group multicriteria decision making European Journal of Operational Research 194 2009 464 473
42. S.H. Zhu, and N.H. Anderson Self-estimation of weight parameters in multi-Attribute analysis Organizational Behavior and Human Decision Processes 48 1991 36 54