Eliciting beliefs: Proper scoring rules, incentives, stakes and hedging

European Economic Review

Volumn 62, Issue , 2013, Pages 17-40

  Armantier, Olivier ^a   Treich, Nicolas ^b  

^a FEDERAL RESERVE BANK OF NEW YORK   (United States)

^b TOULOUSE SCHOOL OF ECONOMICS   (France)

Author keywords

Belief elicitation; Experimental economics; Scoring rules

Indexed keywords

BET-HEDGING; DECISION MAKING; ECONOMIC THEORY; FINANCIAL SYSTEM; PROBABILITY; RISK ASSESSMENT;

EID: 84878505207     PISSN: 00142921     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.euroecorev.2013.03.008     Document Type: Article

References (51)

1. Abramowicz A. The hidden beauty of the quadratic scoring market rule. a uniform liquidity market maker, with variations. The Journal of Prediction Markets 2007, 2:111-125.
2. Allen F. Discovering personal probabilities when utility functions are unknown. Management Science 1987, 33:542-544.
3. Andersen, S., Fountain, J., Harrison, G.W., Rutström, E.E., 2010. Estimating subjective probabilities, Mimeo.
4. Armantier O., Boly A. A controlled field experiment on corruption. European Economic Review 2011, 55(8):1072-1082.
5. Armantier, O., Boly, A., 2012. Can Corruption be Studied in the Lab? Working Paper, Université de Montréal.
6. Armantier O., Treich N. Subjective probabilities in games. an application to the overbidding puzzle. International Economic Review 2009, 50:1079-1102.
7. Armantier, O., Treich, N., 2011. Eliciting Beliefs: Proper Scoring Rules, Incentives, Stakes and Hedging, Working Paper.
8. Blanco M.D., Engelmann D., Koch A.K., Normann H.-T. Belief elicitation in experiments. is there a hedging problem?. Experimental Economics 2010, 13(4):412-438.
9. Camerer C.F. Individual decision making. Handbook of Experimental Economics 1995, 587-703. Princeton University Press, Princeton, NJ. J.H. Kagel, A.E. Roth (Eds.).
10. Camerer C.F., Hogarth R.M. The effects of financial incentives in experiments. a review and capital-labor-production framework. Journal of Risk and Uncertainty 1999, 19:7-42.
11. Chen Y., Pennock D. Designing markets for prediction. AI Magazine 2010, 31(4):42-52.
12. Costa-Gomes M.A., Weizsäcker G. Stated beliefs and play in normal-form games. Review of Economic Studies 2008, 75:729-762.
13. Croson R. Thinking like a game theorist. factors affecting the frequency of equilibrium play. Journal of Economic Behavior and Organization 2000, 41:299-314.
14. Davis D., Holt C. Experimental Economics 1993, Princeton University Press, Princeton, NJ.
15. Fischbacher U., Gächter S. Social preferences, beliefs, and the dynamics of free riding in public good experiments. American Economic Review 2010, 100(1):541-556.
16. Gächter S., Renner E. The effects of (incentivized) belief elicitation in public good experiments. Experimental Economics 2010, 13:364-377.
17. von Gaudecker H.-M., van Soest A., Wengström E. Heterogeneity in risky choice behaviour in a broad population. American Economic Review 2011, 101:664-694.
18. Gneiting T., Raftery A. Strictly proper scoring rules, prediction and estimation. Journal of the American Statistical Association 2007, 102:359-378.
19. Goel, S., Reeves, D., Pennock, D., 2009. Collective revelation: a mechanism for self-verified, weighted, and truthful predictions. In: Proceedings of the Tenth ACM Conference on Electronic Commerce (EC '09), pp. 265-274.
20. Hanson R. Combinatorial information market design. Information Systems Frontiers 2003, 5:107-119.
21. Hanson R. The policy analysis market. a thwarted experiment in the use of prediction markest for public policy. Innovations 2007, 2(3):73-88.
22. Hao L., Houser D. Belief elicitation in the presence of novice participants. an experimental study. Journal of Risk and Uncertainty 2012, 44:161-180.
23. Healy P., Linardi S., Lowery R., Ledyard J. Prediction markets. alternative mechanisms for complex environments with few traders. Management Science 2010, 56:1977-1996.
24. Hollard, G., Massoni, S., Vergnaud, J.-C., 2010. Direct or Indirect Elicitation of Subjective Beliefs? CES Working paper, Universite Paris 1.
25. Holt C., Laury S. Risk aversion and incentive effects. American Economic Review 2002, 92:1644-1655.
26. Jaffray J.Y., Karni E. Elicitation of subjective probabilities when initial endowment is unobservable. Journal of Risk and Uncertainty 1999, 8:5-20.
27. Kadane J.B., Winkler R.L. Separating probability elicitation from utilities. Journal of the American Statistical Association 1988, 83:357-363.
28. Karni E. Elicitation of subjective probabilities when preference are state-dependent. International Economic Review 1999, 40:479-486.
29. Karni E. A mechanism for eliciting probabilities. Econometrica 2009, 77:603-606.
30. Karni E., Safra Z. The impossibility of experimental elicitation of subjective probabilities. Theory and Decision 1995, 38:313-320.
31. Kothiyal A., Spinu V., Wakker P.P. Comonotonic proper scoring rules to measure ambiguity and subjective beliefs. Journal of Multi-Criteria Decision Analysis 2011, 17:101-113.
32. Ledyard J., Hanson R., Ishikida T. An experimental test of combinatorial information markets. Journal of Economic Behavior and Organization 2009, 69:182-189.
33. Manski C. Measuring expectations. Econometrica 2004, 72:1329-1376.
34. McKelvey R.D., Page T. Public and private information. an experimental study of information pooling. Econometrica 1990, 58:1321-1339.
35. Miller N., Resnick P., Zeckhauser R. Eliciting informative feedback. the peer-prediction method. Management Science 2005, 51:1359-1373.
36. Noussair, C.N., Trautmann, S.T., van de Kuilen, G., 2011. Higher Order Risk Attitudes, Demographics, and Financial Decisions. CentER Discussion Paper 2011-055.
37. Nyarko Y., Schotter A. An experimental study of belief learning using elicited beliefs. Econometrica 2002, 70:971-1005.
38. Offerman T., Sonnemans J., van de Kuilen G., Wakker P.P. A truth-serum for non-Bayesians. correcting proper scoring rules for risk attitudes. Review of Economics Studies 2009, 76:1461-1489.
39. Palfrey T.R., Wang S.W. On eliciting beliefs in strategic games. Journal of Economic Behavior and Organization 2009, 71(2):98-109.
40. Prelec D. The probability weighting function. Econometrica 1998, 66:497-527.
41. Pratt J.W. Risk aversion in the small and in the large. Econometrica 1964, 32:122-136.
42. Read D. Monetary incentives, what are they good for?. Journal of Economic Methodology 2005, 12:265-276.
43. Roth A.E., Malouf M.W. Game-theoretic models and the role of information in bargaining. Psychological Review 1979, 86:574-594.
44. Rutström E., Wilcox N. Stated beliefs versus inferred beliefs. a methodological inquiry and experimental test. Games and Economic Behavior 2009, 67(2):616-632.
45. Savage L.J. Elicitation of personal probabilities and expectation. Journal of the American Statistical Association 1971, 66:783-801.
46. Schervish M.J. A general method for comparing probability assessors. The Annals of Statistics 1989, 17:1856-1879.
47. Selten R., Sadrieh A., Abbink K. Money does not induce risk neutral behavior, but binary lotteries do even worse. Theory and Decision 1999, 46:211-249.
48. Sonnemans, J., Offerman, T., 2004. Is the quadratic scoring rule really incentive compatible?, Mimeo, University of Amsterdam.
49. Trautmann, S., van de Kuilen, G., 2011. Beliefs Elicitation: A Horse Race among Truth Serums. Working Paper.
50. Wilcox, N., 2010. A Comparison of Three Probabilistic Models of Binary Discrete Choice under Risk. Working Paper.
51. Winkler R.L., Murphy A.H. Nonlinear utility and the probability score. Journal of Applied Meteorology 1970, 9:143-148.