Elicitability and knowledge-free elicitation with peer prediction

13th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2014

Volumn 1, Issue , 2014, Pages 245-252

  Zhang, Peter ^a   Chen, Yiling ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

Information elicitation; Peer prediction; Proper scoring rules

Indexed keywords

EID: 84911445631     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (20)

1. G. W. Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1): l-3, 1950.
2. A. Dasgupta and A. Ghosh. Crowdsourced judgement elicitation with endogenous proficiency. In Proceedings of the 22nd international conference on World Wide Web, pages 319-330. International World Wide Web Conferences Steering Committee, 2013.
3. T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-378, 2007.
4. R. Jurca and B. Faltings. Minimum payments that reward honest reputation feedback. In Proceedings of the 7th ACM conference on Electronic commerce, EC '06, pages 190-199. ACM, 2006.
5. R. Jurca and B. Faltings. Collusion-resistant, incentive-compatible feedback payments. In Proceedings of the 8th ACM conference on Electronic commerce, EC '07, pages 200-209. ACM, 2007.
6. R. Jurca and B. Faltings. Mechanisms for making crowds truthful. Journal of Artificial Intelligence Research, 34(1):209, 2009.
7. N. Lambert, D. M. Pennock, and Y. Shoham. Eliciting properties of probability distributions. In EC '08: Proceedings of the 9th ACM conference on Electronic commerce, pages 129-138, New York, NY, USA, 2008. ACM.
8. J. E. Matheson and R. L. Winkler. Scoring rules for continuous probability distributions. Management Science, 22(10):1087-1096, 1976.
9. N. Miller, P. Resnick, and R. Zeckhauser. Eliciting informative feedback: The peer-prediction method. Management Science, 51(9):1359-1373, 2005.
10. R. F. Nau. Should scoring rules be 'effective'. Management Science, 31:527-535, 1985.
11. R. G. Nelson and D. A. Bessler. Subjective probabilities and scoring rules: Experimental evidence. Journal of Agricultural Economics, 71(2):363-369, 1989.
12. D. Prelec. A bayesian truth serum for subjective data. Science, 306(5695):462-466, 2004.
13. G. Radanovic and B. Faltings. A robust bayesian truth serum for non-binary signals. In Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI '13, 2013.
14. L. J. Savage. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66(336):783-801, 1971.
15. L. von Ahn and L. Dabbish. Designing games with a purpose. Communications of the ACM, 51(8):58-67, 2008.
16. B. Waggoner and Y. Chen. Information elicitation sans verification. In The ACM EC Workshop on Social Computing and User Generated Content, 2013.
17. R. L. Winkler. Scoring rules and the evaluation of probability assessors. Journal of the American Statistical Association, 64(327):1073-1078, 1969.
18. J. Witkowski, Y. Bachrach, P. Key, and D. C. Parkes. Dwelling on the negative: Incentivizing effort in peer prediction. In Proceedings of the 1st AAAI Conference on Human Computation and Crowdsourcing, HCOMP'13, 2013.
19. J. Witkowski and D. Parkes. Peer prediction without a common prior. In Proceedings of the 13th ACM Conference on Electronic Commerce, EC '12, pages 964-981. ACM, 2012.
20. J. Witkowski and D. Parkes. A robust bayesian truth serum for small populations. In Proceedings of the 26th AAAI Conference on Artificial Intelligence, AAAI '12, 2012.