Adaptive contract design for crowdsourcing markets: Bandit algorithms for repeated principal-agent problems

Journal of Artificial Intelligence Research

Volumn 55, Issue , 2016, Pages 317-359

  Ho, Chien Ju ^a   Slivkins, Aleksandrs ^b   Vaughan, Jennifer Wortman ^b  

^a Department of Obstetrics Gynecology   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BUDGET CONTROL; COMMERCE; COSTS;

CONTINGENT PAYMENTS; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; POPULAR PLATFORM; PRICING PROBLEMS; PRINCIPAL AGENT MODELS; PRINCIPAL-AGENT PROBLEMS; STRATEGIC CHOICE;

CROWDSOURCING;

EID: 84958047672     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.4940     Document Type: Article

References (60)

1. Agrawal, R. (1995). The continuum-armed bandit problem. SIAM J. Control and Opti-mization, 33 (6), 1926-1951.
2. Agrawal, S., & Devanur, N. R. (2014). Bandits with concave rewards and convex knapsacks. In 15th ACM Conf. on Economics and Computation (EC).
3. Agrawal, S., Devanur, N. R., & Li, L. (2015). Contextual bandits with global constraints and objective.. Technical report, arXiv:1506.03374.
4. Antos, A., Bartók, G., Pál, D., & Szepesvári, C. (2013). Toward a classification of finite partial-monitoring games. Theor. Comput. Sci., 473, 77-99.
5. Audibert, J., & Bubeck, S. (2010). Regret Bounds and Minimax Policies under Partial Monitoring. J. of Machine Learning Research (JMLR), 11, 2785-2836.
6. Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multiarmed bandit problem.. Machine Learning, 47 (2-3), 235-256.
7. Auer, P., Ortner, R., & Szepesvári, C. (2007). Improved Rates for the Stochastic Continuum-Armed Bandit Problem. In 20th Conf. on Learning Theory (COLT), pp. 454-468.
8. Babaioff, M., Dughmi, S., Kleinberg, R. D., & Slivkins, A. (2015). Dynamic pricing with limited supply. ACM Trans. on Economics and Computation, 3 (1), 4.
9. Babaioff, M., Feldman, M., & Nisan, N. (2006). Combinatorial agency. In 7th ACM Conf. on Electronic Commerce (EC).
10. Badanidiyuru, A., Kleinberg, R., & Singer, Y. (2012). Learning on a budget: posted price mechanisms for online procurement. In 13th ACM Conf. on Electronic Commerce (EC), pp. 128-145.
11. Badanidiyuru, A., Kleinberg, R., & Slivkins, A. (2013). Bandits with knapsacks. In 54th IEEE Symp. on Foundations of Computer Science (FOCS).
12. Badanidiyuru, A., Langford, J., & Slivkins, A. (2014). Resourceful contextual bandits. In 27th Conf. on Learning Theory (COLT).
13. Bartók, G., Foster, D. P., Pál, D., Rakhlin, A., & Szepesvári, C. (2014). Partial monitoring-classification, regret bounds, and algorithms. Math. Oper. Res., 39 (4), 967-997.
14. Besbes, O., & Zeevi, A. (2009). Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research, 57, 1407-1420.
15. Besbes, O., & Zeevi, A. J. (2012). Blind network revenue management. Operations Research, 60 (6), 1537-1550.
16. Blum, A., Kumar, V., Rudra, A., & Wu, F. (2003). Online learning in online auctions. In 14th ACM-SIAM Symp. on Discrete Algorithms (SODA), pp. 202-204.
17. Bohren, J. A., & Kravitz, T. (2013). Incentives for spot market labor when output is unverifiable. Working paper.
18. Bubeck, S., & Cesa-Bianchi, N. (2012). Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Foundations and Trends in Machine Learning, 5 (1), 1-122.
19. Bubeck, S., Munos, R., Stoltz, G., & Szepesvari, C. (2011a). Online Optimization in XArmed Bandits. J. of Machine Learning Research (JMLR), 12, 1587-1627.
20. Bubeck, S., Stoltz, G., & Yu, J. Y. (2011b). Lipschitz bandits without the lipschitz constant. In 22nd Intl. Conf. on Algorithmic Learning Theory (ALT), pp. 144-158.
21. Bull, A. D. (2013). Adaptive-treed bandits. Tech. rep. 1302.2489, arxiv.org.
22. Cesa-Bianchi, N., & Lugosi, G. (2006). Prediction, learning, and games. Cambridge Univ. Press.
23. Conitzer, V., & Garera, N. (2006). Online learning algorithms for online principal-agent problems (and selling goods online). In International Conference on Machine Learning (ICML).
24. den Boer, A. V. (2015). Dynamic pricing and learning: Historical origins, current research, and new directions. Surveys in Operations Research and Management Science. Forthcoming.
25. Dudik, M., Hsu, D., Kale, S., Karampatziakis, N., Langford, J., Reyzin, L., & Zhang, T. (2011). Efficient optimal leanring for contextual bandits. In 27th Conf. on Uncertainty in Artificial Intelligence (UAI).
26. Garivier, A., & Cappé, O. (2011). The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond. In 24th Conf. on Learning Theory (COLT).
27. Ghosh, A., & Hummel, P. (2011). A game-theoretic analysis of rank-order mechanisms for user-generated content. In 12th ACM Conf. on Electronic Commerce (EC).
28. Ghosh, A., & Hummel, P. (2013). Learning and incentives in user-generated content: Multiarmed bandits with endogenous arms. In Proc. 4th Conference on Innovations in Theoretical Computer Science (ITCS).
29. Ghosh, A., & McAfee, P. (2011). Incentivizing high-quality user-generated content. In 20th Intl. World Wide Web Conf. (WWW).
30. Gittins, J., Glazebrook, K., & Weber, R. (2011). Multi-Armed Bandit Allocation Indices. John Wiley & Sons.
31. Harris, C. G. (2011). You're hired! an examination of crowdsourcing incentive models in human resource tasks. In CSDM.
32. Ho, C., Slivkins, A., Suri, S., & Vaughan, J. W. (2015). Incentivizing high quality crowdwork. In 24th Intl. World Wide Web Conf. (WWW).
33. Ho, C.-J., Zhang, Y., Vaughan, J. W., & van der Schaar, M. (2012). Towards social norm design for crowdsourcing markets. In HCOMP.
34. Horton, J. J., & Chilton, L. B. (2010). The labor economics of paid crowdsourcing. In 11th ACM Conf. on Electronic Commerce (EC).
35. Jain, S., Chen, Y., & Parkes, D. (2012). Designing incentives for online question-and-answer forums. Games and Economic Behavior.
36. Kleinberg, R. (2004). Nearly tight bounds for the continuum-armed bandit problem. In 18th Advances in Neural Information Processing Systems (NIPS).
37. Kleinberg, R., & Leighton, T. (2003). The value of knowing a demand curve: Bounds on regret for online posted-price auctions.. In 44th IEEE Symp. on Foundations of Computer Science (FOCS), pp. 594-605.
38. Kleinberg, R., Slivkins, A., & Upfal, E. (2008). Multi-armed bandits in metric spaces. In 40th ACM Symp. on Theory of Computing (STOC), pp. 681-690.
39. Kleinberg, R. D., & Leighton, F. T. (2003). The value of knowing a demand curve: Bounds on regret for online posted-price auctions. In IEEE Symp. on Foundations of Computer Science (FOCS).
40. Kocsis, L., & Szepesvari, C. (2006). Bandit Based Monte-Carlo Planning. In 17th European Conf. on Machine Learning (ECML), pp. 282-293.
41. Laffont, J.-J., & Martimort, D. (2002). The Theory of Incentives: The Principal-Agent Model. Princeton University Press.
42. Lai, T. L., & Robbins, H. (1985). Asymptotically efficient Adaptive Allocation Rules. Advances in Applied Mathematics, 6, 4-22.
43. Levy, A., & Vukina, T. (2002). Optimal linear contracts with heterogeneous agents. In European Review of Agricultural Economics.
44. Mason, W., & Watts, D. (2009). Financial incentives and the "performance of crowds". In HCOMP.
45. Misra, S., Nair, H. S., & Daljord, O. (2012). Homogenous contracts for heterogeneous agents: Aligning salesforce composition and compensation. Working Paper.
46. Munos, R., & Coquelin, P.-A. (2007). Bandit algorithms for tree search. In 23rd Conf. on Uncertainty in Artificial Intelligence (UAI).
47. Pandey, S., Agarwal, D., Chakrabarti, D., & Josifovski, V. (2007). Bandits for Taxonomies: A Model-based Approach. In SIAM Intl. Conf. on Data Mining (SDM).
48. Radlinski, F., Kleinberg, R., & Joachims, T. (2008). Learning diverse rankings with multiarmed bandits. In 25th Intl. Conf. on Machine Learning (ICML), pp. 784-791.
49. Sannikov, Y. (2008). A continuous-time version of the principal-agent problem. In The Review of Economics Studies.
50. Sannikov, Y. (2012). Contracts: The theory of dynamic principal-agent relationships and the continuous-time approach. In 10th World Congress of the Econometric Society.
51. Singer, Y., & Mittal, M. (2013). Pricing mechanisms in crowdsourcing markets. In 22nd Intl. World Wide Web Conf. (WWW).
52. Singla, A., & Krause, A. (2013). Truthful incentives in crowdsourcing tasks using regret minimization mechanisms. In 22nd Intl. World Wide Web Conf. (WWW).
53. Slivkins, A. (2011). Multi-armed bandits on implicit metric spaces. In 25th Advances in Neural Information Processing Systems (NIPS).
54. Slivkins, A. (2014). Contextual bandits with similarity information. J. of Machine Learning Research (JMLR), 15 (1), 2533-2568. Preliminary version in COLT 2011.
55. Slivkins, A., Radlinski, F., & Gollapudi, S. (2013). Ranked bandits in metric spaces: Learning optimally diverse rankings over large document collections. J. of Machine Learning Research (JMLR), 14 (Feb), 399-436. Preliminary version in 27th ICML, 2010.
56. Thompson, W. R. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.. Biometrika, 25 (3-4), 285-294.
57. Wang, Z., Deng, S., & Ye, Y. (2014). Close the gaps: A learning-while-doing algorithm for single-product revenue management problems. Operations Research, 62 (2), 318-331.
58. Williams, N. (2009). On dynamic principal-agent problems in continuous time. Working Paper.
59. Yin, M., Chen, Y., & Sun, Y.-A. (2013). The effects of performance-contingent financial incentives in online labor markets. In AAAI.
60. Zhang, Y., & van der Schaar, M. (2012). Reputation-based incentive protocols in crowdsourcing applications. In Infocom.