Bandits with knapsacks (extended abstract)

Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

Volumn , Issue , 2013, Pages 207-216

  Badanidiyuru, Ashwinkumar ^a   Kleinberg, Robert ^a   Slivkins, Aleksandrs ^b  

^a Department of Computer Science and School of Operations Research and Information Engineering   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Dynamic ad allocation; Dynamic pricing; Dynamic procurement; Exploration exploitation tradeoff; Multi armed bandits; Regret; Stochastic packing

Indexed keywords

DYNAMIC PRICING; EXPLORATION EXPLOITATIONS; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; POLY-LOGARITHMIC FACTORS; PRIMAL DUAL ALGORITHMS; REGRET; STOCHASTIC INTEGER PROGRAMMING;

COMPUTER SCIENCE; INTEGER PROGRAMMING; OPTIMIZATION; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

ALGORITHMS;

EID: 84893451322     PISSN: 02725428     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FOCS.2013.30     Document Type: Conference Paper

References (33)

1. P. Whittle, "Multi-armed bandits and the Gittins index, " J. Royal Statistical Society, Series B, vol. 42, no. 2, pp. 143- 149, 1980.
2. C. H. Papadimitriou and J. N. Tsitsiklis, "The complexity of optimal queuing network control, " Math. Oper. Res., vol. 24, no. 2, pp. 293-305, 1999.
3. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire, "The nonstochastic multiarmed bandit problem." SIAM J. Comput., vol. 32, no. 1, pp. 48-77, 2002, preliminary version in 36th IEEE FOCS, 1995.
4. O. Besbes and A. J. Zeevi, "Blind network revenue management, " Operations Research, vol. 60, no. 6, pp. 1537-1550, 2012.
5. A. Badanidiyuru, R. Kleinberg, and Y. Singer, "Learning on a budget: posted price mechanisms for online procurement, " in 13th ACM EC, 2012, pp. 128-145.
6. I. Abraham, O. Alonso, V. Kandylas, and A. Slivkins, "Adaptive crowdsourcing algorithms for the bandit survey problem, " in 26th COLT, 2013.
7. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem." Machine Learning, vol. 47, no. 2-3, pp. 235-256, 2002, preliminary version in 15th ICML, 1998.
8. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm, " Information and Computation, vol. 108, no. 2, pp. 212-260, 1994.
9. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting, " Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
10. O. Besbes and A. Zeevi, "Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms, " Operations Research, vol. 57, pp. 1407-1420, 2009.
11. M. Babaioff, S. Dughmi, R. Kleinberg, and A. Slivkins, "Dynamic pricing with limited supply, " in 13th ACM EC, 2012.
12. A. Blum, V. Kumar, A. Rudra, and F. Wu, "Online learning in online auctions, " in 14th ACM-SIAM SODA, 2003, pp. 202- 204.
13. R. Kleinberg and T. Leighton, "The value of knowing a demand curve: Bounds on regret for online posted-price auctions." in 44th IEEE FOCS, 2003, pp. 594-605.
14. A. Singla and A. Krause, "Truthful incentives in crowdsourcing tasks using regret minimization mechanisms, " in 22nd WWW, 2013, pp. 1167-1178.
15. V. Dani, T. P. Hayes, and S. Kakade, "Stochastic Linear Optimization under Bandit Feedback, " in 21th COLT, 2008, pp. 355-366.
16. R. Kleinberg, A. Slivkins, and E. Upfal, "Multi-Armed Bandits in Metric Spaces, " in 40th ACM STOC, 2008, pp. 681- 690.
17. T. L. Lai and H. Robbins, "Asymptotically efficient adaptive allocations rules, " Adv. in Appl. Math., vol. 6, pp. 4-22, 1985.
18. S. Bubeck and N. Cesa-Bianchi, "Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, " Foundations and Trends in Machine Learning, vol. 5, no. 1, pp. 1-122, 2012.
19. S. Guha and K. Munagala, "Multi-armed Bandits with Metric Switching Costs, " in 36th ICALP, 2007, pp. 496-507.
20. A. Gupta, R. Krishnaswamy, M. Molinaro, and R. Ravi, "Approximation algorithms for correlated knapsacks and nonmartingale bandits, " in 52nd IEEE FOCS, 2011, pp. 827-836.
21. L. Tran-Thanh, "Budget-limited multi-armed bandits, " Ph.D. dissertation, University of Southampton, 2012.
22. L. Tran-Thanh, A. Chapman, E. M. de Cote, A. Rogers, and N. R. Jennings, "ε-first policies for budget-limited multiarmed bandits, " in 24th AAAI, 2010, pp. 1211-1216.
23. L. Tran-Thanh, A. Chapman, A. Rogers, and N. R. Jennings, "Knapsack based optimal policies for budget-limited multiarmed bandits, " in 26th AAAI, 2012, pp. 1134-1140.
24. N. Cesa-Bianchi, C. Gentile, and Y. Mansour, "Regret minimization for reserve prices in second-price auctions, " in ACMSIAM SODA, 2013.
25. N. R. Devanur and T. P. Hayes, "The AdWords problem: Online keyword matching with budgeted bidders under random permutations, " in 10th ACM EC, 2009, pp. 71-78.
26. S. Agrawal, Z. Wang, and Y. Ye, "A dynamic nearoptimal algorithm for online linear programming, " 2009, technical report. Available from arXiv at http://arxiv.org/abs/0911.2974.
27. N. R. Devanur, K. Jain, B. Sivan, and C. A. Wilkens, "Near optimal online algorithms and fast approximation algorithms for resource allocation problems, " in 12th ACM EC, 2011, pp. 29-38.
28. J. Feldman, M. Henzinger, N. Korula, V. S. Mirrokni, and C. Stein, "Online stochastic packing applied to display ad allocation, " in 18th ESA, 2010, pp. 182-194.
29. M. Molinaro and R. Ravi, "Geometry of online packing linear programs, " in 39th ICALP, 2012, pp. 701-713.
30. S. Arora, E. Hazan, and S. Kale, "The multiplicative weights update method: A meta-algorithm and applications, " Theory of Computing, vol. 8, no. 1, pp. 121-164, 2012.
31. N. Garg and J. Könemann, "Faster and simpler algorithms for multicommodity flow and other fractional packing problems, " SIAM J. Computing, vol. 37, no. 2, pp. 630-652, 2007.
32. S. A. Plotkin, D. B. Shmoys, and E. Tardos, "Fast approximation algorithms for fractional packing and covering problems, " Mathematics of Operations Research, vol. 20, pp. 257-301, 1995.
33. S. Pandey, D. Agarwal, D. Chakrabarti, and V. Josifovski, "Bandits for Taxonomies: A Model-based Approach, " in SDM, 2007.