Resourceful contextual bandits

Journal of Machine Learning Research

Volumn 35, Issue , 2014, Pages 1109-1134

  Badanidiyuru, Ashwinkumar ^a   Langford, John ^b   Slivkins, Aleksandrs ^b  

^a School of Operations Research and Information Engineering   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

ALGORITHM FOR SOLVING; CONTEXTUAL BANDITS; DYNAMIC PRICING; NEAR-OPTIMAL; REAL-WORLD; RESOURCE CONSTRAINT; STATISTICAL PROPERTIES;

SOFTWARE ENGINEERING;

EID: 84939636813     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (36)

1. Alekh Agarwal, Miroslav Dudik, Satyen Kale, and John Langford. Contextual bandit learning under the realizability assumption. In 15th Intl. Conf. on Artificial Intelligence and Statistics (AISTATS), 2012.
2. Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. Taming the monster: A fast and simple algorithm for contextual bandits. In 31st Intl. Conf. on Machine Learning (ICML), 2013.
3. Shipra Agrawal and Nikhil R. Devanur. Bandits with concave rewards and convex knapsacks. In 15th, 2014.
4. Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. J. of Machine Learning Research (JMLR), 3:397-422, 2002. Preliminary version in 41st IEEE FOCS, 2000.
5. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, 2002. Preliminary version in 36th IEEE FOCS, 1995.
6. Moshe Babaioff, Shaddin Dughmi, Robert Kleinberg, and Aleksandrs Slivkins. Dynamic pricing with limited supply. In 13th ACM Conf. on Electronic Commerce (EC), 2012.
7. Ashwinkumar Badanidiyuru, Robert Kleinberg, and Yaron Singer. Learning on a budget: posted price mechanisms for online procurement. In 13th ACM Conf. on Electronic Commerce (EC), pages 128-145, 2012.
8. Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. In 54th IEEE Symp. on Foundations of Computer Science (FOCS), 2013a.
9. Ashwinkumar Badanidiyuru, Robert Kleinberg, and Aleksandrs Slivkins. Bandits with knapsacks. A technical report on arxiv.org., May 2013b.
10. Omar Besbes and Assaf Zeevi. Dynamic pricing without knowing the demand function: Risk bounds and near-optimal algorithms. Operations Research, 57:1407-1420, 2009.
11. Omar Besbes and Assaf J. Zeevi. Blind network revenue management. Operations Research, 60(6):1537-1550, 2012.
12. Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert E. Schapire. Efficient optimal leanring for contextual bandits. In 14th Intl. Conf. on Artificial Intelligence and Statistics (AISTATS), 2011.
13. Sébastien Bubeck and Nicolo Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multiarmed Bandit Problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012.
14. Sébastien Bubeck and Aleksandrs Slivkins. The best of both worlds: stochastic and adversarial bandits. In 25th Conf. on Learning Theory (COLT), 2012.
15. Nikhil Devanur and Vijay Vazirani. The spending constraint model for market equilibrium: Algorithmic, existence and uniqueness results. In 36th ACM Symp. on Theory of Computing (STOC), 2004.
16. Nikhil R. Devanur and Thomas P. Hayes. The AdWords problem: Online keyword matching with budgeted bidders under random permutations. In 10th ACM Conf. on Electronic Commerce (EC), pages 71-78, 2009.
17. Nikhil R. Devanur, Kamal Jain, Balasubramanian Sivan, and Christopher A. Wilkens. Near optimal online algorithms and fast approximation algorithms for resource allocation problems. In 12th ACM Conf. on Electronic Commerce (EC), pages 29-38, 2011.
18. Miroslav Dudik, Daniel Hsu, Satyen Kale, Nikos Karampatziakis, John Langford, Lev Reyzin, and Tong Zhang. Efficient optimal leanring for contextual bandits. In 27th Conf. on Uncertainty in Artificial Intelligence (UAI), 2011.
19. D. A. Freedman. On tail probabilities for martingales. The Annals of Probability, 3:100-118, 1975.
20. John Gittins, Kevin Glazebrook, and Richard Weber. Multi-Armed Bandit Allocation Indices. John Wiley & Sons, 2011.
21. Sudipta Guha and Kamesh Munagala. Multi-armed Bandits with Metric Switching Costs. In 36th Intl. Colloquium on Automata, Languages and Programming (ICALP), pages 496-507, 2007.
22. Sudipta Guha, Kamesh Munagala, and Peng Shi. Approximation algorithms for restless bandit problems., 2010. Combined final version of papers in IEEE FOCS 2007 and ACM-SIAM SODA 2009.
23. Anupam Gupta, Ravishankar Krishnaswamy, Marco Molinaro, and R. Ravi. Approximation algorithms for correlated knapsacks and non-martingale bandits. In 52nd IEEE Symp. on Foundations of Computer Science (FOCS), pages 827-836, 2011.
24. András György, Levente Kocsis, Ivett Szabó, and Csaba Szepesvári. Continuous time associative bandit problems. In 20th Intl. Joint Conf. on Artificial Intelligence (IJCAI), pages 830-835, 2007.
25. Robert Kleinberg and Aleksandrs Slivkins. Sharp Dichotomies for Regret Minimization in Metric Spaces. In 21st ACM-SIAM Symp. on Discrete Algorithms (SODA), 2010.
26. Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-Armed Bandits in Metric Spaces. In 40th ACM Symp. on Theory of Computing (STOC), pages 681-690, 2008.
27. John Langford and Tong Zhang. The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits. In 21st Advances in Neural Information Processing Systems (NIPS), 2007.
28. Brendan McMahan and Matthew Streeter. Tighter bounds for multi-armed bandits with expert advice. In 22nd Conf. on Learning Theory (COLT), 2009.
29. Adish Singla and Andreas Krause. Truthful incentives in crowdsourcing tasks using regret minimization mechanisms. In 22nd Intl. World Wide Web Conf. (WWW), pages 1167-1178, 2013.
30. Maurice Sion. On general minimax theorems. Pac. J. Math., 8:171176, 1958.
31. Aleksandrs Slivkins. Contextual Bandits with Similarity Information. In 24th Conf. on Learning Theory (COLT), 2011. To appear in J. of Machine Learning Research (JMLR), 2014.
32. Aleksandrs Slivkins. Dynamic ad allocation: Bandits with budgets. A technical report on arxiv.org/abs/1306.0155, June 2013.
33. Aleksandrs Slivkins and Jennifer Wortman Vaughan. Online decision making in crowdsourcing markets: Theoretical challenges. SIGecom Exchanges, 12(2), December 2013. Position Paper and survey.
34. William R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285294, 1933.
35. Long Tran-Thanh, Archie Chapman, Enrique Munoz De Cote, Alex Rogers, and Nicholas R. Jennings. e-first policies for budget-limited multi-armed bandits. In 24th AAAI Conference on Artificial Intelligence (AAAI), pages 1211-1216, 2010.
36. Long Tran-Thanh, Archie Chapman, Alex Rogers, and Nicholas R. Jennings. Knapsack based optimal policies for budget-limited multi-armed bandits. In 26th AAAI Conference on Artificial Intelligence (AAAI), pages 1134-1140, 2012.