Combinatorial multi-armed bandit and its extension to probabilistically triggered arms

Journal of Machine Learning Research

Volumn 17, Issue , 2016, Pages

  Chen, Wei ^a   Wang, Yajun ^a   Yuan, Yang ^b   Wang, Qinshi ^c  

^a MICROSOFT   (United States)

^b Cornell University ^*  (United States)

^c TSINGHUA UNIVERSITY   (China)

Author keywords

Combinatorial multi armed bandit; Online advertising; Online learning; Social influence maximization; Upper confidence bound

Indexed keywords

ALGORITHMS; APPROXIMATION ALGORITHMS; E-LEARNING; ECONOMIC AND SOCIAL EFFECTS; LEARNING ALGORITHMS; MARKETING;

MULTI ARMED BANDIT; ONLINE ADVERTISING; ONLINE LEARNING; SOCIAL INFLUENCE; UPPER CONFIDENCE BOUND;

PROBABILITY DISTRIBUTIONS;

EID: 84979937519     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. Rajeev Agrawal. The continuum-armed bandit problem. SIAM J. Control Optim., 33(6):1926-1951, 1995.
2. Venkatachalam Anantharam, Pravin Varaiya, and Jean Walrand. Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays - Part I: i.i.d. rewards. IEEE Transactions on Automatic Control, AC-32(11):968-976, 1987.
3. Jean-Yves Audibert, Sébastien Bubeck, and Gábor Lugosi. Minimax policies for adversarial and stochastic bandits. In Proceedings of the 22nd Annual Conference on Learning Theory (COLT), 2009.
4. Jean-Yves Audibert, Sébastien Bubeck, and Gábor Lugosi. Minimax policies for combinatorial prediction games. In Proceedings of the 24th Annual Conference on Learning Theory (COLT), 2011.
5. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002a.
6. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, 2002b.
7. Donald A. Berry and Bert Fristedt. Bandit problems: Sequential Allocation of Experiments. Chapman and Hall, 1985.
8. Sébastien Bubeck, Nicolò Cesa-Bianchi, and Sham M. Kakade. Towards minimax policies for online linear optimization with bandit feedback. In Proceedings of the 25th Annual Conference on Learning Theory (COLT), 2012.
9. Felipe Caro and Jérémie Gallien. Dynamic assortment with demand learning for seasonal consumer goods. Management Science, 53:276-292, 2007.
10. Nicolò Cesa-Bianchi and Gábor Lugosi. Combinatorial bandits. In Proceedings of the 22nd Conference on Learning Theory, 2009.
11. Wei Chen, Yajun Wang, and Yang Yuan. Combinatorial multi-armed bandit: General framework, results, and applications. In Proceedings of the 30th International Conference on Machine Learning (ICML), 2013.
12. Yi Gai, Bhaskar Krishnamachari, and Rahul Jain. Learning multiuser channel allocations in cognitive radio networks: A combinatorial multi-armed bandit formulation. In Proceedings of IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks (DySPAN), 2010.
13. Yi Gai, Bhaskar Krishnamachari, and Rahul Jain. Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations. IEEE/ACM Transactions on Networking, 20, 2012.
14. Aurélien Garivier and Olivier Cappé. The KL-UCB algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th Annual Conference on Learning Theory (COLT), 2011.
15. Aditya Gopalan, Shie Mannor, and Yishay mansour. Thompson sampling for complex online problems. In Proceedings of the 31st International Conference on Machine Learning (ICML), 2014.
16. Elad Hazan and Satyen Kale. Online submodular minimization. In Proceedings of the 23rd Annual Conference on Neural Information Processing Systems (NIPS), 2009.
17. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
18. Sham M. Kakade, Adam Tauman Kalai, and Katrina Ligett. Playing games with approximation algorithms. SIAM Journal on Computing, 39(3):1088-1106, 2009.
19. David Kempe, Jon M. Kleinberg, and Éva Tardos. Maximizing the spread of influence through a social network. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 137-146, 2003.
20. Robert Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-armed bandits in metric spaces. In ACM Symposium on Theory of Computing (STOC), 2008.
21. Robert D. Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In Proceedings of the 17th Annual Conference on Neural Information Processing Systems (NIPS), 2004.
22. Branislav Kveton, Zheng Wen, Azin Ashkan, Hoda Eydgahi, and Brian Eriksson. Matroid bandits: Fast combinatorial optimization with learning. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence (UAI), 2014.
23. Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvári. Tight regret bounds for stochastic combinatorial semi-bandits. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, 2015. to appear, with arxiv version arXiv:1410.0949.
24. Tze Leung Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
25. Tian Lin, Bruno Abrahao, Robert Kleinberg, John C. S. Lui, and Wei Chen. Combinatorial partial monitoring game with linear feedback and its applications. In Proceedings of the 31st International Conference on Machine Learning (ICML), 2014.
26. Haoyang Liu, Keqin Liu, and Qing Zhao. Logarithmic weak regret of non-bayesian restless multi-armed bandit. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2011.
27. Keqin Liu and Qing Zhao. Adaptive shortest-path routing under unknown and stochastically varying link states. In Proceedings of the 10th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2012.
28. Shie Mannor and Ohad Shamir. From bandits to experts: On the value of side-observations. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), 2011.
29. Michael Mitzenmacher and Eli Upfal. Probability and Computing. Cambridge University Press, 2005.
30. G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of the approximations for maximizing submodular set functions. Mathematical Programming, 14(1):265-294, 1978.
31. Lijing Qin, Shouyuan Chen, and Xiaoyan Zhu. Contextual combinatorial bandit and its application on diversified online recommendation. In Proceedings of the 2014 SIAM International Conference on Data Mining (SDM), 2014.
32. Filip Radlinski, Robert Kleinberg, and Thorsten Joachims. Learning diverse rankings with multi-armed bandits. In Proceedings of the 25th International Conference on Machine learning (ICML), 2008.
33. Matthew Streeter and Daniel Golovin. An online algorithm for maximizing submodular functions. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS), 2008.
34. Matthew Streeter, Daniel Golovin, and Andreas Krause. Online learning of assignments. In Proceedings of the 23rd Annual Conference on Neural Information Processing Systems (NIPS), 2009.
35. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.
36. Vijay V. Vazirani. Approximation Algorithms. Springer, 2004.