Learning to trade off between exploration and exploitation in multiclass bandit prediction

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2011, Pages 204-212

  Valizadegan, Hamed ^a   Jin, Rong ^b   Wang, Shijun ^c  

^a Learning Research and Development Ctr   (United States)

^b Michigan State University   (United States)

^c NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

Bandit feedback; Exploration vs. exploitation; Multi class classification; Online learning

Indexed keywords

DATA MINING; E-LEARNING; LEARNING ALGORITHMS; LEARNING SYSTEMS; PARAMETER ESTIMATION;

BANDIT FEEDBACKS; EXPLOITATION; EXPLORATION AND EXPLOITATION; EXPLORATION VS.; LEARNING PROBLEM; MULTI-CLASS CLASSIFICATION; ONLINE LEARNING; PARTIAL FEEDBACK; TRADE OFF; TRADEOFF PARAMETERS;

ECONOMIC AND SOCIAL EFFECTS;

EID: 80052674910     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2020408.2020445     Document Type: Conference Paper

References (19)

1. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002. (Pubitemid 34126111)
2. Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert E. Schapire. An optimal high probability algorithm for the contextual bandit problem. Computational Research Repository, abs/1002.4058, 2010.
3. N. Cesa-Bianchi and G. Lugosi. Prediction, learning, and games. Cambridge Univ Pr, 2006.
4. Chih-Chung Chang and Chih-Jen Lin. Libsvm : a library for support vector machines, 2001.
5. Eyal Even-Dar, Shie Mannor, and Yishay Mansour. PAC bounds for multi-armed bandit and markov decision processes. In COLT'02: Proceedings of the 15th Annual Conference on Computational Learning Theory, pages 255-270, 2002.
6. Eyal Even-Dar, Shie Mannor, and Yishay Mansour. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. Journal of Machine Learning Research, 7:1079-1105, 2006. (Pubitemid 43938989)
7. A. Frank and A. Asuncion. UCI machine learning repository, 2010.
8. Sham M. Kakade, Shai Shalev-Shwartz, and Ambuj Tewari. Efficient bandit algorithms for online multiclass prediction. In ICML 2008: Proceedings of the 25th international conference on Machine learning, pages 440-447, 2008.
9. John Langford and Zhang Tong. The epoch-greedy algorithm for contextual multi-armed bandits. In NIPS 2007: Proceeding of the 20th Annual Conference on Neural Information Processing System, 2007.
10. D. D. Lewis, Y. Yang, T. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
11. Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. In WWW'10: Proceedings of the 19th international conference on World wide web, pages 661-670, New York, NY, USA, 2010. ACM.
12. Wei Li, Xuerui Wang, Ruofei Zhang, Ying Cui, Jianchang Mao, and Rong Jin. Exploitation and exploration in a performance based contextual advertising system. In KDD 2010: Knoledge Discovery and Data Mining, pages 27-36, 2010.
13. Shie Mannor and John N. Tsitsiklis. The sample complexity of exploration in the multi-armed bandit problem. Journal of Machine Learning Research, 5:623-648, 2004.
14. Herbert Robbins. some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58:527-535, 1952.
15. Herbert Robins. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc., 58(5):527-535, 2010.
16. F. Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65:386-408, 1958.
17. Joannès Vermorel and Mehryar Mohri. Multi-armed bandit algorithms and empirical evaluation. In In European Conference on Machine Learning, pages 437-448. Springer, 2005.
18. Shijun Wang, Rong Jin, and Hamed Valizadegan. A potential-based framework for online multi-class learning with partial feedback. In ISTATS 2010: Artificial Intelligence and Statistics, 2010.
19. C. Watkins. Learning from delayed Rewards. PhD thesis, Cambridge, 1989.