Improving offline evaluation of contextual bandit algorithms via bootstrapping techniques

31st International Conference on Machine Learning, ICML 2014

Volumn 2, Issue , 2014, Pages 1390-1398

  Nicol, Olivier ^a   Mary, Jérémie ^a   Preux, Philippe ^a  

^a UNIV LILLE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; E-LEARNING; LEARNING ALGORITHMS; LEARNING SYSTEMS; SOCIAL NETWORKING (ONLINE);

CONTEXTUAL BANDITS; CRITICAL ISSUES; DATA-DRIVEN METHODS; MODEL-BASED METHOD; NEWS RECOMMENDATION; OFFLINE EVALUATION; ONLINE LEARNING ALGORITHMS; STATE-OF-THE-ART METHODS;

QUALITY CONTROL;

EID: 84919909115     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Agarwal, Deepak, Chen, Bee-Chung, and Elango, Pradheep. Spatio-temporal models for estimating click- through rate. In Proceedings of the 18th international conference on World wide web(WWW), pp. 21-30, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-487-4. doi: 10.1145/1526709.1526713.
2. Auer, Peter, Cesa-Bianchi, Nicolo, and Fischer, Paul. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47:235-256, May 2002. ISSN 0885- 6125.
3. Auer, Peter, Cesa-Bianchi, Nicolo, Freund, Yoav, and Schapire, Robert E. The nonstochastic multiarmed bandit problem. SIAM J. Comput., 32(1):48-77, January 2003. ISSN 0097-5397. doi: 10.1137/S0097539701398375.
4. Bottou, Leon, Peters, Jonas, Quinonero Candela, Joaquin, Charles, Denis X., Chickering, D. Max, Portugualy, Elon, Ray, Dipankar, Simard, Patrice, and Snelson, Ed. Counterfactual reasoning and learning systems. Technical report, arXiv: 1209.2355, September 2012.
5. Dudik, Miroslav, Langford, John, and Li, Lihong. Doubly robust policy evaluation and learning. CoRR, abs/1103, 4601, 2011.
6. Efron, B. Bootstrap methods: Another look at the jack- knife. The Annals of Statistics, 7(1): 1-26, 1979. ISSN 00905364. doi: 10.2307/2958830.
7. Horowitz, Joel L. The bootstrap. Handbook of econometrics, 5:3159-3228, 2001.
8. Kleiner, Ariel, Talwalkar, Ameet, Sarkar, Puraamrita, and Jordan, Michael. The big data bootstrap. In Langford, John and Pineau, Joelle (eds.), Proceedings of the 29th International Conference on Machine Learning (ICML-12), ICML '12, pp. 1759-1766, New York, NY, USA, July 2012. Omnipress. ISBN 978-1-4503-1285-1.
9. Kohavi, Ron, Longbotham, Roger, Sommerfield, Dan, and Henne, Randal M. Controlled experiments on the web: Survey and practical guide. Journal of Data Mining and Knowledge Discovery, 18:140-181, 2009.
10. Koistinen, Petri and Holmstrom, Lassc. Kernel regression and backpropagation training with noise. In Moody, John E., Hanson, Steve J., and Lippmann, Richard P. (eds.), Advances in Neural Information Processing Systems 4, pp. 1033-1039. San Francisco, CA: Morgan Kaufmann, 1992.
11. Langford, John and Zhang, Tong. The epoch-greedy algorithm for multi-armed bandits with side information. In Proc. NIPS, 2007.
12. Langford, John, Strehl, Alexander, and Wortman, Jennifer. Exploration scavenging. In Proceedings of the International Conference on Machine Learning (ICML), pp. 528-535, 2008.
13. Li, Lihong, Chu, Wei, Langford, John, and Schapire, Robert E. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web (WWW), pp. 661-670, New York, NY, USA, 2010. ACM. ISBN 978-1-60558-799-8. doi: 10.1145/1772690.1772758.
14. Li, Lihong, Chu, Wei, Langford, John, and Wang, Xuanhui. Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms. In King, Irwin, Nejdl, Wolfgang, and Li, Hang (eds.), Proc. Web Search and Data Mining (WSDM), pp. 297-306. ACM, 2011. ISBN 978-1-4503-0493-1.
15. th Artificial Intelligence and Statistics (AI & Stats), JMLR: W&CP 9, May 13-15 2010.
16. Nicol, Olivier. Data-driven evaluation of Contextual Bandit algorithms and applications to Dynamic Recommendation. PhD thesis, Universite Lille 1, Cite Scientifique, Villeneuve d'Ascq, France, 2014.
17. Robbins, Herbert. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527-535, 1952.
18. Scott, D.W. Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley Series in Probability and Statistics. Wiley, 1992. ISBN 9780471547709.
19. Silverman, BW and Young, GA. The bootstrap: To smooth or not to smooth? Biometrika, 74(3):469-479, 1987.
20. Strehl, Alexander L., Langford, John, Li, Lihong, and Kakade, Sham. Learning from logged implicit exploration data. In Proc. NIPS, pp. 2217-2225, 2010.
21. Sutton, R.S. and Barto, A.G. Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series. MIT Press, 1998. ISBN 9780262193986.
22. Thompson, W.R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-294, 1933.
23. Yahoo! Research. R6B - Yahoo! frontpage today module user click log dataset, publicly released via the Yahoo! webscope program, 2012.