Counterfactual reasoning and learning systems: The example of computational advertising

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 3207-3260

  Bottou, Léon ^a   Peters, Jonas ^b,c   Quiñonero Candela, Joaquin ^a,d   Charles, Denis X ^a   Chickering, D Max ^a   Portugaly, Elon ^a   Ray, Dipankar ^a   Simard, Patrice ^a   Snelson, Ed ^a  

^a MICROSOFT   (United States)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^c ETH ZURICH   (Switzerland)

^d FACEBOOK   (United States)

Author keywords

Causation; Computational advertising; Counterfactual reasoning

Indexed keywords

CAUSAL INFERENCES; CAUSATION; COMPLEX LEARNING; COMPUTATIONAL ADVERTISINGS; COUNTERFACTUAL REASONING;

MARKETING; SEARCH ENGINES;

LEARNING SYSTEMS;

EID: 84890014676     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (63)

1. V M. Aleksandrov, V. I. Sysoyev, and V. V. Shemeneva. Stochastic optimization. Engineering Cybernetics, 5:11-16, 1968.
2. Susan Athey and Denis Nekipelov. A structural model of sponsored search advertising. Working paper, 2010. URL http://kuznets.harvard.edu/~athey/papers/ Structural-Sponsored- Search.pdf.
3. Jean-Yves Audibert, RemiMunos, and Csaba Szepesvári. Tuning bandit algorithms in stochastic environments. In Proc. 18th International Conference on Algorithmic Learning Theory (ALT 2007), pages 150-165, 2007.
4. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fisher. Finite time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002. (Pubitemid 34126111)
5. Heejung Bang and JamesM. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61:692-972, 2005.
6. Dirk Bergemann and Maher Said. Dynamic auctions: a survey. Discussion Paper 1757R, Cowles Foundation for Research in Economics, Yale University, 2010.
7. Léon Bottou. From machine learning to machine reasoning. http://arxiv.org/abs/1102.1808v3, Feb 2011.
8. Leo Breiman. Statistical modeling: The two cultures. Statistical Science, 16(3):199-231, 2001. (Pubitemid 33636349)
9. Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In Advances in Neural Information Processing Systems 24, pages 2249-2257. NIPS Foundation, 2011.
10. C. R. Charig, D. R. Webb, S. R. Payne, and J. E. A. Wickham. Comparison of treatment of renal calculi by open surgery, percutaneous nephrolithotomy, and extracorporeal shockwave lithotripsy. British Medical Journal (Clin Res Ed), 292(6254):879-882, 1986. (Pubitemid 16133624)
11. Denis X. Charles and D. Max Chickering. Optimization for paid search auctions. Manuscript in preparation, 2012.
12. Denis X. Charles, D. Max Chickering, and Patrice Simard. Micro-market experimentation for paid search. Manuscript in preparation, 2012.
13. Harald Cramér. Mathematical Methods of Statistics. Princeton University Press, 1946.
14. Denver Dash. Caveats for Causal Reasoning with Equilibrium Models. PhD thesis, University of Pittsburgh, 2003.
15. Miroslav Dudík, Dimitru Erhan, John Langford, and Lihong Li. Sample-efficient nonstationarypolicy evaluation for contextual bandits. In Proceedings of Uncertainty in Artificial Intelligence (UAI), pages 247-254, 2012.
16. Benjamin Edelman, Michael Ostrovsky, and Michael Schwarz. Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. American Economic Review, 97(1):242-259, 2007.
17. Alan Genz. Numerical computation of multivariate normal probabilities. Journal Computation of Multivariate Normal Probabilities, 1:141-149, 1992.
18. John C. Gittins. Bandit Processes and Dynamic Allocation Indices. Wiley, 1989.
19. Peter W. Glynn. Likelihood ratio gradient estimation: an overview. In Proceedings of the 1987 Winter Simulation Conference, pages 366-375, 1987.
20. John Goodwin. Microsoft adCenter. Personal communication, 2011.
21. Thore Graepel, Joaquin Quinonero Candela, Thomas Borchert, and Ralf Herbrich. Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft's Bing search engine. In Proceedings of the 27th International Conference on Machine Learning (ICML 2010), Invited Applications Track. Omnipress, 2010.
22. Ron Kohavi, Roger Longbotham, Dan Sommerfield, and RandalM. Henne. Controlled experiments on the web: Survey and practical guide. Data Mining and Knowledge Discovery, 18(1):140-181, July 2008.
23. Volodymyr Kuleshov and Doina Precup. Algorithms for multi-armed bandit problems, October 2010. http://www.cs.mcgill.ca/~vkules/bandits.pdf.
24. Sébastien Lahaie and R. Preston McAfee. Efficient ranking in sponsored search. In Proc. 7th International Workshop on Internet and Network Economics (WINE 2011), pages 254-265. LNCS 7090, Springer, 2011.
25. Lev Landau and Evgeny Lifshitz. Course in Theoretical Physics, Volume 1: Mechanics. Pergamon Press, 1969. 2nd edition.
26. John Langford and Tong Zhang. The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in Neural Information Processing Systems 20, pages 817-824. MIT Press, Cambridge, MA, 2008.
27. Steffen L. Lauritzen and Thomas S. Richardson. Chain graph models and their causal interpretation. Journal of the Royal Statistical Society, Series B, 64:321-361, 2002.
28. David K. Lewis. Counterfactuals. Harvard University Press, 1973. 2nd edition: Wiley-Blackwell, 2001.
29. Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on the World Wide Web (WWW 2010), pages 661-670. ACM, 2010.
30. Lihong Li,Wei Chu, John Langford, and XuanhuiWang. Unbiased offline evaluation of contextualbandit- based news article recommendation algorithms. In Proc. 4th ACM International Conference on Web Search and Data Mining (WSDM 2011), pages 297-306, 2011.
31. Andreas Maurer and Massimiliano Pontil. Empirical bernstein bounds and sample-variance penalization. In Proc. The 22nd Conference on Learning Theory (COLT 2009), 2009.
32. Paul Milgrom. Putting Auction Theory to Work. Cambridge University Press, 2004.
33. Roger B. Myerson. Optimal auction design. Mathematics of Operations Research, 6(1):58-73, 1981.
34. Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, 2000. 2nd edition: 2009.
35. Judea Pearl. Causal inference in statistics: an overview. Statistics Surveys, 3:96-146, 2009.
36. Judea Pearl. The do-calculus revisited. In Proc. Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI-2012), pages 3-11, 2012.
37. Linda E. Reichl. A Modern Course in Statistical Physics, 2nd Edition. Wiley, 1998.
38. Herbert Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527-535, 1952.
39. James M. Robins, Miguel Angel Hernan, and Babette Brumback. Marginal structural models and causal inference in epidemiology. Epidemiology, 11(5):550-560, Sep 2000. (Pubitemid 30660034)
40. Norbert Sauer. On the density of families of sets. Journal of Combinatorial Theory, 13:145-147, 1972.
41. Yevgeny Seldin, cois Laviolette Fran Nicolò Cesa-Bianchi, John Shawe-Taylor, and Peter Auer. PAC-Bayesian inequalities for martingales. IEEE Transactions on Information Theory, 58(12): 7086-7093, 2012.
42. Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log likelihood function. Journal of Statistical Planning and Inference, 90(2):227-244, 2000.
43. Edward H. Simpson. The interpretation of interaction in contingency tables. Journal of the Royal Statistical Society, Ser. B, 13:238-241, 1951.
44. Alexsandrs Slivkins. Contextual bandits with similarity information. JMLR Conference and Workshop Proceedings, 19:679-702, 2011.
45. Peter Spirtes and Richard Scheines. Causal inference of ambiguous manipulations. Philosophy of Science, 71(5):833-845, Dec 2004. (Pubitemid 40758797)
46. Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction and Search. Springer Verlag, New York, 1993. 2nd edition: MIT Press, Cambridge (Mass.), 2011.
47. StephenM. Stigler. A historical view of statistical concepts in psychology and educational research. American Journal of Education, 101(1):60-70, Nov 1992.
48. Masashi Sugiyama,Matthias Krauledat, and Klaus-RobertMüller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8:985-1005, 2007. (Pubitemid 46798411)
49. Rich S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA, 1998.
50. Diane Tang, Ashish Agarwal, Deirdre O'Brien, and Mike Meyer. Overlapping experiment infrastructure: More, better, faster experimentation. In Proceedings 16th Conference on Knowledge Discovery and Data Mining (KDD 2010), pages 17-26, 2010.
51. Vladimir N. Vapnik. Estimation of Dependences based on Empirical Data. Springer Series in Statistics. Springer Verlag, Berlin, New York, 1982.
52. Vladimir N. Vapnik and Alexey Ya. Chervonenkis. Uniform convegence of the frequencies of occurence of events to their probabilities. Proc. Academy of Sciences of the USSR, 181(4), 1968.
53. English translation: Soviet Mathematics - Doklady, 9:915-918, 1968.
54. Vladimir N. Vapnik and Alexey Ya. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264-280, 1971.
55. Hal R. Varian. Position auctions. International Journal of Industrial Organization, 25:1163-1178, 2007. (Pubitemid 47625640)
56. Hal R. Varian. Online ad auctions. American Economic Review, 99(2):430-434, 2009.
57. Joannes Vermorel and Mehryar Mohri. Multi-armed bandit algorithms and empirical evaluation. In Proc. European Conference on Machine Learning, pages 437-448, 2005.
58. Georg H. von Wright. Explanation and Understanding. Cornell University Press, 1971.
59. Abraham Wald. Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics, 16(2):117-186, 1945.
60. Norbert Wiener. Cybernetics, or Control and Communication in the Animal and the Machine. Hermann et Cie (Paris), MIT Press (Cambridge, Mass.), Wiley and Sons (New York), 1948. 2nd Edition (expanded): MIT Press, Wiley and Sons, 1961.
61. Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(229-256), 1992.
62. James Woodward. Making Things Happen. Oxford University Press, 2005.
63. Sewall S. Wright. Correlation and causation. Journal of Agricultural Research, 20:557-585, 1921.