Diluted treatment effect estimation for trigger analysis in online controlled experiments

WSDM 2015 - Proceedings of the 8th ACM International Conference on Web Search and Data Mining

Volumn , Issue , 2015, Pages 349-358

  Deng, Alex ^a   Hu, Victor ^a  

^a MICROSOFT   (United States)

Author keywords

A b testing; Controlled experiment; Dilution; Feature coverage; Metric; Variance reduction

Indexed keywords

APPROXIMATION ALGORITHMS; DILUTION; WELL TESTING;

A/B TESTING; CONTROLLED EXPERIMENT; FEATURE COVERAGE; METRIC; VARIANCE REDUCTIONS;

DATA MINING;

EID: 84928741338     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2684822.2685307     Document Type: Conference Paper

References (7)

1. Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In Proceedings of the sixth ACM international conference on Web search and data mining, pages 123-132. ACM, 2013.
2. Ron Kohavi, Roger Longbotham, Dan Sommerfield, and Randal M. Henne. Controlled experiments on the web: survey and practical guide. Data Mining Knowledge Discovery, 18:140-181, 2009.
3. Ron Kohavi, Alex Deng, Roger Longbotham, and Ya Xu. Seven rules of thumb for web site experimenters. 2014.
4. Jim Manzi. Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society. Basic Books, 2012.
5. Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
6. Anastasios Tsiatis. Semiparametric Theory and Missing Data. Springer-Verlag, 2006.
7. Li Yang and Anastasios A. Tsiatis. Efficiency study of estimators for a treatment effect in a pretest-posttest trial. The American Statistician, 55, 2001.