Network A/B Testing: From sampling to estimation

WWW 2015 - Proceedings of the 24th International Conference on World Wide Web

Volumn , Issue , 2015, Pages 399-409

  Gui, Huan ^a   Xu, Ya ^b   Bhasin, Anmol ^b   Han, Jiawei ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b LINKEDIN CORPORATION   (United States)

Author keywords

A B testing; Balanced graph partition.; Controlled experiments; Design of experiments; Network A B testing; Network effects; Peer effects

Indexed keywords

DESIGN OF EXPERIMENTS; HUMAN COMPUTER INTERACTION; WORLD WIDE WEB;

A/B TESTING; BALANCED GRAPHS; CONTROLLED EXPERIMENT; NETWORK EFFECTS; PEER EFFECTS;

SOCIAL NETWORKING (ONLINE);

EID: 84968903216     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2736277.2741081     Document Type: Conference Paper

References (29)

1. K. Andreev and H. Racke. Balanced graph partitioning. Theory of Computing Systems, 39(6):929-939, 2006.
2. P. M. Aronow and C. Samii. Estimating average causal effects under general interference. Citeseer, 2012.
3. L. Backstrom and J. Kleinberg. Network bucket testing. In Proceedings of the 20th international conference on World wide web, pages 615-624. ACM, 2011.
4. D. A. Bader, H. Meyerhenke, P. Sanders, and D. Wagner. Graph partitioning and graph clustering, volume 588. American Mathematical Soc., 2013.
5. V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008(10):P10008, 2008.
6. J. Cheng, L. Adamic, P. A. Dow, J. M. Kleinberg, and J. Leskovec. Can cascades be predicted? In Proceedings of the 23rd international conference on World wide web, pages 925-936. International World Wide Web Conferences Steering Committee, 2014.
7. N. Cressie and H. Whitford. How to use the two sample t-Test. Biometrical Journal, 28(2):131-148, 1986.
8. D. Eckles, B. Karrer, and J. Ugander. Design and analysis of experiments in networks: Reducing bias from interference. arXiv preprint arXiv:1404.7530, 2014.
9. H. Gui, Y. Sun, J. Han, and G. Brova. Modeling topic diffusion in multi-relational bibliographic information networks. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pages 649-658. ACM, 2014.
10. M. G. Hudgens and M. E. Halloran. Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 2008.
11. K. L. Judd. The law of large numbers with a continuum of iid random variables. Journal of Economic theory, 35(1):19-25, 1985.
12. L. Katzir, E. Liberty, and O. Somekh. Framework and algorithms for network bucket testing. In Proceedings of the 21st international conference on World Wide Web, pages 1029-1036. ACM, 2012.
13. R. Kohavi, A. Deng, B. Frasca, T. Walker, Y. Xu, and N. Pohlmann. Online controlled experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1168-1176. ACM, 2013.
14. R. Kohavi, A. Deng, R. Longbotham, and Y. Xu. Seven rules of thumb for web site experimenters. Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 2014.
15. C. F. Manski. Identification of endogenous social effects: The reection problem. The review of economic studies, 60(3):531-542, 1993.
16. C. F. Manski. Identification of treatment response with social interactions. The Econometrics Journal, 16(1):S1-S23, 2013.
17. M. McPherson, L. Smith-Lovin, and J. M. Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, pages 415-444, 2001.
18. J. A. Middleton and P. M. Aronow. Unbiased estimation of the average treatment effect in cluster-randomized experiments. Available at SSRN 1803849, 2011.
19. M. E. Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences, 103(23):8577-8582, 2006.
20. S. W. Raudenbush. Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2):173, 1997.
21. M. G. Rodriguez, J. Leskovec, D. Balduzzi, and B. Scholkopf. Uncovering the structure and temporal dynamics of information propagation. Network Science, 2(01):26-65, 2014.
22. P. R. Rosenbaum. Interference between units in randomized experiments. Journal of the American Statistical Association, 102(477), 2007.
23. D. B. Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688, 1974.
24. M. E. Sobel. What do randomized studies of housing mobility demonstrate? causal inference in the face of interference. Journal of the American Statistical Association, 101(476):1398-1407, 2006.
25. L. Tang, R. Rosales, A. Singh, and D. Agarwal. Automatic ad format selection via contextual bandits. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, pages 1587-1594. ACM, 2013.
26. E. J. T. Tchetgen and T. J. VanderWeele. On causal inference in the presence of interference. Statistical Methods in Medical Research, 21(1):55-75, 2012.
27. P. Toulis and E. Kao. Estimation of causal peer inuence effects. In Proceedings of The 30th International Conference on Machine Learning, pages 1489-1497, 2013.
28. J. Ugander, B. Karrer, L. Backstrom, and J. Kleinberg. Graph cluster randomization: network exposure to multiple universes. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 329-337. ACM, 2013.
29. F. Yates. Sir ronald fisher and the design of experiments. Biometrics, 20(2):307-321, 1964.