Inferring causal impact using bayesian structural time-series models

Annals of Applied Statistics

Volumn 9, Issue 1, 2015, Pages 247-274

  Brodersen, Kay H ^a   Gallusser, Fabian ^a   Koehler, Jim ^a   Remy, Nicolas ^a   Scott, Steven L ^a  

^a GOOGLE INC   (United States)

Author keywords

Advertising; Causal inference; Counterfactual; Difference in differences; Econometrics; Market research; Observational; Synthetic control

Indexed keywords

EID: 84929676275     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/14-AOAS788     Document Type: Article

References (63)

1. ABADIE, A. (2005). Semiparametric difference-in-differences estimators. Rev. Econom. Stud. 72 1–19. MR2116973.
2. ABADIE, A., DIAMOND, A. and HAINMUELLER, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. J. Amer. Statist. Assoc. 105 493–505. MR2759929.
3. ABADIE, A. and GARDEAZABAL, J. (2003). The economic costs of conflict: A case study of the basque country. Amer. Econ. Rev. 93 113–132.
4. ANGRIST, J. D. and KRUEGER, A. B. (1999). Empirical strategies in labor economics. Handbook of Labor Economics 3 1277–1366.
5. ANGRIST, J. D. and PISCHKE, J.-S. (2008). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton Univ. Press, Princeton, NJ.
6. ANTONAKIS, J., BENDAHAN, S., JACQUART, P. and LALIVE, R. (2010). On making causal claims: A review and recommendations. Leadersh. Q. 21 1086–1120.
7. ASHENFELTER, O. and CARD, D. (1985). Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67 648–660.
8. ATAMAN, M. B., MELA, C. F. and VAN HEERDE, H. J. (2008). Building brands. Mark. Sci. 27 1036–1054.
9. ATHEY, S. and IMBENS, G. W. (2002). Identification and inference in nonlinear difference-indifferences models. Working Paper 280, National Bureau of Economic Research, Cambridge, MA.
10. BANERJEE, S., KAUFFMAN, R. J. and WANG, B. (2007). Modeling Internet firm survival using Bayesian dynamic models with time-varying coefficients. Electron. Commer. Res. Appl. 6 332–342.
11. BELLONI, A., CHERNOZHUKOV, V., Fernandez-Val, I. and HANSEN, C. (2013). Program evaluation with high-dimensional data. CeMMAP Working Paper CWP77/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies, London.
12. BERNDT, E. R. (1991). The Practice of Econometrics: Classic and Contemporary. Addison-Wesley, Reading, MA.
13. BERTRAND, M., DUFLO, E. and MULLAINATHAN, S. (2002). How much should we trust differences-in-differences estimates? Working Paper 8841, National Bureau of Economic Research, Cambridge, MA.
14. BRADY, H. E. (2002). Models of causal inference: Going beyond the Neyman-Rubin-Holland theory. In Annual Meetings of the Political Methodology Group, Boston, MA.
15. BRODERSEN, K. H., DAUNIZEAU, J., MATHYS, C., CHUMBLEY, J. R., BUHMANN, J. M. and STEPHAN, K. E. (2013). Variational Bayesian mixed-effects inference for classification studies. Neuroimage 76 345–361.
16. CAMILLO, F. and D’Attoma, I. (2010). A new data mining approach to estimate causal effects of policy interventions. Expert Syst. Appl. 37 171–181.
17. CAMPBELL, D. T., STANLEY, J. C. and GAGE, N. L. (1963). Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin, Boston.
18. CARD, D. and KRUEGER, A. B. (1993). Minimum wages and employment: A case study of the fast food industry in New Jersey and Pennsylvania. Technical report, National Bureau of Economic Research, Cambridge, MA.
19. CARTER, C. K. and KOHN, R. (1994). On Gibbs sampling for state space models. Biometrika 81 541–553. MR1311096.
20. CHAN, D., GE, R., GERSHONY, O., HESTERBERG, T. and LAMBERT, D. (2010). Evaluating online ad campaigns in a pipeline: Causal models at scale. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 7–16. ACM, New York.
21. CHIPMAN, H., GEORGE, E. I. and MCCULLOCH, R. E. (2001). The practical implementation of Bayesian model selection. In Model Selection. Institute of Mathematical Statistics Lecture Notes—Monograph Series 38 65–134. IMS, Beachwood, OH. MR2000752.
22. CLAVEAU, F. (2012). The Russo–Williamson Theses in the social sciences: Causal inference drawing on two types of evidence. Stud. Hist. Philos. Biol. Biomed. Sci. 43 806–813.
23. COX, D. and WERMUTH, N. (2001). Causal inference and statistical fallacies. In International Encyclopedia of the Social & Behavioral Sciences (Neil J. Smelser and P. B. Baltes, eds.) 1554–1561. Pergamon, Oxford.
24. DANAHER, P. J. and RUST, R. T. (1996). Determining the optimal return on investment for an advertising campaign. European J. Oper. Res. 95 511–521.
25. DE JONG, P. and SHEPHARD, N. (1995). The simulation smoother for time series models. Biometrika 82 339–350. MR1354233.
26. DONALD, S. G. and LANG, K. (2007). Inference with difference-in-differences and other panel data. Rev. Econ. Stat. 89 221–233.
27. DURBIN, J. and KOOPMAN, S. J. (2002). A simple and efficient simulation smoother for state space time series analysis. Biometrika 89 603–615. MR1929166.
28. Frühwirth-Schnatter, S. (1994). Data augmentation and dynamic linear models. J. Time Series Anal. 15 183–202. MR1263889.
29. GEORGE, E. I. and MCCULLOCH, R. E. (1993). Variable selection via Gibbs sampling. J. Amer. Statist. Assoc. 88 881–889.
30. GEORGE, E. I. and MCCULLOCH, R. E. (1997). Approaches for Bayesian variable selection. Statist. Sinica 7 339–374.
31. GHOSH, J. and CLYDE, M. A. (2011). Rao-Blackwellization for Bayesian variable selection and model averaging in linear and binary regression: A novel data augmentation approach. J. Amer. Statist. Assoc. 106 1041–1052. MR2894762.
32. HANSEN, C. B. (2007a). Asymptotic properties of a robust variance matrix estimator for panel data when T is large. J. Econometrics 141 597–620. MR2413481.
33. HANSEN, C. B. (2007b). Generalized least squares inference in panel and multilevel models with serial correlation and fixed effects. J. Econometrics 140 670–694. MR2408922.
34. HECKMAN, J. J. and VYTLACIL, E. J. (2007). Econometric evaluation of social programs, Part I: Causal models, structural models and econometric policy evaluation. In Handbook of Econometrics 6, Part B (J. J. Heckman and E. E. Leamer, eds.) 4779–4874. Elsevier, Amsterdam.
35. HITCHCOCK, C. (2004). Do all and only causes raise the probabilities of effects? In Causation and Counterfactuals. MIT Press, Cambridge.
36. HOOVER, K. D. (2012). Economic theory and causal inference. In Philosophy of Economics 13 (U. Mäki, ed.) 89–113. Elsevier, Amsterdam.
37. KLEINBERG, S. and HRIPCSAK, G. (2011). A review of causal inference for biomedical informatics. J. Biomed. Inform. 44 1102–1112.
38. LEEFLANG, P. S., BIJMOLT, T. H., VAN DOORN, J., HANSSENS, D. M., VAN HEERDE, H. J., VERHOEF, P. C. and WIERINGA, J. E. (2009). Creating lift versus building the base: Current trends in marketing dynamics. Int. J. Res. Mark. 26 13–20.
39. LESTER, R. A. (1946). Shortcomings of marginal analysis for wage-employment problems. Amer. Econ. Rev. 36 63–82.
40. LEWIS, R. A., RAO, J. M. and REILEY, D. H. (2011). Here, there, and everywhere: Correlated online behaviors can lead to overestimates of the effects of advertising. In Proceedings of the 20th International Conference on World Wide Web. 11 157–166. ACM, New York.
41. LEWIS, R. A. and REILEY, D. H. (2011). Does retail advertising work? Technical report.
42. LIANG, F., PAULO, R., MOLINA, G., CLYDE, M. A. and BERGER, J. O. (2008). Mixtures of g priors for Bayesian variable selection. J. Amer. Statist. Assoc. 103 410–423. MR2420243.
43. MATHYS, C., DAUNIZEAU, J., FRISTON, K. J. and STEPHAN, K. E. (2011). A Bayesian foundation for individual learning under uncertainty. Front. Human Neurosci. 5 39.
44. MEYER, B. D. (1995). Natural and quasi-experiments in economics. J. Bus. Econom. Statist. 13 151.
45. MORGAN, S. L. and WINSHIP, C. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge Univ. Press, Cambridge.
46. NAKAJIMA, J. and WEST, M. (2013). Bayesian analysis of latent threshold dynamic models. J. Bus. Econom. Statist. 31 151–164. MR3055329.
47. POLSON, N. G. and SCOTT, S. L. (2011). Data augmentation for support vector machines. Bayesian Anal. 6 1–23. MR2781803.
48. ROBINSON, G., MCNULTY, J. E. and KRASNO, J. S. (2009). Observing the counterfactual? The search for political experiments in nature. Polit. Anal. 17 341–357.
49. RUBIN, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66 688.
50. RUBIN, D. B. (2008). Statistical inference for causal effects, with emphasis on applications in epidemiology and medical statistics. In Epidemiology and Medical Statistics (J. P. Miller, C. R. Rao and D. C. Rao, eds.). Handbook of Statist. 27 28–63. Elsevier, Amsterdam. MR2500431.
51. RUBIN, D. B. and WATERMAN, R. P. (2006). Estimating the causal effects of marketing interventions using propensity score methodology. Statist. Sci. 21 206–222. MR2324079.
52. SCOTT, J. G. and BERGER, J. O. (2010). Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. Ann. Statist. 38 2587–2619. MR2722450.
53. SCOTT, S. L. and VARIAN, H. R. (2014). Predicting the present with Bayesian structural time series. International Journal of Mathematical Modeling and Optimization 5 4–23.
54. SEGGIE, S. H., CAVUSGIL, E. and PHELAN, S. E. (2007). Measurement of return on marketing investment: A conceptual framework and the future of marketing metrics. Ind. Mark. Manage. 36 834–841.
55. SHADISH, W. R., COOK, T. D. and CAMPBELL, D. T. (2002). Experimental and Quasi- Experimental Designs for Generalized Causal Inference. Wadsworth Cengage Learning, Seattle, WA.
56. SOLON, G. (1984). Estimating Autocorrelations in Fixed-Effects Models. National Bureau of Economic Research, Cambridge, MA.
57. STEWART, D. W. (2009). Marketing accountability: Linking marketing actions to financial results. J. Bus. Res. 62 636–643.
58. TAKADA, H. and BASS, F. M. (1998). Multiple time series analysis of competitive marketing behavior. J. Bus. Res. 43 97–107.
59. VAVER, J. and KOEHLER, J. (2011). Measuring ad effectiveness using geo experiments. Technical report, Google Inc.
60. VAVER, J. and KOEHLER, J. (2012). Periodic measurement of advertising effectiveness using multiple-test-period geo experiments. Technical report, Google Inc.
61. WEST, M. and HARRISON, J. (1997). Bayesian Forecasting and Dynamic Models, 2nd ed. Springer, New York. MR1482232.
62. WINSHIP, C. and MORGAN, S. L. (1999). The estimation of causal effects from observational data. Annu. Rev. Sociol. 659–706.
63. ZELLNER, A. (1986). On assessing prior distributions and Bayesian regression analysis with gprior distributions. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (P. K. Goel and A. Zellner, eds.). Stud. Bayesian Econometrics Statist. 6 233–243. North-Holland, Amsterdam. MR0881437.