CAM: Causal additive models, high-dimensional order search and penalized regression

Annals of Statistics

Volumn 42, Issue 6, 2014, Pages 2526-2556

  Bühlmann, Peter ^a   Peters, Jonas ^a   Ernest, Jan ^a  

^a ETH ZURICH   (Switzerland)

Author keywords

Graphical modeling; Intervention calculus; Nonparametric regression; Regularized estimation; Sparsity; Structural equation model

Indexed keywords

EID: 84987997394     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/14-AOS1260     Document Type: Article

References (48)

1. BREIMAN, L. and FRIEDMAN, J. H. (1985). Estimating optimal transformations for multiple regression and correlation. J. Amer. Statist. Assoc. 80 580-619. With discussion and with a reply by the authors. MR0803258
2. BÜHLMANN, P. and HOTHORN, T. (2007). Boosting algorithms: Regularization, prediction and model fitting. Statist. Sci. 22 477-505. MR2420454
3. BÜHLMANN, P., PETERS, J. and ERNEST, J. (2014). Supplement to "CAM: Causal additive models, high-dimensional order search and penalized regression." DOI:10.1214/14-AOS1260SUPP.
4. BÜHLMANN, P. and VAN DE GEER, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Heidelberg. MR2807761
5. 2 loss: Regression and classification. J. Amer. Statist. Assoc. 98 324-339. MR1995709
6. CHICKERING, D. M. (2002). Optimal structure identification with greedy search. J. Mach. Learn. Res. 3 507-554. MR1991085
7. COLOMBO, D., MAATHUIS, M. H., KALISCH, M. and RICHARDSON, T. S. (2012). Learning high-dimensional directed acyclic graphs with latent and selection variables. Ann. Statist. 40 294-321. MR3014308
8. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer, New York. MR2722294
9. HOTHORN, T., BÜHLMANN, P., KNEIB, T., SCHMID, M. and HOFNER, B. (2010). Model-based boosting 2.0. J. Mach. Learn. Res. 11 2109-2113. MR2719848
10. IMOTO, S., GOTO, T. and MIYANO, S. (2002). Estimation of genetic networks and functional structures between genes by using Bayesian network and nonparametric regression. In Proceedings of the Pacific Symposium on Biocomputing (PSB) 175-186. Lihue, HI.
11. JANZING, D., PETERS, J., MOOIJ, J. M. and SCHÖLKOPF, B. (2009). Identifying confounders using additive noise models. In Proceedings of the 25th Annual Conference on Uncertainty in Artificial Intelligence (UAI) 249-257. AUAI Press, Corvallis, OR.
12. LAURITZEN, S. L. (1996). Graphical Models. Oxford Univ. Press, New York. MR1419991
13. LOH, P. and BÜHLMANN, P. (2013). High-dimensional learning of linear causal networks via inverse covariance estimation. J. Mach. Learn. Res. To appear. Available at arXiv:1311.3492.
14. MAMMEN, E. and PARK, B. U. (2006). A simple smooth backfitting method for additive models. Ann. Statist. 34 2252-2271. MR2291499
15. MARRA, G. and WOOD, S. N. (2011). Practical variable selection for generalized additive models. Comput. Statist. Data Anal. 55 2372-2387. MR2786996
16. MEIER, L., VAN DE GEER, S. and BÜHLMANN, P. (2009). High-dimensional additive modeling. Ann. Statist. 37 3779-3821. MR2572443
17. MEINSHAUSEN, N. and BÜHLMANN, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist. 34 1436-1462. MR2278363
18. MEINSHAUSEN, N. and BÜHLMANN, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B Stat. Methodol. 72 417-473. MR2758523
19. MOOIJ, J. and HESKES, T. (2013). Cyclic causal discovery from continuous equilibrium data. In Proceedings of the 29th Annual Conference on Uncertainty in Artificial Intelligence (UAI) 431-439. AUAI Press, Corvallis, OR.
20. MOOIJ, J., JANZING, D., HESKES, T. and SCHÖLKOPF, B. (2011). On causal discovery with cyclic additive noise models. In Advances in Neural Information Processing Systems 24 (NIPS) 639-647. Curran, Red Hook, NY.
21. MOOIJ, J., JANZING, D., PETERS, J. and SCHÖLKOPF, B. (2009). Regression by dependence minimization and its application to causal inference. In Proceedings of the 26th International Conference on Machine Learning (ICML) 745-752. ACM, New York.
22. NOWZOHOUR, C. and BÜHLMANN, P. (2013). Score-based causal learning in additive noise models. Available at arXiv:1311.6359.
23. PEARL, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, Cambridge. MR1744773
24. PETERS, J. and BÜHLMANN, P. (2013). Structural intervention distance (SID) for evaluating causal graphs. Neural Comput. To appear. Available at arXiv:1306.1043.
25. PETERS, J. and BÜHLMANN, P. (2014). Identifiability of Gaussian structural equation models with equal error variances. Biometrika 101 219-228. MR3180667
26. PETERS, J., MOOIJ, J., JANZING, D. and SCHÖLKOPF, B. (2014). Causal discovery with continuous additive noise models. J. Mach. Learn. Res. 15 2009-2053.
27. RAMSEY, J., ZHANG, J. and SPIRTES, P. (2006). Adjacency-faithfulness and conservative causal inference. In Proceedings of the 22nd Annual Conference on Uncertainty in Artificial Intelligence (UAI) 401-408. AUAI Press, Corvallis, OR.
28. RAVIKUMAR, P., LAFFERTY, J., LIU, H. and WASSERMAN, L. (2009). Sparse additive models. J. R. Stat. Soc. Ser. BStat. Methodol. 71 1009-1030. MR2750255
29. RÉNYI, A. (1959). On measures of dependence. Acta Math. Acad. Sci. Hung. 10 441-451 (unbound insert). MR0115203
30. RICHARDSON, T. (1996). A discovery algorithm for directed cyclic graphs. In Uncertainty in Artificial Intelligence (Portland, OR, 1996) 454-461. Morgan Kaufmann, San Francisco, CA. MR1617227
31. SCHMIDT, M., NICULESCU-MIZIL, A. and MURPHY, K. (2007). Learning graphical model structure using L1-regularization paths. In Proceedings of the National Conference on Artificial Intelligence 22 1278. AAAI Press, Menlo Park, CA.
32. SHIMIZU, S., HOYER, P. O., HYVÄRINEN, A. and KERMINEN, A. (2006). A linear non-Gaussian acyclic model for causal discovery. J. Mach. Learn. Res. 7 2003-2030. MR2274431
33. SPIRTES, P. (1995). Directed cyclic graphical representations of feedback models. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI) 491-499. Morgan Kaufmann, San Francisco, CA.
34. SPIRTES, P., GLYMOUR, C. and SCHEINES, R. (2000). Causation, Prediction, and Search, 2nd ed. MIT Press, Cambridge, MA. MR1815675
35. TEYSSIER, M. and KOLLER, D. (2005). Ordering-based search: A simple and effective algorithm for learning Bayesian networks. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI) 584-590. AUAI Press, Corvallis, OR.
36. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. MR1379242
37. VAN DE GEER, S. (2014). On the uniform convergence of empirical norms and inner products, with application to causal inference. Electron. J. Stat. 8 543-574. MR3211024
38. 0-penalized maximum likelihood for sparse directed acyclic graphs. Ann. Statist. 41 536-567. MR3099113
39. VAN DE GEER, S., BÜHLMANN, P. and ZHOU, S. (2011). The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso). Electron. J. Stat. 5 688-749. MR2820636
40. VOORMAN, A., SHOJAIE, A. and WITTEN, D. (2014). Graph estimation with joint additive models. Biometrika 101 85-101. MR3180659
41. 1-constrained quadratic programming (Lasso). IEEE Trans. Inform. Theory 55 2183-2202. MR2729873
42. WILLE, A., ZIMMERMANN, P., VRANOVÁ, E., FÜRHOLZ, A., LAULE, O., BLEULER, S., HENNIG, L., PRELIC, A., VON ROHR, P., THIELE, L., ZITZLER, E., GRUISSEM, W. and BÜHLMANN, P. (2004). Sparse graphical Gaussian modeling of the isoprenoid gene network in Arabidopsis thaliana. Genome Biol. 5 R92.
43. WOOD, S. N. (2006). Generalized Additive Models: An Introduction with R. Chapman & Hall/CRC, Boca Raton, FL. MR2206355
44. YUAN, M. and LIN, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. BStat. Methodol. 68 49-67. MR2212574
45. ZHANG, C.-H. and HUANG, J. (2008). The sparsity and bias of the Lasso selection in high-dimensional linear regression. Ann. Statist. 36 1567-1594. MR2435448
46. ZHANG, K. and HYVÄRINEN, A. (2009). On the identifiability of the post-nonlinear causal model. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI) 647-655. AUAI Press, Corvallis, OR.
47. ZHAO, P. and YU, B. (2006). On model selection consistency of Lasso. J. Mach. Learn. Res. 7 2541-2563. MR2274449
48. ZOU, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418-1429. MR2279469