Least-squares independence regression for non-linear causal inference under non-Gaussian noise

Machine Learning

Volumn 96, Issue 3, 2014, Pages 249-267

  Yamada, Makoto ^a   Sugiyama, Masashi ^a   Sese, Jun ^a  

^a NONE   (Japan)

Author keywords

Causal inference; Least squares independence regression; Non Gaussian; Non linear; Squared loss mutual information

Indexed keywords

ADDITIVE NOISE; INFERENCE ENGINES; REGRESSION ANALYSIS;

CAUSAL INFERENCES; LEAST SQUARE; MUTUAL INFORMATIONS; NON-GAUSSIAN; NON-LINEAR;

GAUSSIAN NOISE (ELECTRONIC);

EID: 84905447874     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-013-5423-y     Document Type: Article

References (29)

1. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 337-404.
2. Bishop, C. M. (2006). Pattern recognition and machine learning. New York: Springer.
3. Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed.). Hoboken: Wiley.
4. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.
5. Faith, J. J., Hayete, B., Thaden, J. T., Mogno, I., Wierzbowski, J., Cottarel, G., Kasif, S., Collins, J. J., & Gardner, T. S. (2007). Large-scale mapping and validation of Escherichia coli transcriptional regulation from a compendium of expression profiles. PLoS Biology, 5(1), e8.
6. Feuerverger, A. (1993). A consistent test for bivariate dependence. International Statistical Review, 61(3), 419-433.
7. Fukumizu, K., Bach, F. R., & Jordan, M. (2009). Kernel dimension reduction in regression. The Annals of Statistics, 37(4), 1871-1905.
8. Geiger, D., & Heckerman, D. (1994). Learning Gaussian networks. In 10th annual conference on uncertainty in artificial intelligence (UAI1994) (pp. 235-243).
9. Gretton, A., Bousquet, O., Smola, A., & Schölkopf, B. (2005). Measuring statistical dependence with Hilbert-Schmidt norms. In 16th international conference on algorithmic learning theory (ALT 2005) (pp. 63-78).
10. Hoyer, P. O., Janzing, D., Mooij, J. M., Peters, J., & Schölkopf, B. (2009). Nonlinear causal discovery with additive noise models. In D. Koller, D. Schuurmans, Y. Bengio, & L. Botton (Eds.), Advances in neural information processing systems (Vol. 21, pp. 689-696). Cambridge: MIT Press.
11. Janzing, D., & Steudel, B. (2010). Justifying additive noise model-based causal discovery via algorithmic information theory. Open Systems & Information Dynamics, 17(02), 189-212.
12. Kanamori, T., Suzuki, T., & Sugiyama, M. (2012). Statistical analysis of kernel-based least-squares density-ratio estimation. Machine Learning, 86(3), 335-367.
13. Kankainen, A. (1995). Consistent testing of total independence based on the empirical characteristic function. Ph.D. thesis, University of Jyväskylä, Jyväskylä, Finland.
14. Kraskov, A., Stögbauer, H., & Grassberger, P. (2004). Estimating mutual information. Physical Review E, 69, 066138.
15. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79-86.
16. Liu, D. C., & Nocedal, J. (1989). On the limited memory method for large scale optimization. Mathematical Programming Series B, 45, 503-528.
17. Mooij, J., Janzing, D., Peters, J., & Schölkopf, B. (2009). Regression by dependence minimization and its application to causal inference in additive noise models. In 26th annual international conference on machine learning (ICML2009), Montreal, Canada (pp. 745-752).
18. Patriksson, M. (1999). Nonlinear programming and variational inequality problems. Dordrecht: Kluwer Academic.
19. Pearl, J. (2000). Causality: models, reasoning and inference. New York: Cambridge University Press.
20. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, 50, 157-175.
21. Rockafellar, R. T. (1970). Convex analysis. Princeton: Princeton University Press.
22. Schölkopf, B., & Smola, A. J. (2002). Learning with kernels. Cambridge: MIT Press.
23. Shimizu, S., Hoyer, P. O., Hyvärinen, A., & Kerminen, A. J. (2006). A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7, 2003-2030.
24. Steinwart, I. (2001). On the influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research, 2, 67-93.
25. Suzuki, T., & Sugiyama, M. (2013). Sufficient dimension reduction via squared-loss mutual information estimation. Neural Computation, 3(25), 725-758.
26. Suzuki, T., Sugiyama, M., Kanamori, T., & Sese, J. (2009). Mutual information estimation reveals global associations between stimuli and biological processes. BMC Bioinformatics, 10(S52).
27. Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.
28. Yamada, M., & Sugiyama, M. (2010). Dependence minimizing regression with model selection for non-linear causal inference under non-Gaussian noise. In Proceedings of the twenty-fourth AAAI conference on artificial intelligence (AAAI2010) (pp. 643-648).
29. Zhang, K., & Hyvärinen, A. (2009). On the identifiability of the post-nonlinear causal model. In Proceedings of the 25th conference on uncertainty in artificial intelligence (UAI '09) (pp. 647-655). Arlington: AUAI Press.