Kernel Bayes' rule: Bayesian inference with positive definite kernels

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 3753-3783

  Fukumizu, Kenji ^a   Song, Le ^b   Gretton, Arthur ^c  

^a INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

^b Georgia Institute of Technology   (United States)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Bayes' rule; Kernel method; Reproducing kernel Hilbert space

Indexed keywords

BAYES' RULE; BAYESIAN COMPUTATION; CONDITIONAL PROBABILITIES; KERNEL METHODS; PARAMETRIC MODELING; POSITIVE DEFINITE KERNELS; REPRODUCING KERNEL HILBERT SPACES; STATE-SPACE MODELING;

BAYESIAN NETWORKS; COMPUTATION THEORY; INFERENCE ENGINES; LEARNING ALGORITHMS;

PROBABILITY;

EID: 84893460360     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (57)

1. Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3):337-404, 1950.
2. Francis R. Bach andMichael I. Jordan. Kernel independent component analysis. Journal ofMachine Learning Research, 3:1-48, 2002.
3. Charles R. Baker. Joint measures and cross-covariance operators. Transactions of the American Mathematical Society, 186:273-289, 1973.
4. Alain Berlinet and Christine Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Kluwer Academic Publisher, 2004.
5. David Blei and Michael Jordan. Variational inference for dirichlet process mixtures. Journal of Bayesian Analysis, 1(1):121-144, 2006.
6. Byron Boots, Arthur Gretton, and Geoffrey J. Gordon. Hilbert space embeddings of predictive state representations. In Proceedings of Conference on Uncertainty in Artificial Intelligence (UAI2013), pages 92-101, 2013.
7. Aedian W. Bowman. An alternative method of cross-validation for the smoothing of density estimates. Biometrika, 71(2):353-360, 1984.
8. Andrea Caponnetto and Ernesto De Vito. Optimal rates for regularized least-squares algorithm. Foundations of Computational Mathematics, 7(3):331-368, 2007.
9. Arnaud Doucet, Nando De Freitas, and Neil J. Gordon. Sequential Monte Carlo Methods in Practice. Springer, 2001.
10. Mona Eberts and Ingo Steinwart. Optimal learning rates for least squares svms using gaussian kernels. In J. Shawe-Taylor, R.S. Zemel, P. Bartlett, F.C.N. Pereira, and K.Q.Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 1539-1547. Curran Associates, Inc., 2011.
11. HeinzW. Engl, Martin Hanke, and Andreas Neubauer. Regularization of Inverse Problems. Kluwer Academic Publishers, 2000.
12. Shai Fine and Katya Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2001.
13. Kenji Fukumizu, Francis R. Bach, and Michael I. Jordan. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. Journal of Machine Learning Research, 5:73-99, 2004.
14. Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, and Bernhard Schölkopf. Kernel measures of conditional dependence. In Advances in Neural Information Processing Systems 20, pages 489-496. MIT Press, 2008.
15. Kenji Fukumizu, Francis R. Bach, andMichael I. Jordan. Kernel dimension reduction in regression. Annals of Statistics, 37(4):1871-1905, 2009a.
16. Kenji Fukumizu, Bhrath Sriperumbudur, Arthur Gretton, and Bernhard Schölkopf. Characteristic kernels on groups and semigroups. In Advances in Neural Information Processing Systems 21, pages 473-480, Red Hook, NY, 2009b. Curran Associates Inc.
17. Arthur Gretton, Karsten M. Borgwardt, Malte Rasch, Bernhard Schölkopf, and A. Smola. A kernel method for the two-sample-problem. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 513-520, Cambridge,MA, 2007.MIT Press.
18. Arthur Gretton, Kenji Fukumizu, Choon Hui Teo, Le Song, Bernhard Schölkopf, and Alex Smola. A kernel statistical test of independence. In Advances in Neural Information Processing Systems 20, pages 585-592. MIT Press, 2008.
19. Arthur Gretton, Kenji Fukumizu, Zaid Harchaoui, and Bharath Sriperumbudur. A fast, consistent kernel two-sample test. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 673-681. 2009a.
20. Arthur Gretton, Kenji Fukumizu, and Bharath K. Sriperumbudur. Discussion of: Brownian distance covariance. Annals of Applied Statistics, 3(4):1285-1294, 2009b.
21. Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alex Smola. A kernel two-sample test. Journal of Machine Learning Research, 13:723-773, 2012.
22. Steffan Gr ünewälder, Guy Lever, Luca Baldassarre, Sam Patterson, Arthur Gretton, and Massimiliano Pontil. Conditional mean embeddings as regressors. In Proceedings of the 29th International Conference on Machine Learning (ICML2012), pages 1823-1830, 2012.
23. Steffen Gr ünewälder, Arthur Gretton, and John Shawe-Taylor. Smooth operators. In Proceedings of the 30th International Conference on Machine Learning (ICML2013), pages 1184-1192, 2013.
24. Zaid Harchaoui, Francis Bach, and Eric Moulines. Testing for homogeneity with kernel Fisher discriminant analysis. In Advances in Neural Information Processing Systems 21, pages 609-616, Cambridge, MA, 2008. MIT Press.
25. Simon J. Julier and Jeffrey K. Uhlmann. A new extension of the kalman filter to nonlinear systems. In Proceedings of AeroSense: The 11th International Symposium on Aerospace/Defence Sensing, Simulation and Controls, 1997.
26. Annaliisa Kankainen and Nikolai G. Ushakov. A consistent modification of a test for independence based on the empirical characteristic function. Journal ofMathematical Sciencies, 89:1582-1589, 1998.
27. Steven MacEachern. Estimating normal means with a conjugate style Dirichlet process prior. Communications in Statistics-Simulation and Computation, 23(3):727-741, 1994.
28. Steven N.MacEachern,Merlise Clyde, and Jun S. Liu. Sequential importance sampling for nonparametric Bayes models: The next generation. The Canadian Journal of Statistics, 27(2):251-267, 1999.
29. Paul Marjoram, John Molitor, Vincent Plagnol, and Simon Tavare. Markov chain monte carlo without likelihoods. Proceedings of the National Academy of Sciences, 100(26):15324-15328, 2003. (Pubitemid 38020988)
30. Florence Merlevéde, Magda Peligrad, and Sergey Utev. Sharp conditions for the clt of linear processes in a hilbert space. Journal of Theoretical Probability, 10:681-693, 1997. (Pubitemid 127429595)
31. Charles A. Micchelli and Massimiliano Pontil. On learning vector-valued functions. Neural Computation, 17(1):177-204, 2005. (Pubitemid 40305887)
32. SebastianMika, Bernhard Schölkopf, Alex Smola, Klaus- RobertMüller,Matthias Scholz, and Gunnar Rätsch. Kernel PCA and de-noising in feature spaces. In Advances in Neural Information Pecessing Systems 11, pages 536-542. MIT Press, 1999.
33. Valérie Monbet, Pierre Ailliot, and Pierre-François Marteau. l1-convergence of smoothing densities in non-parametric state space models. Statistical Inference for Stochastic Processes, 11:311-325, 2008.
34. Peter Müller and Fernando A. Quintana. Nonparametric Bayesian data analysis. Statistical Science, 19(1):95-110, 2004. (Pubitemid 39056879)
35. Shigeki Nakagome, Kenji Fukumizu, and ShuheiMano. Kernel approximate Bayesian computation for population genetic inferences. Statistical Applications in Genetics and Molecular Biology, 2013. Accepted.
36. Yu Nishiyama, Abdeslam Boularias, Arthur Gretton, and Kenji Fukumizu. Hilbert space embeddings of POMDPs. In Proceedigns of Conference on Uncertainty in Artificial Intelligence (UAI2012), pages 644-653, 2012.
37. Mats Rudemo. Empirical choice of histograms and kernel density estimators. Scandinavian Journal of Statistics, 9(2):pp. 65-78, 1982.
38. Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels. MIT Press, 2002.
39. Albert N. Shiryaev. Probability. Springer, 2nd edition, 1995.
40. Scott A. Sisson, Yanan Fan, and Mark M. Tanaka. Sequential Monte Carlo without likelihoods. Proceedings of the National Academy of Sciences, 104(6):1760-1765, 2007. (Pubitemid 46239852)
41. Steve Smale and Ding-Xuan Zhou. Learning theory estimates via integral operators and their approximation. Constructive Approximation, 26:153-172, 2007.
42. Le Song, Jonathan Huang, Alexander Smola, and Kenji Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th International Conference on Machine Learning (ICML2009), pages 961-968, 2009.
43. Le Song, Arthur Gretton., and Carlos Guestrin. Nonparametric tree graphical models via kernel embeddings. In Proceedings of AISTATS 2010, pages 765-772, 2010a.
44. Le Song, Sajid M. Siddiqi, Geoffrey Gordon, and Alexander Smola. Hilbert space embeddings of hidden Markov models. In Proceedings of the 27th International Conference on Machine Learning (ICML2010), pages 991-998, 2010b.
45. Le Song, Arthur Gretton, Danny Bickson, Yucheng Low, and Carlos Guestrin. Kernel belief propagation. In Proceedings of AISTATS 2011, pages 707-715, 2011.
46. Bharath K. Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert R.G. Lanckriet. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11:1517-1561, 2010.
47. Bharath K. Sriperumbudur, Kenji Fukumizu, and Gert Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12:2389-2410, 2011.
48. John Stachurski. A Hilbert space central limit theorem for geometrically ergodic Markov chains. Working paper, Australian National University, 2012.
49. Ingo Steinwart and Andreas Christmann. Support Vector Machines. Springer, 2008.
50. Charles J. Stone. Optimal global rates of convergence for nonparametric regression. Annals of Statistics, 10(4):1040-1053, 1982.
51. Simon Tavaré, David J. Balding, Robert C. Griffithis, and Peter Donnelly. Inferring coalescence times from dna sequece data. Genetics, 145:505-518, 1997. (Pubitemid 27068401)
52. Sebastian Thrun, John Langford, and Dieter Fox. Monte Carlo hidden Markov models: Learning non-parametric models of partially observable stochastic processes. In Proceedings of International Conference on Machine Learning (ICML 1999), pages 415-424, 1999.
53. Larry Wasserman. All of Nonparametric Statistics. Springer, 2006.
54. MikeWest, PeterMüller, andMichael D. Escobar. Hierarchical priors and mixture models, with applications in regression and density estimation. In P. Freeman et al., editor, Aspects of Uncertainty: A Tribute to D.V. Lindley, pages 363-386. Wiley, 1994.
55. Harold Widom. Asymptotic behavior of the eigenvalues of certain integral equations. Transactions of the American Mathematical Society, 109:278-295, 1963.
56. HaroldWidom. Asymptotic behavior of the eigenvalues of certain integral equations II. Archive for Rational Mechanics and Analysis, 17:215-229, 1964.
57. Kun Zhang, Jan Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. In 27th Conference on Uncertainty in Artificial Intelligence, pages 804-813, 2011.