Robust gaussian process regression with a student-t likelihood

Journal of Machine Learning Research

Volumn 12, Issue , 2011, Pages 3227-3257

  Jylänki, Pasi ^a   Vanhatalo, Jarno ^b   Vehtari, Aki ^a  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITY OF HELSINKI   (Finland)

Author keywords

Approximate inference; Expectation propagation; Gaussian process; Robust regression; Student t distribution

Indexed keywords

APPROXIMATE INFERENCE; EXPECTATION PROPAGATION; GAUSSIAN PROCESSES; ROBUST REGRESSIONS; STUDENT-T DISTRIBUTION;

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; GAUSSIAN NOISE (ELECTRONIC); IMAGE MATCHING; MARKOV PROCESSES; MAXIMUM LIKELIHOOD ESTIMATION; REGRESSION ANALYSIS; STUDENTS;

GAUSSIAN DISTRIBUTION;

EID: 84855386927     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer Science +Business Media LLC, 2006.
2. Botond Cseke and Tom Heskes. Properties of Bethe free energies and message passing in Gaussian models. Journal of Artificial Intelligence Research, 41:1-24, 2011.
3. A. Philip Dawid. Posterior expectations for large observations. Biometrika, 60(3):664-667, 1973.
4. Bruno De Finetti. The Bayesian approach to the rejection of outliers. In Proceedings of the fourth Berkeley Symposium on Mathematical Statistics and Probability, pages 199-210. University of California Press, 1961.
5. Jerome H. Friedman. Multivariate adaptive regression splines. Annals of Statistics, 19(1):1-67, 1991.
6. Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian Data Analysis. Chapman & Hall/CRC, second edition, 2004.
7. John Geweke. Bayesian treatment of the independent Student-t linear model. Journal of Applied Econometrics, 8:519-540, 1993.
8. Charles J. Geyer. Practical markov chain monte carlo. Statistical Science, 7(12):473-483, 1992.
9. Mark N. Gibbs and David J. C. MacKay. Variational Gaussian process classifiers. IEEE Transactions on Neural Networks, 11(6):1458-1464, 2000.
10. Paul W. Goldberg, Christopher K. I. Williams, and Christopher M. Bishop. Regression with inputdependent noise: A Gaussian process treatment. In M. I. Jordan, M. J. Kearns, and S. A Solla, editors, Advances in Neural Information Processing Systems 10. MIT Press, Cambridge, MA, 1998.
11. José Miguel Hernández-Lobato, Tjeerd Dijkstra, and Tom Heskes. Regulator discovery from gene expression time series of malaria parasites: A hierarchical approach. In J. C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 649-656. MIT Press, Cambridge, MA, 2008.
12. Tom Heskes and Onno Zoeter. Expectation propagation for approximate inference in dynamic Bayesian networks. In A. Darwiche and N. Friedman, editors, Uncertainty in Artificial Intelligence: Proceedings of the Eighteenth Conference (UAI-2002), pages 216-233. Morgan Kaufmann, San Francisco, CA, 2002.
13. Tom Heskes, Manfred Opper, Wim Wiegerinck, Ole Winther, and Onno Zoeter. Approximate inference techniques with expectation constraints. Journal of Statistical Mechanics: Theory and Experiment, 2005:P11015, 2005.
14. Malte Kuss. Gaussian Process Models for Robust Regression, Classification, and Reinforcement Learning. PhD thesis, Technische Universität Darmstadt, 2006.
15. Chuanhai Liu and Donald B. Rubin. ML estimation of the t distribution using EMand its extensions, ECM and ECME. Statistica Sinica, 5:19-39, 1995.
16. Thomas P. Minka. A Family of Algorithms for Approximate Bayesian Inference. PhD thesis, Massachusetts Institute of Technology, 2001a.
17. Thomas P. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence (UAI-2001), pages 362-369.Morgan Kaufmann, San Francisco, CA, 2001b.
18. Thomas P. Minka. Power EP. Technical report, Microsoft Research, Cambridge, 2004.
19. Thomas P. Minka. Divergence measures and message passing. Technical report, Microsoft Research, Cambridge, 2005.
20. Thomas P. Minka and John Lafferty. Expectation-propagation for the generative aspect model. In Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI-2002), pages 352-359. Morgan Kaufmann, San Francisco, CA, 2002.
21. Andrew Naish-Guzman and Sean Holden. Robust regression with twinned Gaussian processes. In J. C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 1065-1072. MIT Press, Cambridge, MA, 2008.
22. Radford M. Neal. Monte Carlo Implementation of Gaussian Process Models for Bayesian Regression and Classification. Technical Report 9702, Dept. of statistics and Dept. of Computer Science, University of Toronto, 1997.
23. Radford M. Neal. Annealed importance sampling. Statistics and Computing, 11(2):125-139, 2001. (Pubitemid 33638185)
24. Hannes Nickisch and Carl E. Rasmussen. Approximations for binary Gaussian process classification. Journal of Machine Learning Research, 9:2035-2078, 2008.
25. Anthony O'Hagan. On outlier rejection phenomena in Bayes inference. Journal of the Royal Statistical Society (Series B), 41(3):358-367, 1979.
26. Manfred Opper and Cédric Archambeau. The variational Gaussian approximation revisited. Neural Computation, 21(3):786-792, 2009.
27. Manfred Opper and OleWinther. Expectation consistent approximate inference. Journal ofMachine Learning Research, 6:2177-2204, 2005. (Pubitemid 41832629)
28. Carl E. Rasmussen and Hannes Nickisch. Gaussian processes for machine learning (GPML) toolbox. Journal of Machine Learning Research, 11:3011-3015, 2010.
29. Carl Edward Rasmussen and Christopher K. I.Williams. Gaussian Processes forMachine Learning. MIT Press, Cambridge, MA, 2006.
30. Håvard Rue, Sara Martino, and Nicolas Chopin. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal statistical Society (Series B), 71(2):1-35, 2009.
31. Matthias Seeger. Expectation propagation for exponential families. Technical report, Max Planck Institute for Biological Cybernetics, Tübingen, Germany, 2005.
32. Matthias Seeger. Bayesian inference and optimal design for the sparse linear model. Journal of Machine Learning Research, 9:759-813, 2008.
33. Matthias Seeger and Hannes Nickisch. Fast convergent algorithms for expectation propagation approximate Bayesian inference. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15, pages 652-660. JMLR W&CP, 2011.
34. Lawrence F. Shampine. Vectorized adaptive quadrature in MATLAB. Journal of Computational and Applied Mathematics, 211:131-140, 2008. (Pubitemid 350016632)
35. Oliver Stegle, Sebastian V. Fallert, David J. C. MacKay, and Søren Brage. Gaussian process robust regression for noisy heart rate data. Biomedical Engineering, IEEE Transactions on, 55(9):2143-2151, 2008.
36. Luke Tierney and Joseph B. Kadane. Accurate approximations for posterior moments and marginal densities. Journal of the American Statistical Association, 81(393):82-86, 1986.
37. Michael E. Tipping and Neil D. Lawrence. A variational approach to robust Bayesian interpolation. In In Proceedings of the IEEE InternationalWorkshop on Neural Networks for Signal Processing, pages 229-238. IEEE, 2003.
38. Michael E. Tipping and Neil D. Lawrence. Variational inference for Student-t models: Robust bayesian interpolation and generalised component analysis. Neurocomputing, 69:123-141, 2005. (Pubitemid 41644427)
39. Marcel van Gerven, Botond Cseke, Robert Oostenveld, and Tom Heskes. Bayesian source localization with the multivariate Laplace prior. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1901-1909, 2009.
40. Jarno Vanhatalo and Aki Vehtari. Sparse log Gaussian processes via MCMC for spatial epidemiology. JMLR Workshop and Conference Proceedings, 1:73-89, 2007.
41. Jarno Vanhatalo, Pasi Jylänki, and Aki Vehtari. Gaussian process regression with Student-t likelihood. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1910-1918, 2009.
42. Aki Vehtari and Jouko Lampinen. Bayesian model assessment and comparison using crossvalidation predictive densities. Neural Computation, 14(10):2439-2468, 2002.
43. Mike West. Outlier models and prior distributions in Bayesian linear regression. Journal of the Royal Statistical Society (Series B), 46(3):431-439, 1984.
44. Christopher K. I. Williams and David Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1342-1351, 1998. (Pubitemid 128741378)
45. I-Cheng Yeh. Modeling of strength of high performance concrete using artificial neural networks. Cement and Concrete Research, 28(12):1797-1808, 1998. (Pubitemid 128638428)