Deep Gaussian processes for regression using approximate expectation propagation

33rd International Conference on Machine Learning, ICML 2016

Volumn 3, Issue , 2016, Pages 2187-2208

  Bui, Thang D ^a   Hernández Lobato, Jose Miguel ^b   Hernández Lobato, Daniel ^c   Li, Yingzhen ^a   Turner, Richard E ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b HARVARD UNIVERSITY   (United States)

^c UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BACKPROPAGATION; BACKPROPAGATION ALGORITHMS; BAYESIAN NETWORKS; GAUSSIAN NOISE (ELECTRONIC); LEARNING ALGORITHMS; LEARNING SYSTEMS; MULTILAYER NEURAL NETWORKS; NEURAL NETWORKS; REGRESSION ANALYSIS; UNCERTAINTY ANALYSIS;

APPROXIMATE BAYESIAN; APPROXIMATE INFERENCE; BAYESIAN NEURAL NETWORKS; COMPREHENSIVE ANALYSIS; EXPECTATION PROPAGATION; NON-LINEAR REGRESSION; NON-PARAMETRIC PROBABILISTIC MODELS; UNCERTAINTY ESTIMATES;

GAUSSIAN DISTRIBUTION;

EID: 84999019279     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. Barber, D. and Schottky, B. Radial basis functions: a Bayesian treatment. In Advances in Neural Information Processing Systems 10, 1998.
2. Bui, Thang D., Yan, Josiah, and Turner, Richard E. A unifying framework for sparse Gaussian process approximation using Power Expectation Propagation. arXiv preprint arXiv: 1605.07066, 2016.
3. Dai, Zhenwen, Damianou, Andreas, González, Javier, and Lawrence, Neil. Variational auto-encoded deep Gaussian processes. In 4th International Conference on Learning Representations, 2016.
4. Damianou, Andreas. Deep Gaussian processes and variational propagation of uncertainty. PhD thesis, University of Sheffield, 2015.
5. Damianou, Andreas C and Lawrence, Neil D. Deep Gaussian processes. In 16th International Conference on Artificial Intelligence and Statistics, pp. 207-215, 2013.
6. Deisenroth, Marc and Mohamed, Shakir. Expectation propagation in Gaussian process dynamical systems. In Advances in Neural Information Processing Systems 25, pp. 2609-2617, 2012.
7. Demsar, Jancz. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1-30, 2006.
8. Duvenaud, David, Rippel, Oren, Adams, Ryan P., and Ghahramani, Zoubin. Avoiding pathologies in very deep networks. In 17th International Conference on Artificial Intelligence and Statistics, 2014.
9. Gal, Yarin and Ghahramani, Zoubin. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In 33rd International Conference on Machine Learning, 2016.
10. Girard, Agathe, Rasmussen, Carl Edward, Quinonero-Candela, Joaquin, and Murray-Smith, Roderick. Gaussian process priors with uncertain inputs - application to multiple-step ahead time series forecasting. In Advances in Neural Information Processing Systems 15, pp. 529-536, 2003.
11. Graves, Alex. Practical variational inference for neural networks. In Advances in Neural Information Processing Systems 25, pp. 2348-2356, 2011.
12. Hachmann, Johannes, Olivares-Amaya, Roberto, Atahan-Evrenk, Sule, Amador-Bedolla, Carlos, Sánchez-Carrera, Roel S, Gold-Parker, Aryeh, Vogt, Leslie, Brockway, Anna M, and Aspuru-Guzik, Alán. The Harvard clean energy project: large-scale computational screening and design of organic photovoltaics on the world community grid. The Journal of Physical Chemistry Letters, 2(17):2241-2251, 2011.
13. Hensman, James and Lawrence, Neil D. Nested variational compression in deep Gaussian processes. arXiv preprint arXiv:1412.1370, visited on 25/05/2016, 2014.
14. Hernández-Lobato, José Miguel and Adams, Ryan P. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In 32nd International Conference on Machine Learning, 2015.
15. Kingma, D. P. and Ba, J. Adam: a method for stochastic optimization. In 3rd International Conference on Learning Representations, 2015.
16. Kingma, Diederik P. and Welling, Max. Stochastic gradient VB and the variational auto-encoder. In The International Conference on Learning Representations, 2014.
17. Kushncr, H. J. and Budhiraja, A. S. A nonlinear filtering algorithm based on an approximation of the conditional distribution. IEEE Transactions on Automatic Control, 45(3):580-585, Mar. 2000.
18. Lawrence, Neil D. and Moore, Andrew J. Hierarchical Gaussian process latent variable models. In 24th International Conference on Machine Learning, pp. 481-488, 2007.
19. Lázaro-Gredilla, Miguel. Bayesian warped Gaussian processes. In Advances in Neural Information Processing Systems 25, pp. 1619-1627, 2012.
20. Li, Yingzhen, Hernández-Lobato, José Miguel, and Turner, Richard E. Stochastic expectation propagation. In Advances in Neural Information Processing Systems 29, 2015.
21. Minka, Thomas P. A family of algorithms for approximate Bayesian inference. PhD thesis, Massachusetts Institute of Technology, 2001.
22. Neal, Radford M. Bayesian learning via stochastic dynamics. In Advances in Neural Information Processing Systems 6, pp. 475-482, 1993.
23. Neal, Radford M. Bayesian learning for neural networks. PhD thesis, University of Toronto, 1995.
24. Pyzer-Knapp, Edward O, Li, Kewei, and Aspuru-Guzik, Alan. Learning from the Harvard clean energy project: The use of neural networks to accelerate materials discovery. Advanced Functional Materials, 25(41):6495-6502, 2015.
25. Quiñonero-Candela, Joaquin and Rasmussen, Carl Edward. A unifying view of sparse approximate Gaussian process regression. The Journal of Machine Learning Research, 6:1939-1959, 2005.
26. Rasmussen, Carl Edward and Williams, Christopher K. I. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2005.
27. Seeger, Matthias. Expectation propagation for exponential families. Technical report, Department of EECS, University of California at Berkeley, 2007.
28. Snelson, Edward and Ghahramani, Zoubin. Sparse Gaussian processes using pseudo-inputs. In Advances in Neural Information Processing Systems 19, pp. 1257-1264, 2006.
29. Snelson, Edward, Rasmussen, Carl Edward, and Ghahramani, Zoubin. Warped Gaussian processes. In Advances in Neural Information Processing Systems 17, pp. 337-344, Cambridge, MA, USA, 2004.
30. Sutton, Richard S. and Barto, Andrew G. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998. ISBN 0262193981.
31. Titsias, Michalis K. Variational learning of inducing variables in sparse Gaussian processes. In 12th International Conference on Artificial Intelligence and Statistics, pp. 567-574, 2009.
32. Titsias, Michalis K and Lawrence, Neil D. Bayesian Gaussian process latent variable model. In 13th International Conference on Artificial Intelligence and Statistics, pp. 844-851, 2010.
33. Turner, R. E. and Sahani, M. Two problems with variational expectation maximisation for time-series models. In Barber, D., Cemgil, T, and Chiappa, S. (eds.), Bayesian Time series models, chapter 5, pp. 109-130. Cambridge University Press, 2011.
34. Vanhatalo, Jarno and Vehtari, Aki. MCMC methods for MLP-network and Gaussian process and stuff-a documentation for Matlab toolbox MCMCstuff. 2006. Laboratory of computational engineering, Helsinki University of Technology.
35. Welling, Max and Teh, Yee W. Bayesian learning via stochastic gradient Langevin dynamics. In 28th International Conference on Machine Learning, pp. 681-688, 2011.
36. Wilson, Andrew and Adams, Ryan. Gaussian process kernels for pattern discovery and extrapolation. In 30th International Conference on Machine Learning, pp. 1067-1075, 2013.
37. Demšar, Jancz. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, 7:1-30, 2006.
38. Hensman, James and Lawrence, Neil D. Nested variational compression in deep Gaussian processes. arXiv preprint arXiv: 1412.1370. visited on 25/05/2016. 2014.