Variational auto-encoded deep Gaussian processes

4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings

Volumn , Issue , 2016, Pages

  Dai, Zhenwen ^a   Damianou, Andreas ^a   González, Javier ^a   Lawrence, Neil ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC);

BAYESIAN OPTIMIZATION; GAUSSIAN PROCESSES; GENERATIVE MODEL; NON-PARAMETRIC; POSTERIOR DISTRIBUTIONS; RECOGNITION MODELS; VARIATIONAL FRAMEWORK; VARIATIONAL PARAMETERS;

DEEP LEARNING;

EID: 85083954311     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. Bengio, Yoshua, Mesnil, Grégoire, Dauphin, Yann, and Rifai, Salah. Better Mixing via Deep Representations. In International Conference on Machine Learning, 2013.
2. Bengio, Yoshua, Laufer, Eric, Alain, Guillaume, and Yosinski, Jason. Deep Generative Stochastic Networks Trainable by Backprop. In International Conference on Machine Learning, 2014.
3. Blundell, Charles, Cornebise, Julien, Kavukcuoglu, Koray, and Wierstra, Daan. Weight Uncertainty in Neural Networks. In International Conference on Machine Learning, 2015.
4. Bui, Thang D., Hernández-Lobato, José Miguel, Li, Yingzhen, Hernández-Lobato, Daniel, and Turner, Richard E. Training Deep Gaussian Processes using Stochastic Expectation Propagation and Probabilistic Backpropagation. In Workshop on Advances in Approximate Bayesian Inference, NIPS, 2015.
5. Calandra, Roberto, Peters, Jan, Rasmussen, Carl Edward, and Deisenroth, Marc Peter. Manifold Gaussian processes for regression. Technical report, 2014.
6. Cole, D, Martin-Moran, C, Sheard, AG, Bhadeshia, HKDH, and MacKay, DJC. Modelling creep rupture strength of ferritic steel welds. Science and Technology of Welding & Joining, 5(2):81–89, 2000.
7. Dai, Zhenwen, Damianou, Andreas, Hensman, James, and Lawrence, Neil. Gaussian process models with parallelization and GPU acceleration, 2014.
8. Damianou, Andreas. Deep Gaussian processes and variational propagation of uncertainty. PhD Thesis, University of Sheffield, 2015.
9. Damianou, Andreas and Lawrence, Neil. Semi-described and semi-supervised learning with Gaussian processes. In 31st Conference on Uncertainty in Artificial Intelligence (UAI), 2015.
10. Damianou, Andreas and Lawrence, Neil D. Deep Gaussian processes. In Carvalho, Carlos and Ravikumar, Pradeep (eds.), Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics, volume 31, pp. 207–215, AZ, USA, 4 2013. JMLR W&CP 31.
11. Damianou, Andreas, Titsias, Michalis K., and Lawrence, Neil D. Variational Gaussian process dynamical systems. In Bartlett, Peter, Peirrera, Fernando, Williams, Chris, and Lafferty, John (eds.), Advances in Neural Information Processing Systems, volume 24, Cambridge, MA, 2011. MIT Press.
12. Dasgupta, Sanjoy and McAllester, David (eds.). Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013, volume 28 of JMLR Proceedings, 2013. JMLR.org.
13. Durrande, N, Ginsbourger, D, and Roustant, O. Additive kernels for Gaussian process modeling. Technical report, 2011.
14. Duvenaud, David, Lloyd, James Robert, Grosse, Roger, Tenenbaum, Joshua B., and Ghahramani, Zoubin. Structure discovery in nonparametric regression through compositional kernel search. In Dasgupta & McAllester (2013), pp. 1166–1174.
15. Duvenaud, David, Rippel, Oren, Adams, Ryan, and Ghahramani, Zoubin. Avoiding pathologies in very deep networks. In Kaski, Sami and Corander, Jukka (eds.), Proceedings of the Seventeenth International Workshop on Artificial Intelligence and Statistics, volume 33, Iceland, 2014. JMLR W&CP 33.
16. Ek, Carl Henrik, Rihan, Jon, Torr, Philip, Rogez, Gregory, and Lawrence, Neil D. Ambiguity modeling in latent spaces. In Popescu-Belis, Andrei and Stiefelhagen, Rainer (eds.), Machine Learning for Multimodal Interaction (MLMI 2008), LNCS, pp. 62–73. Springer-Verlag, 28–30 June 2008.
17. Gal, Yarin and Ghahramani, Zoubin. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. arXiv:1506.02142, 2015.
18. Gal, Yarin, van der Wilk, Mark, and Rasmussen, Carl E. Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models. In Advances in Neural Information Processing System, 2014.
19. Gönen, Mehmet and Alpaydin, Ethem. Multiple kernel learning algorithms. Journal of Machine Learning Research, 12:2211–2268, Jul 2011.
20. Goodfellow, Ian, Pouget-Abadie, Jean, Mirza, Mehdi, Xu, Bing, Warde-Farley, David, Ozair, Sherjil, Courville, Aaron, and Bengio, Yoshua. Generative Adversarial Networks. In Advances in Neural Information Processing Systems, 2014.
21. Hensman, James and Lawrence, Neil D. Nested variational compression in deep Gaussian processes. Technical report, University of Sheffield, 2014.
22. Hensman, James, Lawrence, Neil D., and Rattray, Magnus. Hierarchical Bayesian modelling of gene expression time series across irregularly sampled replicates and clusters. BMC Bioinformatics, 14(252), 2013. doi: doi:10.1186/1471-2105-14-252.
23. Hinton, Geoffrey E., Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv: 1207.0580, 2012.
24. Jones, Donald R., Schonlau, Matthias, and Welch, William J. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492, 1998.
25. Kingma, Diederik P and Welling, Max. Auto-Encoding Variational Bayes. In ICLR, 2013.
26. Kingma, Diederik P., Salimans, Tim, and Welling, Max. Variational Dropout and the Local Reparameterization Trick. In Advances in Neural Information Processing System, 2015.
27. Lawrence, Neil D. and Moore, Andrew J. Hierarchical Gaussian process latent variable models. In Ghahramani, Zoubin (ed.), Proceedings of the International Conference in Machine Learning, volume 24, pp. 481–488. Omnipress, 2007. ISBN 1-59593-793-3.
28. Lawrence, Neil D. and Quiñonero Candela, Joaquin. Local distance preservation in the GP-LVM through back constraints. In Cohen, William and Moore, Andrew (eds.), Proceedings of the International Conference in Machine Learning, volume 23, pp. 513–520. Omnipress, 2006. ISBN 1-59593-383-2. doi: 10.1145/1143844.1143909.
29. Lázaro-Gredilla, Miguel. Bayesian warped Gaussian processes. In Bartlett, Peter L., Pereira, Fernando C. N., Burges, Christopher J. C., Bottou, Léon, and Weinberger, Kilian Q. (eds.), Advances in Neural Information Processing Systems, volume 25, Cambridge, MA, 2012.
30. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, November 1998.
31. Li, Yujia, Swersky, Kevin, and Zemel, Richard. Generative Moment Matching Networks. In International Conference on Machine Learning, 2015.
32. Mnih, A. and Gregor, K. Neural Variational Inference and Learning in Belief Networks. In International Conference on Machine Learning, 2014.
33. Netzer, Yuval, Wang, Tao, Coates, Adam, Bissacco, Alessandro, Wu, Bo, and Ng, Andrew Y. Reading Digits in Natural Images with Unsupervised Feature Learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
34. Osborne, Michael. Bayesian Gaussian Processes for Sequential Prediction, Optimisation and Quadrature. PhD thesis, University of Oxford, 2010.
35. Rezende, D J, Mohamed, S, and Wierstra, D. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning, 2014.
36. Salakhutdinov, Ruslan and Hinton, Geoffrey. Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes. In Advances in Neural Information Processing Systems, volume 20, 2008.
37. Snelson, Edward, Rasmussen, Carl Edward, and Ghahramani, Zoubin. Warped Gaussian processes. In Thrun, Sebastian, Saul, Lawrence, and Schölkopf, Bernhard (eds.), Advances in Neural Information Processing Systems, volume 16, Cambridge, MA, 2004. MIT Press.
38. Snoek, Jasper, Adams, Ryan P., and Larochelle, Hugo. Nonparametric Guidance of Autoencoder Representations using Label Information. Journal of Machine Learning Research, 13:2567–2588, 2012a.
39. Snoek, Jasper, Larochelle, Hugo, and Adams, Ryan P. Practical Bayesian optimization of machine learning algorithms, pp. 29512959. 2012b.
40. Titsias, Michalis K. and Lawrence, Neil D. Bayesian Gaussian process latent variable model. In Teh, Yee Whye and Titterington, D. Michael (eds.), Proceedings of the Thirteenth International Workshop on Artificial Intelligence and Statistics, volume 9, pp. 844–851, Chia Laguna Resort, Sardinia, Italy, 13-16 May 2010. JMLR W&CP 9.
41. Uria, Benigno, Murray, Iain, and Larochelle, Hugo. A Deep and Tractable Density Estimator. In International Conference on Machine Learning, 2014.
42. Wilson, Andrew Gordon and Adams, Ryan Prescott. Gaussian process kernels for pattern discovery and extrapolation. In Dasgupta & McAllester (2013), pp. 1067–1075.