Scalable Gaussian process regression using deep neural networks

IJCAI International Joint Conference on Artificial Intelligence

Volumn 2015-January, Issue , 2015, Pages 3576-3582

  Huang, Wenbing ^a   Zhao, Deli ^b   Sun, Fuchun ^a   Liu, Huaping ^a   Chang, Edward ^b  

^a TSINGHUA UNIVERSITY   (China)

^b Civil Engineering Department   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; GAUSSIAN NOISE (ELECTRONIC); MAPPING; REGRESSION ANALYSIS;

DEEP NEURAL NETWORKS; FEATURE MAPPING; FINITE DIMENSIONAL; GAUSSIAN PROCESS MODELS; GAUSSIAN PROCESS REGRESSION; GAUSSIAN PROCESSES; KERNEL MATRICES; STATE-OF-THE-ART ALGORITHMS;

GAUSSIAN DISTRIBUTION;

EID: 84949813690     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
2. C. M. Bishop. Pattern recognition and machine learning. Springer, New York, 2006.
3. Kyung Hyun Cho, Tapani Raiko, and Alexander Ilin. Gaussian-Bernoulli deep Boltzmann machine. In International Joint Conference on Neural Networks (IJCNN), 2013.
4. Andreas C Damianou and Neil D Lawrence. Deep gaussian processes. In Proceedings of the Sixteenth International Workshop on Artificial Intelligence and Statistics, 2012.
5. Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research (JMLR), 11:625-660, 2010.
6. G E Hinton and R R Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, July 2006.
7. G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554, 2006.
8. Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. An empirical evaluation of deep architectures on problems with many factors of variation. In International Conference on Machine learning (ICML), 2007.
9. Neil D. Lawrence, Matthias Seeger, and Ralf Herbrich. Fast sparse Gaussian process methods: the informative vector machine. In Suzanna Becker, Sebastian Thrun, and Klaus Obermayer, editors, Advances in Neural Information Processing Systems (NIPS), pages 609-616, 2002.
10. Miguel Lázaro-Gredilla and Ańibal R Figueiras-Vidal. Marginalized neural network mixtures for large-scale regression. IEEE Transactions on Neural Networks, 21(8):1345-1351, 2010.
11. Honglak Lee, Chaitanya Ekanadham, and Andrew Y Ng. Sparse deep belief net model for visual area v2. In Advances in Neural Information Processing Systems (NIPS), pages 873-880, 2008.
12. Andrew Naish-Guzman and Sean B. Holden. The generalized FITC approximation. In John C. Platt, Daphne Koller, Yoram Singer, and Sam T. Roweis, editors, Advances in Neural Information Processing Systems (NIPS), 2007.
13. Radford M Neal. Bayesian learning for neural networks. PhD thesis, University of Toronto, 1995.
14. Hannes Nickisch and Carl Edward Rasmussen. Approximations for binary Gaussian process classification. Journal of Machine Learning Research (JMLR), 9:2035-2078, October 2008.
15. Joaquin Quiñonero-Candela and Carl Edward Rasmussen. A unifying view of sparse approximate Gaussian process regression. Journal of Machine Learning Research (JMLR), 6:1939-1959, 2005.
16. C. E. Rasmussen and C. K. I. Williams. Gaussian processes for machine learning. The MIT Press, 2006.
17. Ruslan Salakhutdinov and Geoffrey E. Hinton. Using deep belief nets to learn covariance kernels for Gaussian processes. In John C. Platt, Daphne Koller, Yoram Singer, and Sam T. Roweis, editors, Advances in Neural Information Processing Systems (NIPS), 2007.
18. Ruslan Salakhutdinov and Geoffrey E. Hinton. Deep Boltzmann machines. In David A. Van Dyk and Max Welling, editors, International Conference on Artificial Intelligence and Statistics (AISTATS), pages 448-455, 2009.
19. Matthias Seeger. Gaussian processes for machine learning. International Journal of Neural Systems., 14(2):69-106, 2004.
20. Alex J. Smola and Peter L. Bartlett. Sparse greedy Gaussian process regression. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems (NIPS), pages 619-625, 2000.
21. Edward Snelson and Zoubin Ghahramani. Sparse Gaussian processes using pseudo-inputs. In Advances in Neural Information Processing Systems (NIPS), 2005.
22. Jasper Snoek, Ryan Prescott Adams, and Hugo Larochelle. On nonparametric guidance for learning autoencoder representations. Journal of Machine Learning Research (JMLR), 2011.
23. Michalis Titsias and Miguel Lázaro-Gredilla. Variational inference for mahalanobis distance metrics in Gaussian process regression. In Advances in Neural Information Processing Systems (NIPS), pages 279-287, 2013.
24. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In International Conference on Machine learning (ICML), 2008.
25. Christopher KI Williams and David Barber. Bayesian classification with Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 20(12):1342-1351, 1998.
26. Shaohua Kevin Zhou, Bogdan Georgescu, Xi-ang Sean Zhou, and Dorin Comaniciu. Image based regression using boosting method. In IEEE International Conference on Computer Vision (ICCV), 2005.