Probabilistic non-linear principal component analysis with Gaussian process latent variable models

Journal of Machine Learning Research

Volumn 6, Issue , 2005, Pages

  Lawrence, Neil ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Gaussian processes; Latent variable models; Principal component analysis; Spectral methods; Unsupervised learning; Visualisation

Indexed keywords

ALGORITHMS; FUNCTIONS; MAPPING; MATHEMATICAL MODELS; PROBABILITY; SET THEORY; SPECTRUM ANALYSIS; VISUALIZATION;

GAUSSIAN PROCESSES; LATENT VARIABLE MODELS; SPECTRAL METHODS; UNSUPERVISED LEARNING;

PRINCIPAL COMPONENT ANALYSIS;

EID: 27844605876     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. David J. Bartholomew. Latent Variable Models and Factor Analysis. Charles Griffin & Co. Ltd, London, 1987.
2. Alexander Basilevsky. Statistical Factor Analysis and Related Methods. Wiley, New York, 1994.
3. Christopher M. Bishop. Bayesian PCA. In Michael J. Kearns, Sara A. Solla, and David A. Cohn, editors, Advances in Neural Information Processing Systems, volume 11, pages 482-388, Cambridge, MA, 1999. MIT Press.
4. Christopher M. Bishop and Gwilym D. James. Analysis of multiphase flows using dual-energy gamma densitometry and neural networks. Nuclear Instruments and Methods in Physics Research, A327:580-593, 1993.
5. Christopher M. Bishop, Marcus Svensén, and Christopher K. I. Williams. A fast EM algorithm for latent variable density models. In D. S. Touretzky, Michael C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8, pages 465-471. MIT Press, 1996.
6. Christopher M. Bishop, Marcus Svensén, and Christopher K. I. Williams. GTM: a principled alternative to the Self-Organizing Map. In Advances in Neural Information Processing Systems, volume 9, pages 354-360. MIT Press, 1997.
7. Christopher M. Bishop, Marcus Svensén, and Christopher K. I. Williams. GTM: the Generative Topographic Mapping. Neural Computation, 10(1):215-234, 1998.
8. Lehel Csató. Gaussian Processes - Iterative Sparse Approximations. PhD thesis, Aston University, 2002.
9. Keith Grochow, Steven L. Martin, Aaron Hertzmann, and Zoran Popovic. Style-based inverse kinematics. In ACM Transactions on Graphics (SIGGRAPH 2004), 2004.
10. Geoffrey E. Hinton, Peter Dayan, Brendan J. Frey, and Radford M. Neal. The wake-sleep algorithm for unsupervised neural networks. Science, 268:1158-1161, 1995.
11. Geoffrey E. Hinton and Sam T. Roweis. Stochastic neighbor embedding. In Sue Becker, Sebastian Thrun, and Klaus Obermayer, editors, Advances in Neural Information Processing Systems, volume 15, pages 857-864, Cambridge, MA, 2003. MIT Press.
12. Antti Honkela and Harri Valpola. Unsupervised variational Bayesian learning of nonlinear models. In Lawrence Saul, Yair Weiss, and Léon Bouttou, editors, Advances in Neural Information Processing Systems, volume 17, pages 593-600, Cambridge, MA, 2005. MIT Press.
13. Teuvo Kohonen. The self-organizing map. Proceedings of the IEEE, 78(9):1464-1480, 1990.
14. Joseph B. Kruskal. Multidimensional scaling by optimizing goodness-of-fit to a nonmetric hypothesis. Psychometrika, 29(1):1-28, 1964.
15. Solomon Kullback and Richard A. Leibler. On information and sufficiency. Annals of Mathematical Statistics, 22:79-86, 1951.
16. Neil D. Lawrence. Gaussian process models for visualisation of high dimensional data. In Sebastian Thrun, Lawrence Saul, and Bernhard Schölkopf, editors, Advances in Neural Information Processing Systems, volume 16, pages 329-336, Cambridge, MA, 2004. MIT Press.
17. Neil D. Lawrence, Matthias Seeger, and Ralf Herbrich. Fast sparse Gaussian process methods: The informative vector machine. In Sue Becker, Sebastian Thrun, and Klaus Obermayer, editors, Advances in Neural Information Processing Systems, volume 15, pages 625-632, Cambridge, MA, 2003. MIT Press.
18. David Lowe and Michael E. Tipping. Feed-forward neural networks and topographic mappings for exploratory data analysis. Neural Computing and Applications, 4(83), 1996.
19. David B. MacKay and J. L. Zinnes. A probabilistic model for the multidimensional scaling of proximity and preference data. Marketing Sciences, 5:325-334, 1986.
20. David J. C. MacKay. Bayesian neural networks and density networks. Nuclear Instruments and Methods in Physics Research, A, 354(1):73-80, 1995.
21. Jan R. Magnus and Heinz Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley and Sons, Chichester, West Sussex, 2nd edition, 1999.
22. Kantilal V. Mardia, John T. Kent, and John M. Bibby. Multivariate analysis. Academic Press, London, 1979.
23. Peter S. Maybeck. Stochastic Models, Estimation and Control, Volume 1, volume 141 of Mathematics in Science and Engineering. Academic Press, New York, NY, 1979. ISBN 0-12-4807011.
24. Thomas P. Minka. A family of algorithms for approximate Bayesian inference. PhD thesis, Massachusetts Institute of Technology, 2001.
25. Martin F. Møller. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6(4):525-533, 1993.
26. Man-Suk Oh and Adrian E. Raftery. Bayesian multidimensional scaling and choice of dimension. Journal of the American Statistical Association, 96:1031-1044, 2001.
27. Anthony O'Hagan. Some Bayesian numerical analysis. In José M. Bernardo, James O. Berger, A. Phillip Dawid, and Adrian F. M. Smith, editors, Bayesian Statistics 4, pages 345-363, Valencia, 1992. Oxford University Press.
28. Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323-2326, 2000.
29. John W. Sammon. A nonlinear mapping for data structure analysis. IEEE Transactions on Computers, C-18(5):401-409, 1969.
30. Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998.
31. Bernhard Schölkopf and Alexander J. Smola. Learning with Kernels. MIT Press, 2001.
32. Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888-905, 2000.
33. Joshua B. Tenenbaum, Virginia de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000.
34. Michael E. Tipping. Topographic Mappings and Feed-Forward Neural Networks. PhD thesis, Aston University, Aston Street, Birmingham B4 7ET, U.K., 1996.
35. Michael E. Tipping. Probabilistic visualisation of high-dimensional binary data. In Michael J. Kearns, Sara A. Solla, and David A. Cohn, editors, Advances in Neural Information Processing Systems, volume 11, pages 592-598, Cambridge, MA, 1999. MIT Press.
36. Michael E. Tipping. Sparse kernel principal component analysis. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 633-639, Cambridge, MA, 2001. MIT Press.
37. Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, B, 6(3):611-622, 1999.
38. Warren S. Torgerson. Multidimensional scaling: I. theory and method. Psychometrika, 17:401-419, 1952.
39. Christopher K. I. Williams. Computing with infinite networks. In Michael C. Mozer, Michael I. Jordan, and Thomas Petsche, editors, Advances in Neural Information Processing Systems, volume 9, Cambridge, MA, 1997. MIT Press.
40. Christopher K. I. Williams. Prediction with Gaussian processes: From linear regression to linear prediction and beyond. In Michael I. Jordan, editor, Learning in Graphical Models, volume 89 of Series D: Behavioural and Social Sciences, Dordrecht, The Netherlands, 1998. Kluwer.
41. Christopher K. I. Williams. On a connection between kernel PCA and metric multidimensional scaling. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 675-681, Cambridge, MA, 2001. MIT Press.