A unifying probabilistic perspective for spectral dimensionality reduction: Insights and new models

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 1609-1638

  Lawrence, Neil D ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DIMENSIONALITY REDUCTION; GAUSSIAN MARKOV RANDOM FIELD; GAUSSIAN RANDOM FIELDS; HUMAN MOTION CAPTURE DATA; LAPLACIAN EIGENMAPS; LOCALLY LINEAR EMBEDDING; MAXIMUM ENTROPY; MAXIMUM ENTROPY PRINCIPLE; MAXIMUM VARIANCE; NEW MODEL; NONLINEAR GENERALIZATIONS; PARAMETER FITTING; ROBOT NAVIGATION;

GAUSSIAN DISTRIBUTION; MAXIMUM ENTROPY METHODS; MAXIMUM LIKELIHOOD; VISUALIZATION;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84862001101     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (31)

1. Onureena Banerjee, Laurent El Ghaoui, and Alexandre d'Aspremont. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 2007.
2. Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373-1396, 2003. doi: 10.1162/089976603321780317. (Pubitemid 37049796)
3. Yoshua Bengio, Olivier Delalleau, Jean-Francois Palement, Nicolas Le Roux, Marie Ouimet, and Pascal Vincent. Learning eigenfunctions links spectral embedding and kernel PCA. Neural Computation, 16(10):2197-2219, 2004a. (Pubitemid 39137820)
4. Yoshua Bengio, Jean-Francois Paiement, Pascal Vincent, Olivier Delalleau, Nicolas Le Roux, and Marie Ouimet. Out-of-sample extensions for LLE, isomap, MDS, eigenmaps, and spectral clustering. In Sebastian Thrun, Lawrence Saul, and Bernhard Schölkopf, editors, Advances in Neural Information Processing Systems, volume 16, pages 177-184, Cambridge, MA, 2004b. MIT Press.
5. Julian Besag. Statistical analysis of non-lattice data. The Statistician, 24(3):179-195, 1975.
6. George E. P. Box and Norman R. Draper. Empirical Model-Building and Response Surfaces. John Wiley and Sons, 1987. ISBN 0471810339.
7. Brian D. Ferris, Dieter Fox, and Neil D. Lawrence. WiFi-SLAM using Gaussian process latent variable models. In Manuela M. Veloso, editor, Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pages 2480-2485, 2007.
8. Brendan J. Frey, Geoffrey E. Hinton, and Peter Dayan. Does the wake-sleep algorithm learn good density estimators? In David Touretzky, Michael Mozer, and Mark Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8, pages 661-670, Cambridge, MA, 1996. MIT Press.
9. Jerome Friedman, Trevor Hastie, and Robert Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3):432-441, Jul. 2008. doi: 10.1093/biostatistics/kxm045. (Pubitemid 351882084)
10. Jihun Ham, Daniel D. Lee, Sebastian Mika, and Bernhard Schölkopf. A kernel view of dimensionality reduction of manifolds. In Russell Greiner and Dale Schuurmans, editors, Proceedings of the International Conference in Machine Learning, volume 21. Omnipress, 2004.
11. John M. Hammersley and Peter Clifford. Markov fields on finite graphs and lattices. Technical report, 1971. URL http://www.statslab.cam.ac.uk/-grg/books/ hammfest/hamm-cliff.pdf.
12. Stefan Harmeling. Exploring model selection techniques for nonlinear dimensionality reduction. Technical Report EDI-INF-RR-0960, University of Edinburgh, 2007.
13. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer-Verlag, 2nd edition, 2009.
14. Edwin T. Jaynes. Bayesian methods: General background. In J. H. Justice, editor, Maximum Entropy and Bayesian Methods in Applied Statistics, pages 1-25. Cambridge University Press, 1986.
15. Charles Kemp and Joshua B. Tenenbaum. The discovery of structural form. Proc. Natl. Acad. Sci. USA, 105(31), 2008.
16. Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009. ISBN 978-0-262-01319-2.
17. Solomon Kullback and Richard A. Leibler. On information and sufficiency. Annals of Mathematical Statistics, 22:79-86, 1951.
18. Hugo Larochelle and Iain Murray. The neural autoregressive distribution estimator. JMLR: W&CP, 15:29-37, 2011.
19. Neil D. Lawrence. Gaussian process models for visualisation of high dimensional data. In Sebastian Thrun, Lawrence Saul, and Bernhard Schölkopf, editors, Advances in Neural Information Processing Systems, volume 16, pages 329-336, Cambridge, MA, 2004. MIT Press.
20. Neil D. Lawrence. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research, 6:1783-1816, 11 2005.
21. Na Li and Matthew Stephens. Modeling linkage disequilibrium and identifying recombination hotspots using single-nucleotide polymorphism data. Genetics, 165:2213-2233, 2003. URL http://www.genetics.org/cgi/content/abstract/ 165/4/2213. (Pubitemid 38040302)
22. Kantilal V. Mardia, John T. Kent, and John M. Bibby. Multivariate Analysis. Academic Press, London, 1979. ISBN 0-12-471252-5.
23. Radford M. Neal. Connectionist learning of belief networks. Artificial Intelligence, 56:71-113, 1992.
24. Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323-2326, 2000. doi: 10.1126/science.290.5500.2323. (Pubitemid 32041578)
25. Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998. doi: 10.1162/089976698300017467. (Pubitemid 128463674)
26. Joshua B. Tenenbaum, Virginia de Silva, and John C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000. doi: 10.1126/science.290.5500.2319. (Pubitemid 32041577)
27. Michael E. Tipping and Christopher M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statistical Society, B, 6(3):611-622, 1999. doi: doi:10.1111/1467-9868.00196.
28. Larry A. Wasserman. All of Statistics. Springer-Verlag, New York, 2003. ISBN 9780387402727.
29. Kilian Q. Weinberger, Fei Sha, and Lawrence K. Saul. Learning a kernel matrix for nonlinear dimensionality reduction. In Russell Greiner and Dale Schuurmans, editors, Proceedings of the International Conference in Machine Learning, volume 21, pages 839-846. Omnipress, 2004. (Pubitemid 40290888)
30. Christopher K. I. Williams. On a connection between kernel PCA and metric multidimensional scaling. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems, volume 13, pages 675-681, Cambridge, MA, 2001. MIT Press.
31. Xiaojin Zhu, John Lafferty, and Zoubin Ghahramani. Semi-supervised learning: From Gaussian fields to Gaussian processes. Technical Report CMU-CS-03-175, Carnegie Mellon University, 2003.