Nonparametric guidance of autoencoder representations using label information

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 2567-2588

  Snoek, Jasper ^a   Adams, Ryan P ^b   Larochelle, Hugo ^c  

^a UNIVERSITY OF TORONTO   (Canada)

^b HARVARD UNIVERSITY   (United States)

^c UNIVERSITÉ DE SHERBROOKE   (Canada)

Author keywords

Autoencoder; Gaussian process; Gaussian process latent variable model; Representation learning; Unsupervised learning

Indexed keywords

AUTOENCODERS; COVARIATES; DATA SETS; DENSITY MODELING; DISCRIMINATIVE FUNCTIONS; DISCRIMINATIVE MODELS; EXPLORATORY DATA ANALYSIS; GAUSSIAN PROCESSES; LABEL INFORMATION; LATENT VARIABLE MODELS; NON-PARAMETRIC; NON-PARAMETRIC MODEL; NONPARAMETRIC APPROACHES; PRIORI INFORMATION; REAL-WORLD APPLICATION; REPRESENTATION LEARNING; STATISTICAL VARIATIONS;

DATA VISUALIZATION; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); IMAGE RECOGNITION; PRINCIPAL COMPONENT ANALYSIS;

UNSUPERVISED LEARNING;

EID: 84869179179     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (40)

1. Ryan P. Adams, Zoubin Ghahramani, and Michael I. Jordan. Tree-structured stick breaking for hierarchical data. In Advances in Neural Information Processing Systems, 2010.
2. Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems, 2007.
3. Christopher M. Bishop and G. D. James. Analysis of multiphase flows using dual-energy gamma densitometry and neural networks. Nuclear Instruments and Methods in Physics Research, 1993.
4. Eric Brochu, VladM. Cora, and Nando de Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. pre-print, 2010. arXiv:1012.2599.
5. Adam Coates, Honglak Lee, and Andrew Y. Ng. An analysis of single-layer networks in unsupervised feature learning. In Conference on Artificial Intelligence and Statistics, 2011.
6. Garrison W. Cottrell, Paul Munro, and David Zipser. Learning internal representations from grayscale images: An example of extensional programming. In Conference of the Cognitive Science Society, 1987.
7. Li Deng, Mike Seltzer, Dong Yu, Alex Acero, Abdel-Rahman Mohamed, and Geoffrey E. Hinton. Binary coding of speech spectrograms using a deep autoencoder. In Interspeech, 2010.
8. Jacob Goldberger, Sam Roweis, Geoff Hinton, and Ruslan Salakhutdinov. Neighbourhood components analysis. In Advances in Neural Information Processing Systems, 2004.
9. Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In IEEE Conference on Computer Vision and Pattern Recognition, 2006.
10. Rajibul Huq, Patricia Kan, Robby Goetschalckx, Debbie H'ebert, Jesse Hoey, and Alex Mihailidis. A decision-theoretic approach in the design of an adaptive upper-limb stroke rehabilitation robot. In International Conference of Rehabilitation Robotics (ICORR), 2011.
11. Patricia Kan, Rajibul Huq, Jesse Hoey, Robby Goestschalckx, and Alex Mihailidis. The development of an adaptive upper-limb stroke rehabilitation robotic system. Neuroengineering and Rehabilitation, 2011.
12. Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. An empirical evaluation of deep architectures on problems with many factors of variation. In International Conference on Machine Learning, 2007.
13. Neil D. Lawrence. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. Journal of Machine Learning Research, 6:1783-1816, 2005.
14. Neil D. Lawrence and Joaquin Qui nonero-Candela. Local distance preservation in the GP-LVM through back constraints. In International Conference on Machine Learning, 2006.
15. Neil D. Lawrence and Raquel Urtasun. Non-linear matrix factorization with Gaussian processes. In International Conference on Machine Learning, 2009.
16. Yann LeCun, Fu Jie Huang, and Ĺeon Bottou. Learning methods for generic object recognition with invariance to pose and lighting. IEEE Conference on Computer Vision and Pattern Recognition, 2004.
17. David Lowe and Michael E. Tipping. Neuroscale: Novel topographic feature extraction using RBF networks. In Advances in Neural Information Processing Systems, 1997.
18. Elaine Lu, Rosalie Wang, Rajibul Huq, Don Gardner, Paul Karam, Karl Zabjek, Debbie H́ebert, Jennifer Boger, and Alex Mihailidis. Development of a robotic device for upper limb stroke rehabilitation: A user-centered design approach. Journal of Behavioral Robotics, 2(4):176-184, 2011.
19. David J. C. MacKay. Introduction to Gaussian processes. Neural Networks and Machine Learning, 1998.
20. David J.C. MacKay. Bayesian neural networks and density networks. In Nuclear Instruments and Methods in Physics Research, A, pages 73-80, 1994.
21. Jonas Mockus, Vytautas Tiesis, and Antanas Zilinskas. The application of Bayesian methods for seeking the extremum. Towards Global Optimization, 2:117-129, 1978.
22. Vinod Nair and Geoffrey E. Hinton. 3D object recognition with deep belief nets. In Advances in Neural Information Processing Systems, 2009.
23. Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning, 2010.
24. Radford Neal. Bayesian learning for neural networks. Lecture Notes in Statistics, 118, 1996.
25. Marc'Aurelio Ranzato and Martin Szummer. Semi-supervised learning of compact document representations with deep networks. In International Conference on Machine Learning, 2008.
26. Carl Edward Rasmussen and Christopher K. I.Williams. Gaussian Processes forMachine Learning. MIT Press, Cambridge, MA, 2006.
27. Ruslan Salakhutdinov and Geoffrey Hinton. Learning a nonlinear embedding by preserving class neighbourhood structure. In Conference on Artificial Intelligence and Statistics, 2007.
28. Ruslan Salakhutdinov and Geoffrey Hinton. Using deep belief nets to learn covariance kernels for Gaussian processes. In Advances in Neural Information Processing Systems, 2008.
29. Ruslan Salakhutdinov and Hugo Larochelle. Efficient learning of deep Boltzmann machines. In Conference on Artificial Intelligence and Statistics, 2010.
30. Aaron P. Shon, Keith Grochow, Aaron Hertzmann, and Rajesh P. N. Rao. Learning shared latent structure for image synthesis and robotic imitation. In Advances in Neural Information Processing Systems, 2005.
31. Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, 2012.
32. Babak Taati, Rosalie Wang, Rajibul Huq, Jasper Snoek, and Alex Mihailidis. Vision-based posture assessment to detect and categorize compensation during robotic rehabilitation therapy. In International Conference on Biomedical Robotics and Biomechatronics, 2012.
33. Antonio Torralba, Rob Fergus, andWilliam T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis andMachine Intelligence, 30(11):1958-1970, 2008.
34. Raquel Urtasun and Trevor Darrell. Discriminative Gaussian process latent variable model for classification. In International Conference on Machine Learning, 2007.
35. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In International Conference on Machine Learning, 2008.
36. Francesco Vivarelli and Christopher K. I. Williams. Discovering hidden features with Gaussian process regression. In Advances in Neural Information Processing Systems, 1999.
37. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. Multifactor Gaussian process models for style-content separation. In International Conference on Machine Learning, 2007.
38. Jack M. Wang, David J. Fleet, and Aaron Hertzmann. Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.
39. Christopher K. I. Williams. Computation with infinite neural networks. Neural Computation, 10 (5):1203-1216, 1998. (Pubitemid 128463669)
40. Richard S. Zemel, Christopher K. I. Williams, and Michael C. Mozer. Lending direction to neural networks. Neural Networks, 8:503-512, 1995.