Better mixing via deep representations

30th International Conference on Machine Learning, ICML 2013

Volumn , Issue PART 1, 2013, Pages 552-560

  Bengio, Yoshua ^a   Mesnil, Grégoire ^a,b   Dauphin, Yann ^a   Rifai, Salah ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

^b UNIVERSITÉ DE ROUEN   (France)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; MARKOV PROCESSES;

EXPERIMENTAL EVIDENCE; MEASUREMENTS OF; UNDERLYING FACTORS;

MIXING;

EID: 84882266451     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. Alain, G., Bengio, Y., and Rifai, S. (2012). Regularized auto-encoders estimate local statistics. Technical Report Arxiv report 1211.4246, Université de Montréal.
2. Baxter, J. (1997). A Bayesian/information theoretic model of learning via multiple task sampling. Machine Learning, 28, 7-40.
3. Bengio, Y. (2009). Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1), 1-127. Also published as a book. Now Publishers, 2009.
4. Bengio, Y. and Delalleau, O. (2011). On the expressive power of deep architectures. In ALT'2011.
5. Bengio, Y. and LeCun, Y. (2007). Scaling learning algorithms towards AI. In L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, editors, Large Scale Kernel Machines. MIT Press.
6. Bengio, Y., Delalleau, O., and Le Roux, N. (2006). The curse of highly variable functions for local kernel machines. In NIPS'05, pages 107-114. MIT Press, Cambridge, MA.
7. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy).
8. Breuleux, O., Bengio, Y., and Vincent, P. (2011). Quickly generating representative samples from an rbm-derived process. Neural Computation, 23(8), 2053-2073.
9. Caruana, R. (1995). Learning many related tasks at the same time with backpropagation. In NIPS'94, pages 657-664, Cambridge, MA. MIT Press.
10. Cayton, L. (2005). Algorithms for manifold learning. Technical Report CS2008-0923, UCSD.
11. Cho, K., Raiko, T., and Ilin, A. (2010). Parallel tempering is efficient for learning restricted Boltzmann machines. In Proceedings of the International Joint Conference on Neural Networks (IJCNN 2010), Barcelona, Spain.
12. Collobert, R. and Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. In W. W. Cohen, A. McCallum, and S. T. Roweis, editors, ICML 2008, pages 160-167. ACM.
13. Desjardins, G., Courville, A., Bengio, Y., Vincent, P., and Delalleau, O. (2010). Tempered Markov chain monte carlo for training of restricted Boltzmann machine. In JMLR W&CP: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010), volume 9, pages 145-152.
14. Flor, H. (2003). Remapping somatosensory cortex after injury. Advances in Neurology, 83, 195-204.
15. Glorot, X., Bordes, A., and Bengio, Y. (2011). Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the Twenty-eight International Conference on Machine Learning (ICML'11), volume 27, pages 97-110.
16. Goodfellow, I., Le, Q., Saxe, A., and Ng, A. (2009). Measuring invariances in deep networks. In NIPS'2009, pages 646-654.
17. Håstad, J. (1986). Almost optimal lower bounds for small depth circuits. In Proceedings of the 18th annual ACM Symposium on Theory of Computing, pages 6-20, Berkeley, California. ACM Press.
18. Håstad, J. and Goldmann, M. (1991). On the power of small-depth threshold circuits. Computational Complexity, 1, 113-129.
19. Hinton, G. E. and Zemel, R. S. (1994). Autoencoders, minimum description length, and helmholtz free energy. In NIPS'93, pages 3-10. Morgan Kaufmann Publishers, Inc.
20. Hinton, G. E., Osindero, S., and Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554. (Pubitemid 44024729)
21. Lang, K. J. and Hinton, G. E. (1988). The development of the time-delay neural network architecture for speech recognition. Technical Report CMU-CS-88-152, Carnegie-Mellon University.
22. LeCun, Y. (1987). Modèles connexionistes de l'apprentissage. Ph.D. thesis, Université de Paris VI.
23. LeCun, Y. (1989). Generalization and network design strategies. Technical Report CRG-TR-89-4, University of Toronto.
24. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
25. Lee, H., Pham, P., Largman, Y., and Ng, A. (2009). Unsupervised feature learning for audio classification using convolutional deep belief networks. In NIPS'2009.
26. Narayanan, H. and Mitter, S. (2010). Sample complexity of testing the manifold hypothesis. In NIPS'2010.
27. Neal, R. M. (1994). Sampling from multimodal distributions using tempered transitions. Technical Report 9421, Dept. of Statistics, University of Toronto.
28. Rifai, S., Vincent, P., Muller, X., Glorot, X., and Bengio, Y. (2011a). Contracting auto-encoders: Explicit invariance during feature extraction. In Proceedings of the Twenty-eight International Conference on Machine Learning (ICML'11).
29. Rifai, S., Dauphin, Y., Vincent, P., Bengio, Y., and Muller, X. (2011b). The manifold tangent classifier. In NIPS'2011. Student paper award.
30. Rifai, S., Bengio, Y., Dauphin, Y., and Vincent, P. (2012). A generative process for sampling contractive autoencoders. In ICML'2012.
31. Salakhutdinov, R. (2010a). Learning deep Boltzmann machines using adaptive MCMC. In L. Bottou and M. Littman, editors, Proceedings of the Twenty-seventh International Conference on Machine Learning (ICML-10), volume 1, pages 943-950. ACM.
32. Salakhutdinov, R. (2010b). Learning in Markov random fields using tempered transitions. In NIPS'09.
33. Salakhutdinov, R. and Hinton, G. E. (2009). Deep Boltzmann machines. In AISTATS'2009, volume 5, pages 448-455.
34. Susskind, J., Anderson, A., and Hinton, G. E. (2010). The Toronto face dataset. Technical Report UTML TR 2010-001, U. Toronto.