An introduction to deep learning

ESANN 2011 proceedings, 19th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning

Volumn , Issue , 2010, Pages 477-488

  Arnold, Ludovic ^a,b   Rebecchi, Sébastien ^a   Chevallier, Sylvain ^a   Paugam Moisy, Hélène ^a,c  

^a INRIA SACLAY   (France)

^b UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^c UNIVERSITÉ LUMIÈRE LYON 2   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION TASKS; CURSE OF DIMENSIONALITY; DEEP LEARNING; LEARNING APPROACH; LEARNING SCHEMES; MULTIPLE LAYERS; NONLINEAR PROCESSING;

ARTIFICIAL INTELLIGENCE; COGNITIVE SYSTEMS; COMPUTATIONAL METHODS; SOFTWARE ENGINEERING;

NEURAL NETWORKS;

EID: 84887116981     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (46)

1. Y. Bengio and Y. LeCun. Scaling learning algorithms towards AI. In Large-Scale Kernel Machines. 2007.
2. Y. Bengio, O. Delalleau, and N. Le Roux. The curse of highly variable functions for local kernel machines. In NIPS, 2005.
3. Y. Bengio, P. Lamblin, V. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. In NIPS, 2007.
4. G. E. Hinton, S. Osindero, and Yee-Whye Teh. A fast learning algorithm for deep belief nets. Neur. Comput., 18:1527-1554, 2006.
5. P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In Parallel Distributed Processing, pages 194-281. 1986.
6. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski. A learning algorithm for Boltzmann machines. Cognitive Science, 9:147-169, 1985.
7. Z. Ghahramani. Unsupervised learning. In Adv. Lect. Mach. Learn., pages 72-112. 2004.
8. M. Ranzato, Y.-L. Boureau, S. Chopra, and Y. LeCun. A unified energy-based framework for unsupervised learning. In AISTATS, 2007.
9. V. Nair and G. E. Hinton. Rectified linear units improve restricted Boltzmann machines. In ICML, 2010.
10. A. Krizhevsky. Convolutional deep belief networks on CIFAR-10. Technical report, Univ. Toronto, 2010.
11. G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neur. Comput., 14:1771-1800, 2002.
12. Y. Bengio and O. Delalleau. Justifying and generalizing contrastive divergence. Neur. Comput., 21:1601-1621, 2009.
13. A. Fischer and C. Igel. Training RBMs depending on the signs of the CD approximation of the log-likelihood derivatives. In ESANN, 2011.
14. Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2:1-127, 2009.
15. H. Bourlard and Y. Kamp. Auto-association by multilayer perceptrons and singular value decomposition. Biological Cybernetics, 59:291-294, 1988.
16. G. E. Hinton. Connectionist learning procedures. Artificial Intelligence, 40:185-234, 1989.
17. N. Japkowicz, S. J. Hanson, and M. A. Gluck. Nonlinear autoassociation is not equivalent to PCA. Neur. Comput., 12:531-545, 2000.
18. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol. Extracting and composing robust features with denoising autoencoders. In ICML, 2008.
19. H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio. An empirical evaluation of deep architectures on problems with many factors of variation. In ICML, 2007.
20. M. A. Ranzato, C. Poultney, S. Chopra, and Y. LeCun. Efficient learning of sparse representations with an energy-based model. In NIPS, 2006.
21. M. Ranzato, Y-L. Boureau, and Y. LeCun. Sparse feature learning for deep belief networks. In NIPS, 2008.
22. P. Mirowski, M. Ranzato, and Y. LeCun. Dynamic auto-encoders for semantic indexing. In NIPS WS8, 2010.
23. Y. Cho and L. Saul. Kernel methods for deep learning. In NIPS, 2009.
24. B. Schölkopf, A. J. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neur. Comput., 10:1299-1319, 1998.
25. F. Yger, M. Berar, G. Gasso, and A. Rakotomamonjy. A supervised strategy for deep kernel machine. In ESANN, 2011.
26. R. Rosipal, L. J. Trejo, and B. Matthews. Kernel PLS-SVC for linear and nonlinear classification. In ICML, 2003.
27. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Handwritten digit recognition with a back-propagation network. In NIPS, 1990.
28. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86:2278-2324, 1998.
29. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In ICCV, 2009.
30. G. Desjardins and Y. Bengio. Empirical evaluation of convolutional RBMs for vision. Technical report, Univ. Montréal, 2008.
31. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In ICML, 2009.
32. K. Kavukcuoglu, M. A. Ranzato, R. Fergus, and Y. LeCun. Learning invariant features through topographic filter maps. In CVPR, 2009.
33. K. Kavukcuoglu, P. Sermanet, Y.-L. Boureau, K. Gregor, M. Mathieu, and Y. LeCun. Learning convolutional feature hierarchies for visual recognition. In NIPS. 2010.
34. R. Hadsell, P. Sermanet, J. Ben, A. Erkan, M. Scoffier, K. Kavukcuoglu, U. Muller, and Y. LeCun. Learning long-range vision for autonomous off-road driving. J. Field Robot., 26:120-144, 2009.
35. H. Lee, Y. Largman, P. Pham, and A. Y. Ng. Unsupervised feature learning for audio classification using convolutional deep belief networks. In NIPS, 2009.
36. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML, 2008.
37. R. Salakhutdinov and G. E. Hinton. Using deep belief nets to learn covariance kernels for gaussian processes. In NIPS, 2008.
38. M. D. Zeiler, G. W. Taylor, N. F. Troje, and G. E. Hinton. Modeling pigeon behaviour using a conditional restricted Boltzmann machine. In ESANN, 2009.
39. G. E. Hinton and R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313:504-507, 2006.
40. R. Salakhutdinov and G. E. Hinton. Semantic hashing. Int. J. Approximate Reasoning, 50:969-978, 2009.
41. A. Krizhevsky and G. E. Hinton. Using very deep autoencoders for content-based image retrieval. In ESANN, 2011.
42. M. Ranzato, A. Krizhevsky, and G. E. Hinton. Factored 3-way restricted Boltzmann machines for modeling natural images. In AISTATS, 2010.
43. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 2010.
44. D. Erhan, Y. Bengio, A. Courville, P.-A. Manzagol, P. Vincent, and S. Bengio. Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res., 11:625-660, 2010.
45. A. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Ng. On random weights and unsupervised feature learning. In NIPS WS8, 2010.
46. L. Arnold, H. Paugam-Moisy, and M. Sebag. Unsupervised layer-wise model selection in deep neural networks. In ECAI, 2010.