Greedy layer-wise training of deep networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2007, Pages 153-160

  Bengio, Yoshua ^a   Lamblin, Pascal ^a   Popovici, Dan ^a   Larochelle, Hugo ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEXITY THEORY; COMPUTATIONAL ELEMENTS; DEEP BELIEF NETWORKS; DISTRIBUTED REPRESENTATION; GENERATIVE MODEL; GRADIENT-BASED OPTIMIZATION; HIGH-LEVEL ABSTRACTION; LAYER-WISE; LOCAL MINIMUMS; OPTIMIZATION PROBLEMS; UNSUPERVISED TRAINING;

LEARNING ALGORITHMS; OPTIMIZATION;

NETWORK ARCHITECTURE;

EID: 84864073449     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Allender, E. (1996). Circuit complexity before the dawn of the new millennium. In 16th Annual Conference on Foundations of Software Technology and Theoretical Computer Science, pp. 1-18. Lecture Notes in Computer Science 1180.
2. Bengio, Y., Delalleau, O., & Le Roux, N. (2006). The curse of highly variable functions for local kernel machines. In Weiss, Y., Schölkopf, B., & Platt, J. (Eds.), Advances in Neural Information Processing Systems 18, pp. 107-114. MIT Press, Cambridge, MA.
3. Bengio, Y., & Le Cun, Y. (2007). Scaling learning algorithms towards AI. In Bottou, L., Chapelle, O., DeCoste, D., & Weston, J. (Eds.), Large Scale Kernel Machines. MIT Press.
4. Bengio, Y., Le Roux, N., Vincent, P., Delalleau, O., & Marcotte, P. (2006). Convex neural networks. In Weiss, Y., Schölkopf, B., & Platt, J. (Eds.), Advances in Neural Information Processing Systems 18, pp. 123-130. MIT Press, Cambridge, MA.
5. Chen, H., & Murray, A. (2003). A continuous restricted boltzmann machine with an implementable training algorithm. IEE Proceedings of Vision, Image and Signal Processing, 150(3), 153-158.
6. Fahlman, S., & Lebiere, C. (1990). The cascade-correlation learning architecture. In Touretzky, D. (Ed.), Advances in Neural Information Processing Systems 2, pp. 524-532 Denver, CO. Morgan Kaufmann, San Mateo.
7. Hastad, J. T. (1987). Computational Limitations for Small Depth Circuits. MIT Press, Cambridge, MA.
8. Hinton, G. E., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554.
9. Hinton, G. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771-1800.
10. Hinton, G., Dayan, P., Frey, B., & Neal, R. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1558-1161.
11. Hinton, G., & Salakhutdinov, R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504-507.
12. Lengellé, R., & Denoeux, T. (1996). Training MLPs layer by layer using an objective function for internal representations. Neural Networks, 9, 83-97.
13. Movellan, J., Mineiro, P., &Williams, R. (2002). A monte-carlo EM approach for partially observable diffusion processes: theory and applications to neural networks. Neural Computation, 14, 1501-1544.
14. Tesauro, G. (1992). Practical issues in temporal difference learning. Machine Learning, 8, 257-277.
15. Utgoff, P., & Stracuzzi, D. (2002). Many-layered learning. Neural Computation, 14, 2497-2539.
16. Welling, M., Rosen-Zvi, M., & Hinton, G. E. (2005). Exponential family harmoniums with an application to information retrieval. In Advances in Neural Information Processing Systems, Vol. 17 Cambridge, MA. MIT Press.