Deep, narrow sigmoid belief networks are universal approximators

Neural Computation

Volumn 20, Issue 11, 2008, Pages 2629-2636

  Sutskever, Ilya ^a   Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; HUMAN; LEARNING; NONLINEAR SYSTEM;

ALGORITHMS; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 55749105263     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2008.12-07-661     Document Type: Article

References (13)

1. Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007). Greedy layer-wise training of deep networks. In M. I. Jordan, Y. Le Cun, & S. A. Solla (Eds.), Advances in neural information processing systems, 19 (pp. 153-160). Cambridge, MA: MIT Press.
2. Bengio, Y., & Le Cun, Y. (2007). Scaling learning algorithms towards AI. In L. Bottou, O. Chapelle, D. DeCoste, & J. Weston (Eds.), Large scale kernel machines (pp. 321-359). Cambridge, MA: MIT Press.
3. Hinton, G. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771-1800.
4. Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527-1554.
5. Hinton, G., & Salakhutdinov, R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504-507.
6. Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359-366.
7. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007). An empirical evaluation of deep architectures on problems with many factors of variation. In Proceedings of the Annual International Conference on Machine Learning (ICML-2007) (pp. 473-480). N.p.: Omni Press.
8. Le Roux, N., & Bengio, Y. (2007). Representational power of restricted Boltzmann machines and deep belief networks (Tech. Rep. 1294, DIRO).Montreal: University of Montreal.
9. Neal, R. (1990). Learning stochastic feedforward networks (Tech. Rep. CRG-TR-90-7). Toronto: Department of Computer Science, University of Toronto.
10. Rojas, R. (2003). Networks of width one are universal classifiers. Proceedings of the International Joint Conference on Neural Networks, 4, 3124-3127.
11. Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by back-propagating errors. Nature, 323(99), 533-536.
12. Salakhutdinov, R., & Hinton, G. (2007a). Semantic hashing. In Proceedings of the SIGIR Workshop on Graphical Models. Amsterdam.
13. Salakhutdinov, R., & Hinton, G. (2007b). Using deep belief nets to learn covariance kernels for gaussian processes. In J. C. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), Advances in neural information processing systems, 20. Cambridge, MA: MIT Press.