Restricted Boltzmann Machines are hard to approximately evaluate or simulate

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 703-710

  Long, Philip M ^a   Servedio, Rocco A ^b  

^a GOOGLE INC   (United States)

^b Columbia University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL TASK; FOLLOWING PROBLEM; HARDNESS RESULT; HIDDEN NODES; INPUT STRING; MULTIPLICATIVE FACTORS; POLYNOMIAL-TIME; POLYNOMIAL-TIME ALGORITHMS; POSITIVE CONSTANT; PROBABILITY MODELS; RANDOMIZED ALGORITHMS; RESTRICTED BOLTZMANN MACHINE; TOTAL VARIATION;

LEARNING SYSTEMS; PARAMETER ESTIMATION;

PROBABILITY DISTRIBUTIONS;

EID: 77956527887     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. Abe, N. and Warmuth, M. K. On the computational complexity of approximating distributions by probabilistic automata. Machine Learning, 9(2-3):205-260, 1992.
2. Alon, N. and Naor, A. Approximating the cut-norm via Grothendieck's inequality. In STOC, pp. 72-80, 2004.
3. Bengio, Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
4. Bengio, Y. and Delalleau, O. Justifying and generalizing contrastive divergence. Neural Computation, 21 (6):1601-1621, 2009.
5. Bogdanov, A., Mossel, E., and Vadhan, S. P. The complexity of distinguishing markov random fields. In APPROX-RANDOM, pp. 331-342, 2008.
6. Erhan, D., Bengio, Y., Courville, A., Manzagol, P., Vincent, P., and Bengio, S. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research. To appear.
7. Freund, Y. and Haussler, D. Unsupervised learning of distributions of binary vectors using 2-layer networks. In NIPS, pp. 912-919, 1991.
8. Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771-1800, 2002.
9. Hinton, G. E., Osindero, S., and Teh, Y. W. A fast learning algorithm for deep belief nets. Neural Computation, 18(7):1527-1554, 2006.
10. Jerrum, M., Valiant, L. G., and Vazirani, V. V. Random generation of combinatorial structures from a uniform distribution. Theor. Comput. Sci, 43:169-188, 1986.
11. Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R., and Sellie, L. On the learnability of discrete distributions. In Proc. of Twenty-sixth ACM Symposium on Theory of Computing, pp. 273-282, 1994.
12. Larochelle, H., Erhan, D., Courville, A. C., Bergstra, J., and Bengio, Y. An empirical evaluation of deep architectures on problems with many factors of variation. In ICML, pp. 473-480, 2007.
13. Roth, D. On the hardness of approximate reasoning. Artificial Intelligence, 82(1-2):273-302, 1996.
14. Smolensky, P. Information processing in dynamical systems: Foundations of harmony theory. In Rumel-hart, D. E., McClelland, J. L., et al. (eds.), Parallel Distributed Processing: Volume 1: Foundations, pp. 194-281. MIT Press, Cambridge, 1987.
15. Yasuda, M. and Tanaka, K. Approximate learning algorithm for restricted boltzmann machines. In CIMCA/IAWTIC/ISE, pp. 692-697, 2008.