A spike and slab Restricted Boltzmann Machine

Journal of Machine Learning Research

Volumn 15, Issue , 2011, Pages 233-241

  Courville, Aaron ^a   Bergstra, James ^a   Bengio, Yoshua ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BINARY VARIABLES; GIBBS SAMPLING; HIDDEN LAYERS; RESTRICTED BOLTZMANN MACHINE;

OBJECT RECOGNITION;

ARTIFICIAL INTELLIGENCE;

EID: 84862276319     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (29)

1. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: a CPU and GPU math expression compiler. In SciPy, 2010.
2. M. A. Carreira-Perpiñan and G. E. Hinton. On contrastive divergence learning. In AISTATS 10, pages 33-40, 2005.
3. H. Chen and A. F. Murray. A continuous restricted Boltzmann machine with an implementable training algorithm. IEE P-Vis. Image Sign., 150(3):153-158, 2003.
4. D. C. Cireşan, U. Meier, J. Masci, L. M. Gambardella, and J. Schmidhuber. High-performance neural networks for visual object classification. arXiv, February 2011.
5. A. Coates, H. Lee, and A. Y. Ng. An analysis of singlelayer networks in unsupervised feature learning. In AISTATS 14, 2011.
6. Y. Freund and D. Haussler. Unsupervised learning of distributions on binary vectors using two layer networks. Technical Report UCSC-CRL-94-25, University of California, Santa Cruz, 1994.
7. R. Grosse, R. Raina, H. Kwong, and A. Y. Ng. Shiftinvariant sparse coding for audio classification. In UAI 2007, 2007.
8. G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14:1771-1800, 2002.
9. A. Krizhevsky. Convolutional deep belief networks on cifar-10. Technical report, University of Toronto, Aug 2010.
10. A. Krizhevsky and G. E. Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
11. H. Larochelle and Y. Bengio. Classification using discriminative restricted Boltzmann machines. In ICML 25, pages 536-543. ACM, 2008.
12. T. J. Mitchell and J. J. Beauchamp. Bayesian variable selection in linear regression. J. Amer. Statistical Assoc., 83(404):1023-1032, 1988.
13. V. Nair and G. E. Hinton. Implicit mixtures of restricted boltzmann machines. In NIPS 21, pages 1145-1152. MIT Press, 2009.
14. V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML 27, pages 807-814. ACM, 2010.
15. R. M. Neal. Bayesian Learning for Neural Networks. PhD thesis, Dept. of Computer Science, University of Toronto, 1994.
16. B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: a strategy employed by V1? Vision Research, 37:3311-3325, 1997.
17. M. Ranzato and G. H. Hinton. Modeling pixel means and covariance using factorized third-order boltzmann machines. In CVPR, pages 2551-2558. IEEE Press, 2010.
18. M. Ranzato, A. Krizhevsky, and G. E. Hinton. Factored 3-way restricted boltzmann machines for modeling natural images. In AISTATS 13, pages 621-628, 2010.
19. R. Salakhutdinov and G. E. Hinton. Deep Boltzmann machines. In AISTAT 12, pages 448-455, 2009.
20. R. Salakhutdinov, A. Mnih, and G. E. Hinton. Restricted Boltzmann machines for collaborative filtering. In ICML 24, pages 791-798. ACM, 2007.
21. I. Sutskever, G. E. Hinton, and G. Taylor. The recurrent temporal restricted boltzmann machine. In NIPS 21, pages 1601-1608. MIT Press, 2009.
22. G. Taylor and G. E. Hinton. Factored conditional restricted Boltzmann machines for modeling motion style. In ICML 26, pages 1025-1032. ACM, 2009.
23. T. Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In ICML 25, pages 1064-1071. ACM, 2008.
24. A. Torralba, R. Fergus, and W. T. Freeman. 80 million tiny images: A large dataset for non-parametric object and scene recognition. IEEE T. Pattern Anal., 30(11):1958-1970, 2008.
25. M. Welling, G. E. Hinton, and S. Osindero. Learning sparse topographic representations with products of Student-t distributions. In NIPS 15, pages 1359-1366. MIT Press, 2003.
26. C. K. I. Williams. Continuous-valued boltzmann machines. Unpublished Manuscript, March 1993.
27. C. K. I. Williams and F. V. Agakov. Products of Gaussians and Probabilistic Minor Component Analysis. Neural Computation, 14(5):1169-1182, 2002.
28. L. Younes. On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates. In Stochastics and Stochastics Models, pages 177-228, 1998.
29. R. S. Zemel, C. K. I. Williams, and M. Mozer. Directional-unit boltzmann machines. In NIPS 5, pages 172-179. MIT Press, 1993.