Techniques for learning binary stochastic feedforward neural networks

3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings

Volumn , Issue , 2015, Pages

  Raiko, Tapani ^a   Berglund, Mathias ^a   Alain, Guillaume ^b   Dinh, Laurent ^b  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; STOCHASTIC SYSTEMS;

BENCHMARK TESTS; GENERALIZATION PERFORMANCE; INTERNAL STRUCTURE; MULTI LAYER PERCEPTRON; POTENTIAL BENEFITS; STOCHASTIC NETWORKS; STRUCTURED PREDICTION; TRAINING ALGORITHMS;

FEEDFORWARD NEURAL NETWORKS;

EID: 85083952074     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian J., Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012 Workshop, 2012.
2. Bengio, Yoshua, Léonard, Nicholas, and Courville, Aaron. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013.
3. Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010. Oral Presentation.
4. Bishop, Christopher M et al. Pattern recognition and machine learning, volume 1. Springer New York, 2006.
5. Esser, Steve K, Andreopoulos, Alexander, Appuswamy, Rathinakumar, Datta, Pallab, Barch, Davis, Amir, Arnon, Arthur, John, Cassidy, Andrew, Flickner, Myron, Merolla, Paul, et al. Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores. In Neural Networks (IJCNN), The 2013 International Joint Conference on, pp. 1–10. IEEE, 2013.
6. Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
7. Kingma, Diederik P and Welling, Max. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114, 2013.
8. LeCun, Yann, Bottou, Léon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
9. Mnih, Andriy and Gregor, Karol. Neural variational inference and learning in belief networks. arXiv preprint arXiv:1402.0030, 2014.
10. Neal, Radford M. Learning stochastic feedforward networks. Department of Computer Science, University of Toronto, 1990.
11. Neal, Radford M. Connectionist learning of belief networks. Artificial intelligence, 56(1):71–113, 1992.
12. Raiko, Tapani, Valpola, Harri, and LeCun, Yann. Deep learning made easier by linear transformations in perceptrons. In International Conference on Artificial Intelligence and Statistics, pp. 924–932, 2012.
13. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan. Stochastic back-propagation and variational inference in deep latent Gaussian models. arXiv preprint arXiv:1401.4082, 2014.
14. Saul, Lawrence K, Jaakkola, Tommi, and Jordan, Michael I. Mean field theory for sigmoid belief networks. arXiv preprint cs/9603102, 1996.
15. Sietsma, Jocelyn and Dow, Robert JF. Creating artificial neural networks that generalize. Neural Networks, 4(1):67–79, 1991.
16. Susskind, Joshua, Anderson, Adam, and Hinton, Geoffrey. The Toronto face database. Technical report, University of Toronto, UTML TR 2010001, 06 2010.
17. Tang, Yichuan and Salakhutdinov, Ruslan. Learning stochastic feedforward neural networks. In Advances in Neural Information Processing Systems, pp. 530–538, 2013.
18. Weaver, Lex and Tao, Nigel. The optimal reward baseline for gradient-based reinforcement learning. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence, pp. 538–545. Morgan Kaufmann Publishers Inc., 2001.