Rectified linear units improve Restricted Boltzmann machines

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 807-814

  Nair, Vinod ^a   Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DATA SETS; FACE VERIFICATION; FEATURE DETECTOR; HIDDEN UNITS; INFERENCE RULES; INFINITE NUMBERS; MULTIPLE LAYERS; NEGATIVE BIAS; RELATIVE INTENSITY; RESTRICTED BOLTZMANN MACHINE;

FACE RECOGNITION; LEARNING SYSTEMS; OBJECT RECOGNITION;

FEATURE EXTRACTION;

EID: 77956509090     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Bengio, Y. and LeCun, Y. Scaling learning algorithms towards AI. 2007.
2. Chopra, S., Hadsell, R., and LeCun, Y. Learning a similarity metric discriminatively, with application to face verification. In CVPR, pp. 539-546, Washington, DC, USA, 2005. IEEE Computer Society.
3. Freund, Y. and Haussler, D. Unsupervised learning of distributions on binary vectors using two layer networks. Technical report, Santa Cruz, CA, USA, 1994.
4. Hahnloser, Richard H. R., Seung, H. Sebastian, and Slotine, Jean-Jacques. Permitted and forbidden sets in symmetric threshold-linear networks. Neural Computation, 15(3):621-638, 2003. ISSN 0899-7667.
5. Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8): 1711-1800, 2002.
6. Hinton, G. E. and Salakhutdinov, R. Reducing the dimensionality of data with neural networks. Science, 313: 504-507, 2006.
7. Hinton, G. E., Sallans, B., and Ghahramani, Z. A hierarchical community of experts, pp. 479-494, 1999.
8. Hinton, G. E., Osindero, S., and Teh, Y. A fast learning algorithm for deep belief nets. Neural Computation, 18: 1527-1554, 2006.
9. Huang, G. B., Ramesh, M., Berg, T., and Learned-Miller, E. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. Technical Report 07-49, University of Massachusetts, Amherst, 2007.
10. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. What is the best multi-stage architecture for object recognition? In Proc. International Conference on Computer Vision (ICCV'09). IEEE, 2009.
11. Kumar, N., Berg, A. C., Belhumeur, P. N., and Nayar, S. K. Attribute and simile classifiers for face verification. In International Conference on Computer Vision, 2009.
12. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., and Bengio., Y. An empirical evaluation of deep architectures on problems with many factors of variation. In ICML, pp. 473-480, 2007.
13. LeCun, Y., Huang, F. J., and Bottou., L. Learning methods for generic object recognition with invariance to pose and lighting. In CVPR, Washington, D.C., 2004.
14. Marks, T. K. and Movellan, J. R. Diffusion networks, products of experts, and factor analysis. Technical Report UCSD MPLab TR 2001.02, 2001.
15. Mohamed, A. and Hinton, G. E. Phone recognition using restricted boltzmann machines. In ICASSP, Dallas, TX, USA, 2010.
16. Nair, V. and Hinton, G. E. Implicit mixtures of restricted boltzmann machines. In Neural information processing systems, 2008.
17. Salakhutdinov, R. and Hinton, G. E. Replicated softmax: an undirected topic model. In Advances in Neural Information Processing Systems 22, 2009.
18. Salakhutdinov, R., Mnih, A., and Hinton, G. E. Restricted Boltzmann machines for collaborative filtering. In Proceedings of the International Conference on Machine Learning, volume 24, pp. 791-798, 2007.
19. Taylor, G. W., Hinton, G. E., and Roweis, S. Modeling human motion using binary latent variables. In Advances in Neural Information Processing Systems 19, Cambridge, MA, 2006. MIT Press.
20. Teh, Y.W. and Hinton, G. E. Rate-coded restricted boltzmann machines for face recognition. In Advances in Neural Information Processing Systems, volume 13, 2001.
21. Wolf, L., Hassner, T., and Taigman, Y. Similarity scores based on background samples. In Asian Conference on Computer Vision, 2009.