Adaptive dropout for training deep neural networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Ba, Lei Jimmy ^a   Frey, Brendan ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS;

CLASSIFICATION ERROR RATE; DEEP NEURAL NETWORKS; FEATURE LEARNING; LOCAL EXPECTATIONS; NETWORK PARAMETERS; NEURAL NETWORK PARAMETERS; RESTRICTED BOLTZMANN MACHINE; STOCHASTIC GRADIENT DESCENT;

NEURAL NETWORKS;

EID: 84896515095     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. Advances in neural information processing systems, 19:153, 2007.
2. Yoshua Bengio. Learning deep architectures for ai. Foundations and TrendsR in Machine Learning, 2(1):1-127, 2009.
3. C.M. Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7(1):108-116, 1995.
4. A. Coates and A.Y. Ng. The importance of encoding versus training with sparse coding and vector quantization. In International Conference on Machine Learning, volume 8, page 10, 2011.
5. Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In International Conference on Machine Learning, 2010.
6. G.E. Hinton, S. Osindero, and Y.W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554, 2006.
7. G.E. Hinton and R.R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.
8. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R.R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
9. Kevin Jarrett, Koray Kavukcuoglu, MarcAurelio Ranzato, and Yann LeCun. What is the best multi-stage architecture for object recognition? In Computer Vision, 2009 IEEE 12th International Conference on, pages 2146-2153. IEEE, 2009.
10. A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25, 2012.
11. Yann LeCun, Fu Jie Huang, and Leon Bottou. Learning methods for generic object recognition with invariance to pose and lighting. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 2, pages II-97. IEEE, 2004.
12. V. Nair and G. Hinton. 3d object recognition with deep belief nets. Advances in Neural Information Processing Systems, 22:1339-1347, 2009.
13. V. Nair and G.E. Hinton. Rectified linear units improve restricted boltzmann machines. In Proc. 27th International Conference on Machine Learning, pages 807-814. Omnipress Madison, WI, 2010.
14. S. Rifai, P. Vincent, X. Muller, X. Glorot, and Y. Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the Twenty-eight International Conference on Machine Learning (ICML11), 2011.
15. Ruslan Salakhutdinov and Hugo Larochelle. Efficient learning of deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics. Citeseer, 2010.
16. J. Sietsma and R.J.F. Dow. Creating artificial neural networks that generalize. Neural Networks, 4(1):67-79, 1991.
17. Jasper Snoek, Ryan P Adams, and Hugo Larochelle. Nonparametric guidance of autoencoder representations using label information. Journal of Machine Learning Research, 13:2567- 2588, 2012.
18. Jasper Snoek, Hugo Larochelle, and Ryan Adams. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25, pages 2960- 2968, 2012.
19. Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. On the importance of initialization and momentum in deep learning.
20. Tijmen Tieleman. Gnumpy: an easy way to use gpu boards in python. Department of Computer Science, University of Toronto, 2010.
21. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11:3371-3408, 2010.