Understanding dropout

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Baldi, Pierre ^a   Sadowski, Peter ^a  

^a Irvine   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FEATURE EXTRACTION;

CO-ADAPTATION; DEEP NEURAL NETWORKS; ERROR FUNCTION; FEATURE DETECTOR; GEOMETRIC MEAN; RECURSIVE EQUATIONS; STOCHASTIC GRADIENT DESCENT;

NEURAL NETWORKS;

EID: 84896497059     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. P. Baldi and P. Sadowski. The Dropout Learning Algorithm. Artificial Intelligence, 2014. In press.
2. E. F. Beckenbach and R. Bellman. Inequalities. Springer-Verlag Berlin, 1965.
3. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), Austin, TX, June 2010. Oral Presentation.
4. L. Bottou. Online algorithms and stochastic approximations. In D. Saad, editor, Online Learning and Neural Networks. Cambridge University Press, Cambridge, UK, 1998.
5. L. Bottou. Stochastic learning. In O. Bousquet and U. von Luxburg, editors, Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3176, pages 146-168. Springer Verlag, Berlin, 2004.
6. D. Cartwright and M. Field. A refinement of the arithmetic mean-geometric mean inequality. Proceedings of the American Mathematical Society, pages 36-38, 1978.
7. G. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. http://arxiv.org/abs/1207.0580, 2012.
8. E. Neuman and J. Sándor. On the Ky Fan inequality and related inequalities i. MATHEMATICAL INEQUALITIES AND APPLICATIONS, 5:49-56, 2002.
9. E. Neuman and J. Sandor. On the Ky Fan inequality and related inequalities ii. Bulletin of the Australian Mathematical Society, 72(1):87-108, 2005.
10. S. Nitish. Improving Neural Networks with Dropout. PhD thesis, University of Toronto, Toronto, Canada, 2013.
11. H. Robbins and D. Siegmund. A convergence theorem for non negative almost supermartingales and some applications. Optimizing methods in statistics, pages 233-257, 1971.
12. D. Warde-Farley, I. Goodfellow, P. Lamblin, G. Desjardins, F. Bastien, and Y. Bengio. pylearn2. 2011. http://deeplearning.net/software/pylearn2.