Towards dropout training for convolutional neural networks

Neural Networks

Volumn 71, Issue , 2015, Pages 1-10

  Wu, Haibing ^a   Gu, Xiaodong ^a  

^a FUDAN UNIVERSITY   (China)

Author keywords

Convolutional neural networks; Deep learning; Max pooling dropout

Indexed keywords

NEURAL NETWORKS; STOCHASTIC SYSTEMS;

CONVOLUTIONAL NEURAL NETWORK; DATA AUGMENTATION; DEEP LEARNING; MAX-POOLING; MODEL AVERAGING; MULTINOMIAL DISTRIBUTIONS; STATE-OF-THE-ART PERFORMANCE; STOCHASTICITY;

CONVOLUTION;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CONVOLUTIONAL NEURAL NETWORK; INFORMATION PROCESSING; INTERMETHOD COMPARISON; MACHINE LEARNING; MATHEMATICAL PARAMETERS; MAX POOLING DROPOUT; PRIORITY JOURNAL; PROBABILISTIC WEIGHTED POOLING; PROCESS OPTIMIZATION; QUALITY CONTROL; STOCHASTIC POOLING; VALIDATION STUDY; ALGORITHM; MARKOV CHAIN; STATISTICAL MODEL;

ALGORITHMS; MACHINE LEARNING; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); STOCHASTIC PROCESSES;

EID: 84939541134     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2015.07.007     Document Type: Article

References (20)

1. Baldi P., Sadowski P. The dropout learning algorithm. Artificial Intelligence 2014, 210:78-122.
2. Bengio Y., Courville A., Vincent P. Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 2013, 35:1798-1828.
3. Breiman L. Bagging predictors. Machine Learning 1996, 24:123-140.
4. Ciresan, D., Meier, U., & Schmidhuber, J. (2012). Multi-column deep neural networks for image classification. In Proceedings of 2014 IEEE conference on computer vision and pattern recognition. CVPR 2012.
5. Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013). Maxout networks. In Proceedings of 30th international conference on machine learning. ICML 2013.
6. Hinton G.E., Salakhutdinov R.R. Reducing the dimensionality of data with neural networks. Science 2006, 313:504-507.
7. Hinton, G. E., Srivastave, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaption of feature detectors. arxiv:1207.0580.
8. Krizhevsky A. Learning multiple layers of features from tiny images 2009, (M.S. diss), Department of Computer Science, University of Toronto.
9. Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Advances in neural information processing systems 2012.
10. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE.
11. Ledoux M., Talagrand M. Probability in Banach spaces 1991, Springer.
12. Lin, M., Chen, Q., & Yan, S. (2014). Network in network. In Proceedings of 3rd international conference on learning representations. ICLR 2014.
13. Mackay, D. C. (1995). Probable networks and plausible predictions: A review of practical Bayesian methods for supervised neural networks. In Bayesian methods for backpropagation networks.
14. Springenberg, J. T., & Riedmiller, M. (2014). Improving deep neural networks with probabilistic maxout units. In Proceedings of 3rd international conference on learning representations. ICLR 2014.
15. Srivastava N., Hinton G.E., Krizhevsky A., Sutskever I., Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 2014, 15:1929-1958.
16. Vinod, N., & Hinton, G. E. (2010). Rectified linear units improve restricted Boltzmann machines. In Proceedings 27th of international conference on machine learning. ICML 2010.
17. Wager S., Wang S., Liang P. Dropout training as adaptive regularization. Advances in neural information processing systems 2013.
18. Wan, L., Zeiler, M. D., Zhang, S., LeCun, Y., & Fergus, R. (2013). Regularization of neural networks using DropConnect. In Proceedings of 30th international conference on machine learning. ICML 2013.
19. Warde, F. D., Goodfellow, I. J., Courville, A., & Bengio, Y. (2014). An empirical analysis of dropout in piecewise linear networks. In Proceedings of 3rd international conference on learning representations. ICLR 2014.
20. Zeiler, M. D., & Fergus, R. (2013). Stochastic pooling for regularization of deep convolutional neural networks. In Proceedings of 2nd international conference on learning representations. ICLR 2013.