Modified convolutional neural network based on dropout and the stochastic gradient descent optimizer

Algorithms

Volumn 11, Issue 3, 2018, Pages

  Yang, Jing ^a   Yang, Guanci ^a  

^a GUIZHOU UNIVERSITY   (China)

Author keywords

Activate function; Convolutional neural network; Dropout; SGD optimizer

Indexed keywords

CONVOLUTION; GRADIENT METHODS; LEARNING ALGORITHMS; NEURAL NETWORKS; OPTIMIZATION; STOCHASTIC SYSTEMS; SUPPORT VECTOR MACHINES;

ACTIVATE FUNCTION; CONVOLUTIONAL NEURAL NETWORK; DROPOUT; NETWORK-BASED; NEURAL NETWORKS ALGORITHMS; OPTIMIZERS; PERFORMANCE; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT DESCENT OPTIMIZER; TIME COST;

BENCHMARKING;

EID: 85044101701     PISSN: None     EISSN: 19994893     Source Type: Journal    
DOI: 10.3390/a11030028     Document Type: Article

References (28)

1. Vieira, S., Pinaya, W.H.L., Mechelli, A. Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications. Neurosci. Biobehav. Rev. 2017 , 74 , 58-75.
2. Li, S., Dou, Y., Niu, X., Lv, Q., Wang, Q. A Fast and Memory Saved GPU Acceleration Algorithm of Convolutional Neural Networks for Target Detection. Neurocomputing 2017 , 230 , 48-59.
3. Gong, T., Fan, T., Guo, J., Cai, Z. GPU-based parallel optimization of immune convolutional neural network and embedded system. Eng. Appl. Artif. Intell. 2016 , 36 , 226-238.
4. Zhang, Y.N., Qu, L., Chen, J.W., Liu, J.R., Guo, D.S. Weights and structure determination method of multiple-input Sigmoid activation function neural network. Appl. Res. Comput. 2012 , 29 , 4113-4116.
5. Chen, L., Wu, C., Fan, W., Sun, J., Naoi, S. Adaptive Local Receptive Field Convolutional Neural Networks for Handwritten Chinese Character Recognition. In Chinese Conference on Pattern Recognition ; Springer: Berlin/Heidelberg, Germany, 2014; pp. 455-463.
6. Singh, P., Verma, A., Chaudhari, N.S. Deep Convolutional Neural Network Classifier for Handwritten Devanagari Character Recognition. In Information Systems Design and Intelligent Applications ; Springer: New Delhi, India, 2016.
7. Sun, W., Su, F. A novel companion objective function for regularization of deepconvolutional neural networks. Image Vis. Comput. 2016 , 56 , 110-126.
8. Wachinger, C., Reuter, M., Klein, T. DeepNAT: Deep convolutional neural network for segmenting neuroanatomy. NeuroImage 2017 .
9. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. Comput. Sci. 2016 , 26 , 357-361.
10. Izotov, P.Y., Kazanskiy, N.L., Golovashkin, D.L., Sukhanov, S.V. CUDA-enabled implementation of a neural network algorithm for handwritten digit recognition. Opt. Mem. Neural Netw. 2011 , 20 , 98-106.
11. Hao, H.W., Jiang, R.R. Training sample selection method for Neural Networks based on Nearest neighbor rule. Acta Autom. Sin. 2007 , 33 , 1247-1251.
12. Akeret, J., Chang, C., Lucchi, A., Refregier, A. Radio frequency interference mitigation using deep convolutional neural networks. Astron. Comput. 2017 , 18 , 35-39.
13. Costarelli, D., Vinti, G. Pointwise and uniform approximation by multivariate neural network operators of the max-product type. Neural Netw. 2016 , 81 , 81-90.
14. Lee, C.Y., Xie, S., Gallagher, P., Zhang, Z., Tu, Z. Deeply-supervised nets. In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, San Diego, CA, USA, 21 February 2015; pp. 562-570.
15. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998 , 86 , 2278-2323.
16. Zhou, F.-Y., Jin, L.-P., Dong, J. Review of Convolutional Neural Network. Chin. J. Comput. 2017 , 40 , 1229-1251.
17. Najafabadi, M.M., Khoshgoftaar, T.M., Villanustre, F., Holt, J. Large-scale distributed L-BFGS. J. Big Data 2017 , 4 , 22.
18. Zinkevich, M., Weimer, M., Li, L., Smola, A.J. Parallelized stochastic gradient descent. In Advances in Neural Information Processing Systems ; Neural Information Processing Systems Foundation, Inc.: La Jolla, CA, USA, 2010; pp. 2595-2603.
19. Hardt, M., Recht, B., Singer, Y. Train faster, generalize better: Stability of stochastic gradient descent. arXiv 2015 , arXiv:1509.01240.
20. Hou, Z.H., Fang, H.J. Robust fault-tolerant control for networked control system with packet dropout. J. Syst. Eng. Electron. 2007 , 18 , 76-82.
21. Luo, P., Li, H.F. Research on Quantum Neural Network and its Applications Based on Tanh Activation Function. Comput. Digit. Eng. 2012 , 16 , 33-39.
22. Tang, Z., Luo, L., Peng, H., Li, S. A joint residual network with paired ReLUs activation for image super-resolution. Neurocomputing 2018 , 273 , 37-46.
23. Günnemann, N., Pfeffer, J. Predicting Defective Engines using Convolutional Neural Networks on Temporal Vibration Signals. In Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications, Munich, Germany, 11 October 2017; pp. 92-102.
24. Jin, X., Xu, C., Feng, J., Wei, Y., Xiong, J., Yan, S. Deep learning with S-shaped rectified linear activation units. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI Press), Phoenix, AZ, USA, 12-17 February 2016; pp. 1737-1743.
25. Shi, X.B., Fang, X.J., Zhang, D.Y., Guo, Z.Q. Image Classification Based on Mixed Deep Learning Model Transfer Learning. J. Syst. Simul. 2016 , 28 , 167-173.
26. Yang, W., Jin, L., Tao, D., Xie, Z., Feng, Z. DropSample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition. Pattern Recognit. 2016 , 58 , 190-203.
27. Shen, F., Wang, L., Zhang, J. Reduced extreme learning machine employing SVM technique. J. Huazhong Univ. Sci. Technol. 2014 , 42 , 107-110.
28. Yang, G.-C., Yang, J., Su, Z.D., Chen, Z.-J. Improved YOLO feature extraction algorithm and its application to privacy situation detection of social robots. Acta Autom. Sin. 2018 , 1-12.