Weighted residuals for very deep networks

2016 3rd International Conference on Systems and Informatics, ICSAI 2016

Volumn , Issue , 2017, Pages 936-941

  Shen, Falong ^a   Gan, Rui ^a   Zeng, Gang ^a  

^a PEKING UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

DIFFERENT LAYERS; FASTER CONVERGENCE; HIGH-ACCURACY; NETWORK INITIALIZATION; RESIDUAL STRUCTURE; STOCHASTIC GRADIENT DESCENT; WEIGHTED RESIDUALS;

STOCHASTIC SYSTEMS;

EID: 85015337212     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICSAI.2016.7811085     Document Type: Conference Paper

References (20)

1. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al., "Imagenet large scale visual recognition challenge, " International Journal of Computer Vision, vol. 115, no. 3, pp. 211-252, 2015.
2. C. Szegedy, S. Ioffe, and V. Vanhoucke, "Inception-v4, inception-resnet and the impact of residual connections on learning, " arXiv preprint arXiv:1602.07261, 2016.
3. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition, " arXiv preprint arXiv:1512.03385, 2015.
4. K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks, " arXiv preprint arXiv:1603.05027, 2016.
5. S. Targ, D. Almeida, and K. Lyman, "Resnet in resnet: Generalizing residual architectures, " arXiv preprint arXiv:1603.08029, 2016.
6. G. Huang, Y. Sun, Z. Liu, D. Sedra, and K. Weinberger, "Deep networks with stochastic depth, " arXiv preprint arXiv:1603.09382, 2016.
7. N. K. Sergey Zagoruyko, "Deep networks with stochastic depth, " arXiv preprint arXiv:1605.07146, 2016.
8. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Advances in neural information processing systems, pp. 1097-1105, 2012.
9. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " arXiv preprint arXiv:1409.1556, 2014.
10. J. Dai, K. He, and J. Sun, "Instance-Aware semantic segmentation via multi-Task network cascades, " arXiv preprint arXiv:1512.04412, 2015.
11. S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-Time object detection with region proposal networks, " in Advances in Neural Information Processing Systems, pp. 91-99, 2015.
12. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift, " arXiv preprint arXiv:1502.03167, 2015.
13. R. K. Srivastava, K. Greff, and J. Schmidhuber, "Training very deep networks, " in Advances in Neural Information Processing Systems, pp. 2368-2376, 2015.
14. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks, " in International conference on artificial intelligence and statistics, pp. 249-256, 2010.
15. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, " in Proceedings of the IEEE International Conference on Computer Vision, pp. 1026-1034, 2015.
16. A. Krizhevsky, "Learning multiple layers of features from tiny images, " 2009.
17. A. Shah, E. Kadam, H. Shah, and S. Shinde, "Deep residual networks with exponential linear unit, " arXiv preprint arXiv:1604.04112, 2016.
18. R. Hecht-Nielsen, "Theory of the backpropagation neural network, " in Neural Networks, 1989. IJCNN., International Joint Conference on, pp. 593-605, IEEE, 1989.
19. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding, " in Proceedings of the ACM International Conference on Multimedia, pp. 675-678, ACM, 2014.
20. C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu, "Deeply-supervised nets, " arXiv preprint arXiv:1409.5185, 2014.