Rank-based pooling for deep convolutional neural networks

Neural Networks

Volumn 83, Issue , 2016, Pages 21-31

  Shi, Zenglin ^a   Ye, Yangdong ^a   Wu, Yunpeng ^a  

^a ZHENGZHOU UNIVERSITY   (China)

Author keywords

Convolutional neural network; Deep learning; Image classification; Pooling

Indexed keywords

ARTIFICIAL INTELLIGENCE; BENCHMARKING; CHEMICAL ACTIVATION; COMPUTER VISION; CONVOLUTION; LEARNING SYSTEMS; NEURAL NETWORKS; STOCHASTIC SYSTEMS;

CLASSIFICATION PERFORMANCE; CONVOLUTIONAL NEURAL NETWORK; DEEP LEARNING; POOLING; ROBUST PERFORMANCE; TRANSLATION INVARIANCE; VISION COMMUNITIES; WEIGHTING STRATEGIES;

IMAGE CLASSIFICATION;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; DEEP CONVOLUTIONAL NEURAL NETWORK; MACHINE LEARNING; NETWORK LEARNING; PRIORITY JOURNAL; RANK BASED AVERAGE POOLING; RANK BASED POOLING; RANK BASED STOCHASTIC POOLING; RANK BASED WEIGHTED POOLING; STATISTICAL ANALYSIS; STATISTICAL CONCEPTS; STATISTICAL PARAMETERS;

MACHINE LEARNING; NEURAL NETWORKS (COMPUTER);

EID: 84982149359     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2016.07.003     Document Type: Article

References (41)

1. Bing Xu, T. C., Wang, Naiyan, & Li, M. (2015). Empirical evaluation of rectified activations in convolutional network. CoRR, abs/1505.00853.
2. Boureau, Y.-L., Ponce, J., & LeCun, Y. (2010). A theoretical analysis of feature pooling in visual recognition. In Proceedings of the 27th international conference on machine learning (pp. 111–118).
3. Boureau, Y.-L., Roux, N. L., Bach, F., Ponce, J., & LeCun, Y. (2011). Ask the locals: multi-way local pooling for image recognition. In 2011 IEEE international conference on computer vision (pp. 2651–2658).
4. Breiman, L., Bagging predictors. Machine Learning 24 (1996), 123–140.
5. Cai, M., Shi, Y., & Liu, J. (2014). Stochastic pooling maxout networks for low-resource speech recognition. In 2014 IEEE international conference on acoustics, speech and signal processing (pp. 3266–3270).
6. Ciresan, D., Meier, U., & Schmidhuber, J. (2012). Multi-column deep neural networks for image classification. In 2012 IEEE conference on computer vision and pattern recognition (pp. 3642–3649).
7. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In 2005 IEEE conference on computer vision and pattern recognition (pp. 886–893).
8. Feng, J., Ni, B., Tian, Q., & Yan, S. (2011). Geometric lp-norm feature pooling for image classification. In 2011 IEEE conference on computer vision and pattern recognition (pp. 2609–2704).
9. Fukushima, K., Miyake, S., Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition 15 (1982), 455–469.
10. Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013). Maxout networks. In Proceedings of the 30th international conference on machine learning (pp. 1319–1327).
11. Graham, B. (2015). Fractional max-pooling. CoRR, abs/1412.6071.
12. Gulcehre, C., Cho, K., Pascanu, R., Bengio, Y., Learned-norm pooling for deep feedforward and recurrent neural networks. Machine learning and knowledge discovery in databases, 2014, 530–546.
13. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. CoRR, abs/1502.01852.
14. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580.
15. Hubel, D.H., Wiesel, T.N., Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology, 160, 1962, 106.
16. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014). Caffe: Convolutional architecture for fast feature embedding. CoRR, abs/1408.5093.
17. Kahou, S. E., Bouthillier, X., Lamblin, P., & Gulcehre, C. (2015). Emonets: Multimodal deep learning approaches for emotion recognition in video. CoRR, abs/1503.01800.
18. Koenderink, J.J., Van Doorn, A.J., The structure of locally orderless images. International Journal of Computer Vision 31 (1999), 159–168.
19. Krizhevsky, A., Hinton, G., Learning multiple layers of features from tiny images. Tech. Rep., 1., 2009, Computer Science Department, University of Toronto, 7.
20. Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 2012, 1097–1105.
21. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (1998), 2278–2324.
22. LeCun, Y., Huang, F. J., & Bottou, L. (2004). Learning methods for generic object recognition with invariance to pose and lighting. In 2004 IEEE computer society conference on computer vision and pattern recognition, Vol. 2 (pp. II-97-104).
23. Lee, C.-Y., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014). Deeply-supervised nets. In Proceedings of the 18th international conference on artificial intelligence and statistics (pp. 562–570).
24. Lin, M., Chen, Q., & Yan, S. (2014). Network in network. CoRR, abs/1312.4400.
25. Lowe, D.G., Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60 (2004), 91–110.
26. Maas, A. L., Hannun, A. Y., & Ng, A. Y. (2013). Rectifier nonlinearities improve neural network acoustic models. In Proceedings of the 27th international conference on machine learning, Vol. 30.
27. Michalewicz, Z., Genetic algorithms+ data structures= evolution programs. 2013, Springer Science & Business Media.
28. Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (pp. 807–814).
29. Riesenhuber, M., Poggio, T., Hierarchical models of object recognition in cortex. Nature Neuroscience 2 (1999), 1019–1025.
30. Sainath, T.N., Kingsbury, B., Saon, G., Soltau, H., Mohamed, A.-r., Dahl, G., Ramabhadran, B., Deep convolutional neural networks for large-scale speech tasks. Neural Networks 64 (2015), 39–48.
31. Sermanet, P., Chintala, S., & LeCun, Y. (2012). Convolutional neural networks applied to house numbers digit classification. In 2012 international conference on pattern recognition (pp. 3288–3291).
32. Silva, L.M., de Sá, J.M., Alexandre, L.A., Data classification with multilayer perceptrons using a generalized error function. Neural Networks 21 (2008), 1302–1310.
33. Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556.
34. Srivastava, N., Improving neural networks with dropout. (Master's thesis), 2013, University of Toronto.
35. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research 15 (2014), 1929–1958.
36. Srivastava, R. K., Masci, J., Kazerounian, S., Gomez, F., & Schmidhuber, J. (2013). Compete to compute. (pp. 2310–2318).
37. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014). Going deeper with convolutions. In 2015 IEEE conference on computer vision and pattern recognition (pp. 1–9).
38. Wan, L., Zeiler, M., Zhang, S., Cun, Y. L., & Fergus, R. (2013). Regularization of neural networks using dropconnect. In Proceedings of the 30th international conference on machine learning (pp. 1058–1066).
39. Wang, M. (2014). Multi-path convolutional neural networks for complex image classification. CoRR, abs/1506.04701.
40. Zeiler, M. D., & Fergus, R. (2013). Stochastic pooling for regularization of deep convolutional neural networks. CoRR, abs/1301.3557.
41. Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision, Vol. 2014 (pp. 818–833).