Understanding and improving convolutional neural networks via concatenated rectified linear units

33rd International Conference on Machine Learning, ICML 2016

Volumn 5, Issue , 2016, Pages 3276-3284

  Shang, Wenling ^a,d   Sohn, Kihvuk ^b   Almeida, Diogo ^c   Lee, Honglak ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

^b NEC LABORATORIES AMERICA   (United States)

^c Enlitic   (United States)

^d NONE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER VISION; CONVOLUTION; NETWORK ARCHITECTURE; NEURAL NETWORKS;

CNN MODELS; CONVOLUTIONAL NEURAL NETWORK; GENERIC METHOD; LINEAR UNITS; RECONSTRUCTION PROPERTY; STATE OF THE ART;

LEARNING SYSTEMS;

EID: 84998679622     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Bruna, J. and Mallat, S. Invariant scattering convolution networks. PAMI, 2013.
2. Bruna, J., Szlam, A., and LeCun, Y. Signal recovery from pooling representations. In ICML, 2013.
3. Christensen, O. An introduction to frames and Riesz. bases. Birkhuser Basel, 2003.
4. Deng, J., Dong, W, Socher, R, Li, L.-j, Li, K, and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
5. Dosovitskiy, A. and Brox, T. Inverting convolutional networks with convolutional networks. In CVPR, 2015.
6. Girshick, R., Donahue, J., Darrell, T., and Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.
7. Goodfellow, I., Lee, H, Le, Q. V., Saxe, A., and Ng, A. Measuring invariances in deep networks. In NIPS, 2009.
8. Goodfellow, I., Warde-Farley, D., Mirza, M., Courville, A., and Bengio, Y. Maxout networks. In ICML, 2013.
9. Han, S., Pool, J., Tran, J., and Dally, W. J. Learning both weights and connections for efficient neural network. In NIPS, 2015.
10. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
11. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. What is the best multi-stage architecture for object recognition? In CVPR, 2009.
12. Krizhevsky, A. Learning multiple layers of features from tiny images, 2009.
13. Krizhevsky, A., Sutskever, I., and Hinton, G. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
14. Lee, C.-y., Gallagher, P. W., and Tu, Z. Generalizing pooling functions in convolutional neural networks: Mixed, gated, and tree. In AISTATS, 2016.
15. Liang, M. and Hu, X. Recurrent convolutional neural network for object recognition. In CVPR, 2015.
16. Lin, M., Chen, Q., and Yan, S. Network in network. In ICLR, 2013.
17. Maas, A., Hannun, A. Y., and Ng, A. Rectifier nonlineari-ties improve neural network acoustic models. In ICML, .
18. Mahendran, A. and Vedaldi, A. Understanding deep image representations by inverting them. In CVPR, 2015.
19. Nair, V. and Hinton, G. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
20. Rippel, O., Snoek, J., and Adams, R. Spectral representations for convolutional neural networks. In NIPS, 2015.
21. Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In ICLR, .
22. Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., Patwary, M. M. A., and Adams, R. Scalable Bayesian optimization using deep neural networks. In ICML, 2015.
23. Springenberg, J., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. In ICLR Workshop, 2014.
24. Srivastava, R., Greff, K., and Schmidhuber, J. Training very deep networks. In NIPS, 2015.
25. Szegedy, C., Liu, W, Jia, Y., Sermanet, P., Reed, S, Anguelov, D., Erhan, D., Vanhoucke, V., and Rabi-novich, A. Going deeper with convolutions. In CVPR, 2015.
26. Wan, L., Zeiler, M., Zhang, S., LeCun, Y., and Fergus, R. Regularization of neural networks using dropconnect. In ICML, 2013.
27. Xu, B., Wang, N, Chen, T., and Li, M. Empirical evaluation of rectified activations in convolutional network. In ICML Workshop, 2015.
28. Yang, Z., Moczulski, M., Denil, M., de Freitas, N., Smola, A., Song, L., and Wang, Z. Deep fried convnets. In ICCV, 2015.
29. Zagoruyko, S. Torch blog. http://torch.ch/blog/2015/07/30/cifar.html, 2015.
30. Zeiler, M. D. and Fergus, R. Stochastic pooling for regularization of deep convolutional neural networks. In ICLR, 2013.
31. Zhao, J., Mathieu, M., Goroshin, R., and Lecun, Y. Stacked what-where auto-encoders. In ICLR, 2015.