Practical Block-Wise Neural Network Architecture Generation

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2018, Pages 2423-2432

  Zhong, Zhao ^a,c   Yan, Junjie ^b   Wu, Wei ^b   Shao, Jing ^b   Liu, Cheng Lin ^a,c,d  

^a INSTITUTE OF AUTOMATION   (China)

^b SENSETIME RESEARCH   (China)

^c UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^d Shenzhen Fundamental Research Institutions   (China)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; NEURAL NETWORKS; PROGRAM PROCESSORS;

ASYNCHRONOUS FRAMEWORK; CONVOLUTIONAL NEURAL NETWORK; GENERATION PROCESS; GREEDY EXPLORATION; HIGH PERFORMANCE NETWORKS; NETWORK GENERATION; OPTIMAL NETWORKS; STATE OF THE ART;

NETWORK ARCHITECTURE;

EID: 85062863340     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2018.00257     Document Type: Conference Paper

References (37)

1. M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, and N. de Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, pages 3981-3989, 2016. 3
2. B. Baker, O. Gupta, N. Naik, and R. Raskar. Designing neural network architectures using reinforcement learning. In 6th International Conference on Learning Representations, 2017. 1, 2, 4, 5, 6, 7, 8
3. J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems, pages 2546-2554, 2011. 3
4. F. Chollet. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. 8
5. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, A. Senior, P. Tucker, K. Yang, Q. V. Le, et al. Large scale distributed deep networks. In Advances in neural information processing systems, pages 1223-1231, 2012. 5
6. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248-255. IEEE, 2009. 2
7. T. Domhan, J. T. Springenberg, and F. Hutter. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In IJCAI, pages 3460-3468, 2015. 2
8. K. He and J. Sun. Convolutional neural networks at constrained time cost. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5353-5360, 2015. 5
9. K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015. 6
10. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016. 1, 2, 7
11. K. He, X. Zhang, S. Ren, and J. Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pages 630-645. Springer, 2016. 1, 2, 3, 7, 8
12. S. Hochreiter, A. S. Younger, and P. R. Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, pages 87-94. Springer, 2001. 3
13. G. Huang, Z. Liu, K. Q. Weinberger, and L. van der Maaten. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017. 7
14. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pages 448-456, 2015. 1, 2, 8
15. D. Kingma and J. Ba. Adam: A method for stochastic optimization. In 3rd International Conference for Learning Representations, 2015. 6
16. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems, pages 1097-1105, 2012. 1
17. Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521(7553):436-444, 2015. 1
18. M. Li, L. Zhou, Z. Yang, A. Li, F. Xia, D. G. Andersen, and A. Smola. Parameter server for distributed machine learning. In Big Learning NIPS Workshop, volume 6, page 2, 2013. 5
19. L.-J. Lin. Reinforcement learning for robots using neural networks. Technical report, Carnegie-Mellon Univ Pittsburgh PA School of Computer Science, 1993. 1, 6
20. M. Lin, Q. Chen, and S. Yan. Network in network. In International Conference on Learning Representations, 2013. 4
21. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015. 1, 6
22. A. Y. Ng, D. Harada, and S. Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, volume 99, pages 278-287, 1999. 5
23. S. Saxena and J. Verbeek. Convolutional neural fabrics. In Advances in Neural Information Processing Systems, pages 4053-4061, 2016. 2
24. J. D. Schaffer, D. Whitley, and L. J. Eshelman. Combinations of genetic algorithms and neural networks: A survey of the state of the art. In Combinations of Genetic Algorithms and Neural Networks, 1992., COGANN-92. International Workshop on, pages 1-37. IEEE, 1992. 2
25. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference for Learning Representations, 2015. 1, 7, 8
26. K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based encoding for evolving large-scale neural networks. Artificial life, 15(2):185-212, 2009. 2
27. K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary computation, 10(2):99-127, 2002. 2
28. M. Suganuma, S. Shirakawa, and T. Nagao. A genetic programming approach to designing convolutional neural network architectures. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 497-504, 2017. 2
29. R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction, volume 1. mit press Cambridge, 1998. 5
30. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pages 1-9, 2015. 1, 2, 8
31. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2818-2826, 2016. 1, 2
32. R. Vilalta and Y. Drissi. A perspective view and survey of meta-learning. Artificial Intelligence Review, 18(2):77-95, 2002. 3
33. C. J. C. H. Watkins. Learning from delayed rewards. PhD thesis, King's College, Cambridge, 1989. 1
34. L. Xie and A. Yuille. Genetic cnn. In Proceedings of the International Conference on Computer Vision, 2017. 2
35. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5987-5995. IEEE, 2017. 8
36. S. Zagoruyko and N. Komodakis. Wide residual networks. In British Machine Vision Conference, 2016. 7
37. B. Zoph and Q. V. Le. Neural architecture search with reinforcement learning. In 6th International Conference on Learning Representations, 2017. 1, 2, 5, 7, 8