Path-level network transformation for efficient architecture search

35th International Conference on Machine Learning, ICML 2018

Volumn 2, Issue , 2018, Pages 1069-1080

  Cai, Han ^a   Yang, Jiacheng ^a   Zhang, Weinan ^a   Han, Song ^b   Yu, Yong ^a  

^a SHANGHAI JIAO TONG UNIVERSITY   (China)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); EFFICIENCY; FORESTRY; IMAGE ENHANCEMENT; REINFORCEMENT LEARNING; TOPOLOGY;

CLASSIFICATION DATASETS; COMPUTATIONAL RESOURCES; EFFICIENT ARCHITECTURE; MOBILE SETTINGS; NETWORK TRANSFORMATION; NEURAL ARCHITECTURES; TEST ACCURACY; TREE-STRUCTURED;

NETWORK ARCHITECTURE;

EID: 85057246386     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (41)

1. Ashok, A., Rhinehart, N., Beainy, F., and Kitani, K. M. N2n learning: Network to network compression via policy gradient reinforcement learning. ICLR, 2018.
2. Baker, B., Gupta, O., Naik, N., and Raskar, R. Designing neural network architectures using reinforcement learning. ICLR, 2017.
3. Bergstra, J. and Bengio, Y. Random search for hyperparameter optimization. Journal of Machine Learning Research, 2012.
4. Cai, H., Chen, T., Zhang, W., Yu, Y., and Wang, J. Efficient architecture search by network transformation. In AAAI, 2018.
5. Chen, T., Goodfellow, I., and Shlens, J. Net2net: Accelerating learning via knowledge transfer. ICLR, 2016.
6. Chollet, F. Xception: Deep learning with depthwise separable convolutions. arXiv preprint, 2016.
7. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
8. DeVries, T. and Taylor, G. W. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552, 2017.
9. Domhan, T., Springenberg, J. T., and Hutter, F. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In IJCAI, 2015.
10. Gastaldi, X. Shake-shake regularization. arXiv preprint arXiv:1705.07485, 2017.
11. Han, D., Kim, J., and Kim, J. Deep pyramidal residual networks. CVPR, 2017.
12. Han, S., Pool, J., Tran, J., and Dally, W. Learning both weights and connections for efficient neural network. In NIPS, 2015.
13. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In CVPR, 2016.
14. Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
15. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural computation, 1997.
16. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.
17. Huang, G., Liu, S., van der Maaten, L., and Weinberger, K. Q. Condensenet: An efficient densenet using learned group convolutions. arXiv preprint arXiv:1711.09224, 2017a.
18. Huang, G., Liu, Z., Weinberger, K. Q., and van der Maaten, L. Densely connected convolutional networks. CVPR, 2017b.
19. Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
20. Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
21. Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. 2009.
22. Liu, C., Zoph, B., Shlens, J., Hua, W., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., and Murphy, K. Progressive neural architecture search. arXiv preprint arXiv:1712.00559, 2017.
23. Liu, H., Simonyan, K., Vinyals, O., Fernando, C., and Kavukcuoglu, K. Hierarchical representations for ef-ficient architecture search. ICLR, 2018.
24. Loshchilov, I. and Hutter, F. Sgdr: Stochastic gradient descent with restarts. arXiv preprint arXiv:1608.03983, 2016.
25. Mendoza, H., Klein, A., Feurer, M., Springenberg, J. T., and Hutter, F. Towards automatically-tuned neural networks. In Workshop on Automatic Machine Learning, 2016.
26. Negrinho, R. and Gordon, G. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792, 2017.
27. Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., and Dean, J. Efficient neural architecture search via parameter sharing. arXiv preprint arXiv:1802.03268, 2018.
28. Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y. L., Le, Q., and Kurakin, A. Large-scale evolution of image classifiers. ICML, 2017.
29. Real, E., Aggarwal, A., Huang, Y., and Le, Q. V. Regularized evolution for image classifier architecture search. arXiv preprint arXiv:1802.01548, 2018.
30. Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
31. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. Going deeper with convolutions. In CVPR, 2015.
32. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. In CVPR, 2016.
33. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, 2017.
34. Tai, K. S., Socher, R., and Manning, C. D. Improved semantic representations from tree-structured long short-term memory networks. ACL, 2015.
35. Veit, A., Wilber, M. J., and Belongie, S. Residual networks behave like ensembles of relatively shallow networks. In NIPS, 2016.
36. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Reinforcement Learning. 1992.
37. Xie, S., Girshick, R., Dollar, P., Tu, Z., and He, K. Aggre-gated residual transformations for deep neural networks. In CVPR, 2017.
38. Zhang, X., Zhou, X., Lin, M., and Sun, J. Shufflenet: An extremely efficient convolutional neural network for mobile devices. arXiv preprint arXiv:1707.01083, 2017.
39. Zhong, Z., Yan, J., and Liu, C.-L. Practical network blocks design with q-learning. arXiv preprint arXiv:1708.05552, 2017.
40. Zoph, B. and Le, Q. V. Neural architecture search with reinforcement learning. ICLR, 2017.
41. Zoph, B., Vasudevan, V., Shlens, J., and Le, Q. V. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012, 2017.