Efficient architecture search by network transformation

32nd AAAI Conference on Artificial Intelligence, AAAI 2018

Volumn , Issue , 2018, Pages 2787-2794

  Cai, Han ^a   Chen, Tianyao ^a   Zhang, Weinan ^a   Yu, Yong ^a   Wang, Jun ^b  

^a SHANGHAI JIAO TONG UNIVERSITY   (China)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP NEURAL NETWORKS; NETWORK LAYERS; NEURAL NETWORKS; PROGRAM PROCESSORS; REINFORCEMENT LEARNING;

COMPETITIVE NETWORK; COMPUTATIONAL COSTS; COMPUTATIONAL RESOURCES; CONVOLUTIONAL NEURAL NETWORK; EFFICIENT ARCHITECTURE; MODERN ARCHITECTURES; NETWORK TRANSFORMATION; REINFORCEMENT LEARNING AGENT;

NETWORK ARCHITECTURE;

EID: 85055694066     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. Neural machine translation by jointly learning to align and translate. ICLR.
2. Baker, B.; Gupta, O.; Naik, N.; and Raskar, R. 2017. Designing neural network architectures using reinforcement learning. ICLR.
3. Bergstra, J., and Bengio, Y. 2012. Random search for hyperparameter optimization. JMLR.
4. Cai, H.; Ren, K.; Zhang, W.; Malialis, K.; Wang, J.; Yu, Y.; and Guo, D. 2017. Real-time bidding by reinforcement learning in display advertising. In WSDM.
5. Chen, T.; Goodfellow, I.; and Shlens, J. 2015. Net2net: Accelerating learning via knowledge transfer. ICLR.
6. Domhan, T.; Springenberg, J. T.; and Hutter, F. 2015. Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In IJCAI.
7. Gastaldi, X. 2017. Shake-shake regularization. arXiv preprint arXiv:1705.07485.
8. Goodfellow, I. J.; Warde-Farley, D.; Mirza, M.; Courville, A.; and Bengio, Y. 2013. Maxout networks. ICML.
9. Han, S.; Pool, J.; Tran, J.; and Dally, W. 2015. Learning both weights and connections for efficient neural network. In NIPS.
10. He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016a. Deep residual learning for image recognition. In CVPR.
11. He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016b. Identity mappings in deep residual networks. In ECCV.
12. Huang, G.; Liu, Z.; Weinberger, K. Q.; and van der Maaten, L. 2017. Densely connected convolutional networks. CVPR.
13. Ioffe, S., and Szegedy, C. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML.
14. Kakade, S. 2002. A natural policy gradient. NIPS.
15. Kingma, D., and Ba, J. 2015. Adam: A method for stochastic optimization. ICLR.
16. Klein, A.; Falkner, S.; Springenberg, J. T.; and Hutter, F. 2017. Learning curve prediction with bayesian neural networks. ICLR.
17. Krizhevsky, A., and Hinton, G. 2009. Learning multiple layers of features from tiny images.
18. Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS.
19. Lin, M.; Chen, Q.; and Yan, S. 2013. Network in network. arXiv preprint arXiv:1312.4400.
20. Mendoza, H.; Klein, A.; Feurer, M.; Springenberg, J. T.; and Hutter, F. 2016. Towards automatically-tuned neural networks. In Workshop on Automatic Machine Learning.
21. Miller, G. F.; Todd, P. M.; and Hegde, S. U. 1989. Designing neural networks using genetic algorithms. In ICGA. Morgan Kaufmann Publishers Inc.
22. Negrinho, R., and Gordon, G. 2017. Deeparchitect: Automatically designing and training deep architectures. arXiv preprint arXiv:1704.08792.
23. Netzer, Y.; Wang, T.; Coates, A.; Bissacco, A.; Wu, B.; and Ng, A. Y. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning.
24. Real, E.; Moore, S.; Selle, A.; Saxena, S.; Suematsu, Y. L.; Le, Q.; and Kurakin, A. 2017. Large-scale evolution of image classifiers. ICML.
25. Schulman, J.; Levine, S.; Abbeel, P.; Jordan, M. I.; and Moritz, P. 2015. Trust region policy optimization. In ICML.
26. Schuster, M., and Paliwal, K. K. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing.
27. Silver, D.; Huang, A.; Maddison, C. J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. 2016. Mastering the game of go with deep neural networks and tree search. Nature.
28. Simonyan, K., and Zisserman, A. 2015. Very deep convolutional networks for large-scale image recognition. ICLR.
29. Snoek, J.; Larochelle, H.; and Adams, R. P. 2012. Practical bayesian optimization of machine learning algorithms. In NIPS.
30. Springenberg, J. T.; Dosovitskiy, A.; Brox, T.; and Ried-miller, M. 2014. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.
31. Stanley, K. O., and Miikkulainen, R. 2002. Evolving neural networks through augmenting topologies. Evolutionary computation.
32. Sutskever, I.; Martens, J.; Dahl, G.; and Hinton, G. 2013. On the importance of initialization and momentum in deep learning. In ICML.
33. Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NIPS.
34. Sutton, R. S., and Barto, A. G. 1998. Reinforcement learning: An introduction. MIT press Cambridge.
35. Williams, R. J. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning.
36. Zagoruyko, S., and Komodakis, N. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146.
37. Zoph, B., and Le, Q. V. 2017. Neural architecture search with reinforcement learning. ICLR.