Autoaugment: Learning augmentation strategies from data

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2019-June, Issue , 2019, Pages 113-123

  Cubuk, Ekin D ^a   Zoph, Barret ^a   Mane, Dandelion ^a   Vasudevan, Vijay ^a   Le, Quoc V ^a  

^a GOOGLE INC   (United States)

Author keywords

Deep Learning

Indexed keywords

COMPUTER VISION; DEEP LEARNING;

ADDITIONAL DATUM; CURRENT DATA; DATA AUGMENTATION; IMAGE CLASSIFIERS; IMAGE PROCESSING FUNCTIONS; SEARCH ALGORITHMS; SEARCH SPACES; STATE OF THE ART;

IMAGE ENHANCEMENT;

EID: 85078722683     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2019.00020     Document Type: Conference Paper

References (72)

1. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, OSDI'16, pages 265-283, Berkeley, CA, USA, 2016. USENIX Association.
2. A. Antoniou, A. Storkey, and H. Edwards. Data augmentation generative adversarial networks. arXiv preprint arXiv: 1711.04340, 2017.
3. H. S. Baird. Document image defect models. In Structured Document Image Analysis, pages 546-556. Springer, 1992.
4. B. Baker, O. Gupta, N. Naik, and R. Raskar. Designing neural network architectures using reinforcement learning. In International Conference on Learning Representations, 2017.
5. I. Bello, B. Zoph, V. Vasudevan, and Q. V. Le. Neural optimizer search with reinforcement learning. In International Conference on Machine Learning, 2017.
6. J. Bergstra and Y. Bengio. Random search for hyperparameter optimization. Journal of Machine Learning Research, 13(Feb): 281-305, 2012.
7. A. Brock, T. Lim, J. M. Ritchie, and N. Weston. Smash: Oneshot model architecture search through hypernetworks. In International Conference on Learning Representations, 2017.
8. D. Ciregan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 3642-3649. IEEE, 2012.
9. E. D. Cubuk, B. Zoph, S. S. Schoenholz, and Q. V. Le. Intriguing properties of adversarial examples. arXiv preprint arXiv: 1711.02846, 2017.
10. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009.
11. T. DeVries and G. W. Taylor. Dataset augmentation in feature space. arXiv preprint arXiv: 1702.05538, 2017.
12. T. DeVries and G. W. Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 1708.04552, 2017.
13. T. Elsken, J.-H. Metzen, and F. Hutter. Simple and efficient architecture search for convolutional neural networks. arXiv preprint arXiv: 1711.04528, 2017.
14. Y. Em, F. Gag, Y. Lou, S. Wang, T. Huang, and L.-Y. Duan. Incorporating intra-class variance to fine-grained visual recognition. In Multimedia and Expo (ICME), 2017 IEEE International Conference on, pages 1452-1457. IEEE, 2017.
15. L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Computer vision and Image understanding, 106(1): 59-70, 2007.
16. K. Fukushima and S. Miyake. Neocognitron: A selforganizing neural network model for a mechanism of visual pattern recognition. In Competition and cooperation in neural nets, pages 267-285. Springer, 1982.
17. X. Gastaldi. Shake-shake regularization. arXiv preprint arXiv: 1705.07485, 2017.
18. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.
19. D. Han, J. Kim, and J. Kim. Deep pyramidal residual networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6307-6315. IEEE, 2017.
20. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
21. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
22. A. G. Howard. Some improvements on deep convolutional neural network based image classification. arXiv preprint arXiv: 1312.5402, 2013.
23. J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. arXiv preprint arXiv: 1709.01507, 2017.
24. H. Inoue. Data augmentation by pairing samples for images classification. arXiv preprint arXiv: 1801.02929, 2018.
25. K. Jarrett, K. Kavukcuoglu, Y. LeCun, et al. What is the best multi-stage architecture for object recognition? In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 2146-2153. IEEE, 2009.
26. S. Kornblith, J. Shlens, and Q. V. Le. Do better imagenet models transfer better? arXiv preprint arXiv: 1805.08974, 2018.
27. J. Krause, J. Deng, M. Stark, and L. Fei-Fei. Collecting a large-scale dataset of fine-grained cars. In Second Workshop on Fine-Grained Visual Categorization, 2013.
28. A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
29. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, 2012.
30. M. Kumar, G. E. Dahl, V. Vasudevan, and M. Norouzi. Parallel architecture and hyperparameter search via successive halving and classification. arXiv preprint arXiv: 1805.10255, 2018.
31. S. Laine and T. Aila. Temporal ensembling for semisupervised learning. arXiv preprint arXiv: 1610.02242, 2016.
32. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278-2324, 1998.
33. J. Lemley, S. Bazrafkan, and P. Corcoran. Smart augmentation learning an optimal data augmentation strategy. IEEE Access, 5: 5858-5869, 2017.
34. C. Liu, B. Zoph, J. Shlens, W. Hua, L.-J. Li, L. Fei-Fei, A. Yuille, J. Huang, and K. Murphy. Progressive neural architecture search. arXiv preprint arXiv: 1712.00559, 2017.
35. H. Liu, K. Simonyan, O. Vinyals, C. Fernando, and K. Kavukcuoglu. Hierarchical representations for efficient architecture search. In International Conference on Learning Representations, 2018.
36. I. Loshchilov and F. Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv: 1608.03983, 2016.
37. D. Mahajan, R. Girshick, V. Ramanathan, K. He, M. Paluri, Y. Li, A. Bharambe, and L. Van Der Maaten. Exploring the limits of weakly supervised pretraining. arXiv preprint arXiv: 1805.00932, 2018.
38. S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv: 1306.5151, 2013.
39. H. Mania, A. Guy, and B. Recht. Simple random search provides a competitive approach to reinforcement learning. arXiv preprint arXiv: 1803.07055, 2018.
40. T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii. Virtual adversarial training: A regularization method for supervised and semi-supervised learning. In International Conference on Learning Representations, 2016.
41. S. Mun, S. Park, D. K. Han, and H. Ko. Generative adversarial network based acoustic scene training set augmentation and selection using SVM hyper-plane. In Detection and Classification of Acoustic Scenes and Events Workshop, 2017.
42. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.
43. M.-E. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In Computer Vision, Graphics & Image Processing, 2008. ICVGIP'08. Sixth Indian Conference on, pages 722-729. IEEE, 2008.
44. A. Oliver, A. Odena, C. Raffel, E. D. Cubuk, and I. J. Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. arXiv preprint arXiv: 1804.09170, 2018.
45. L. Perez and J. Wang. The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv: 1712.04621, 2017.
46. H. Pham, M. Y. Guan, B. Zoph, Q. V. Le, and J. Dean. Efficient neural architecture search via parameter sharing. In International Conference on Machine Learning, 2018.
47. A. J. Ratner, H. Ehrenberg, Z. Hussain, J. Dunnmon, and C. Ré. Learning to compose domain-specific transformations for data augmentation. In Advances in Neural Information Processing Systems, pages 3239-3249, 2017.
48. E. Real, A. Aggarwal, Y. Huang, and Q. V. Le. Regularized evolution for image classifier architecture search. arXiv preprint arXiv: 1802.01548, 2018.
49. E. Real, S. Moore, A. Selle, S. Saxena, Y. L. Suematsu, J. Tan, Q. Le, and A. Kurakin. Large-scale evolution of image classifiers. In International Conference on Machine Learning, 2017.
50. B. Recht, R. Roelofs, L. Schmidt, and V. Shankar. Do cifar-10 classifiers generalize to cifar-10? arXiv preprint arXiv: 1806.00451, 2018.
51. M. Sajjadi, M. Javanmardi, and T. Tasdizen. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In Advances in Neural Information Processing Systems, pages 1163-1171, 2016.
52. I. Sato, H. Nishimura, and K. Yokoi. Apac: Augmented pattern classification with neural networks. arXiv preprint arXiv: 1505.03229, 2015.
53. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.
54. P. Y. Simard, D. Steinkraus, J. C. Platt, et al. Best practices for convolutional neural networks applied to visual document analysis. In Proceedings of International Conference on Document Analysis and Recognition, 2003.
55. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. Advances in Neural Information Processing Systems, 2015.
56. L. Sixt, B. Wild, and T. Landgraf. Rendergan: Generating realistic labeled data. arXiv preprint arXiv: 1611.01331, 2016.
57. I. Sutskever, J. Schulman, T. Salimans, and D. Kingma. Requests For Research 2.0.https://blog.openai.com/requests-for-research-2, 2018.
58. C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, 2017.
59. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, et al. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
60. A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semisupervised deep learning results. In Advances in neural information processing systems, pages 1195-1204, 2017.
61. T. Tran, T. Pham, G. Carneiro, L. Palmer, and I. Reid. A Bayesian data augmentation approach for learning deep models. In Advances in Neural Information Processing Systems, pages 2794-2803, 2017.
62. L. Wan, M. Zeiler, S. Zhang, Y. Le Cun, and R. Fergus. Regularization of neural networks using dropconnect. In International Conference on Machine Learning, pages 1058-1066, 2013.
63. L. Xie and A. Yuille. Genetic CNN. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017.
64. S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He. Aggregated residual transformations for deep neural networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5987-5995, 2017.
65. Y. Yamada, M. Iwamura, and K. Kise. Shakedrop regularization. arXiv preprint arXiv: 1802.02375, 2018.
66. F. Yu, D. Wang, and T. Darrell. Deep layer aggregation. arXiv preprint arXiv: 1707.06484, 2017.
67. S. Zagoruyko and N. Komodakis. Wide residual networks. In British Machine Vision Conference, 2016.
68. H. Zhang, M. Cisse, Y. N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 1710.09412, 2017.
69. Z. Zhong, L. Zheng, G. Kang, S. Li, and Y. Yang. Random erasing data augmentation. arXiv preprint arXiv: 1708.04896, 2017.
70. X. Zhu, Y. Liu, Z. Qin, and J. Li. Data augmentation in emotion classification using generative adversarial networks. arXiv preprint arXiv: 1711.00648, 2017.
71. B. Zoph and Q. V. Le. Neural architecture search with reinforcement learning. In International Conference on Learning Representations, 2017.
72. B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le. Learning transferable architectures for scalable image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017.