Which training methods for GANs do actually converge?

35th International Conference on Machine Learning, ICML 2018

Volumn 8, Issue , 2018, Pages 5589-5626

  Mescheder, Lars ^a   Geiger, Andreas ^a,b   Nowozin, Sebastian ^c  

^a University in Tübingen   (Germany)

^b ETH ZURICH   (Switzerland)

^c MICROSOFT RESEARCH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

ABSOLUTE CONTINUITY; CONVERGENCE RESULTS; DATA DISTRIBUTION; EQUILIBRIUM POINT; LOCAL CONVERGENCE; LOWER DIMENSIONAL MANIFOLDS; STABILITY PROBLEM; TRAINING METHODS;

LEARNING SYSTEMS;

EID: 85057290214     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kud- lur, M., Levenberg, J., Monga, R., Moore, S., Murray, D. G., Steiner, B., Tucker, P. A., Vasudevan, V., Warden, P., Wicke, M., Yu, Y and Zheng, X. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implemen-tation, OSDI2016, Savannah, GA, USA, November 2-4, 2016., pp. 265-283, 2016.
2. Aijovsky, M. and Bottou, L. Towards principled methods for training generative adversarial networks. CoRR, abs/1701.04862, 2017.
3. Arjovsky, M., Chintala, S., and Bottou, L. Wasserstein GAN. CoRR, abs/1701.07875, 2017.
4. Barratt, S. and Sharma, R. A note on the inception score. arXivpreprint arXiv:1801.01973, 2018.
5. Berthelot, D., Schumm, T., and Metz, L. BEGAN: Boundary equilibrium generative adversarial networks. CoRR, abs/1703.10717, 2017.
6. Bertsekas, D. P. Nonlinear programming. Athena scientific Belmont, 1999.
7. Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. C., and Bengio, Y. Generative adversarial nets. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pp. 2672-2680, 2014.
8. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. C. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 5769-5779, 2017.
9. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
10. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 6629-6640, 2017.
11. Karras, T., Aila, T., Laine, S., and Lehtinen, J. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
12. Khalil, H. K. Nonlinear systems. Prentice-Hall, New Jersey, 2(5):5-10, 1996.
13. Kodali, N., Abernethy, J. D., Hays, J., and Kira, Z. How to train your DRAGAN. CoRR, abs/1705.07215, 2017.
14. Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. 2009.
15. Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
16. Mescheder, L. M., Nowozin, S., and Geiger, A. The numerics of gans. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 1823-1833, 2017.
17. Nagarajan, V. and Kolter, J. Z. Gradient descent GAN optimization is locally stable. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 5591-5600, 2017.
18. Nowozin, S., Cseke, B., and Tomioka, R. f-gan: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 271-279, 2016.
19. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. Automatic differentiation in pytorch. 2017.
20. Radford, A., Metz, L., and Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
21. Roth, K., Lucchi, A., Nowozin, S., and Hofmann, T. Stabilizing training of generative adversarial networks through regularization. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pp. 2015-2025, 2017.
22. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252, 2015. doi:10.1007/sll263-015-0816-y.
23. Salimans, T., Goodfellow, I. J., Zaremba, W., Cheung, V., Radford, A., and Chen, X. Improved techniques fpr training gans. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2226-2234, 2016.
24. Sonderby, C. K., Caballero, J., Theis, L., Shi, W., and Huszar, F. Amortised MAP inference for image super- resolution. CoRR, abs/1610.04490, 2016.
25. Tieleman, T. and Hinton, G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude, 2012.
26. Yu, F., Zhang, Y., Song, S., Seff, A., and Xiao, J. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
27. Zhao, J. J., Mathieu, M., and LeCun, Y. Energy-based generative adversarial network. CoRR, abs/1609.03126, 2016.