Gradient descent GAN optimization is locally stable

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 5586-5596

  Nagarajan, Vaishnavh ^a   Kolter, J Zico ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADVERSARIAL NETWORKS; ASYMPTOTICALLY STABLE; EQUILIBRIUM POINT; GRADIENT DESCENT; LOCAL STABILITY; OPTIMIZATION PROCEDURES; REGULARIZATION TERMS; STABILITY ANALYSIS;

OPTIMIZATION;

EID: 85047004888     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In International Conference on Learning Representations (ICLR), 2017.
2. Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, Volume 70 of Proceedings of Machine Learning Research, pages 214-223, 2017.
3. Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and equilibrium in generative adversarial nets (GANs). In Proceedings of the 34th International Conference on Machine Learning, Volume 70 of Proceedings of Machine Learning Research, pages 224-232, 2017.
4. Vivek S Borkar and Sean P Meyn. The ode method for convergence of stochastic approximation and reinforcement learning. SIAM Journal on Control and Optimization, 38(2): 447-469, 2000.
5. Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. Mode regularized generative adversarial networks. In Fifth International Conference on Learning Representations (ICLR). 2017.
6. Emily L Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus. In Advances in Neural Information Processing Systems 28, pages 1486-1494. 2015.
7. Harris Drucker and Yann Le Cun. Improving generalization performance using double backpropagation. IEEE Transactions on Neural Networks, 3(6): 991-997, 1992.
8. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems 27, pages 2672-2680. 2014.
9. Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein GANs. In Thirty-first Annual Conference on Neural Information Processing Systems (NIPS). 2017.
10. Daniel Jiwoong Im, Chris Dongjoo Kim, Hui Jiang, and Roland Memisevic. Generating images with recurrent adversarial networks. arXiv preprint arXiv: 1602.05110, 2016.
11. Hassan K Khalil. Non-linear Systems. Prentice-Hall, New Jersey, 1996.
12. Harold Kushner and George Yin. Stochastic Approximation and Recursive Algorithms and Applications, Volume 35 of Stochastic Modelling and Applied Probability. Springer-Verlag New York, The address, 2003.
13. Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo-realistic single image super-resolution using a generative adversarial network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
14. Jan R Magnus, Heinz Neudecker, et al. Matrix differential calculus with applications in statistics and econometrics. 1995.
15. Michael Mathieu, Camille Couprie, and Yann LeCun. Deep multi-scale video prediction beyond mean square error. In Fourth International Conference on Learning Representations (ICLR). 2016.
16. L. Mescheder, S. Nowozin, and A. Geiger. The numerics of GANs. In Thirty-first Annual Conference on Neural Information Processing Systems (NIPS). 2017.
17. Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In Fifth International Conference on Learning Representations (ICLR). 2017.
18. Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative networks: Conditional iterative generation of images in latent space. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
19. Ben Poole, Alexander A Alemi, Jascha Sohl-Dickstein, and Anelia Angelova. Improved generator objectives for GANs. arXiv preprint arXiv: 1612.02780, 2016.
20. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In Fourth International Conference on Learning Representations (ICLR). 2016.
21. K. Roth, A. Lucchi, S. Nowozin, and T. Hofmann. Stabilizing training of generative adversarial networks through regularization. In Thirty-first Annual Conference on Neural Information Processing Systems (NIPS). 2017.
22. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In Advances in Neural Information Processing Systems 29, pages 2234-2242. 2016.
23. Jiajun Wu, Chengkai Zhang, Tianfan Xue, Bill Freeman, and Josh Tenenbaum. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In Advances in Neural Information Processing Systems 29, pages 82-90. 2016.