Stabilizing training of generative adversarial networks through regularization

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 2019-2029

  Roth, Kevin ^a   Lucchi, Aurelien ^a   Nowozin, Sebastian ^b   Hofmann, Thomas ^a  

^a ETH ZURICH   (Switzerland)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING;

ADVERSARIAL NETWORKS; COMPUTATIONAL COSTS; DATA DISTRIBUTION; FUNDAMENTAL LIMITATIONS; IMAGE GENERATIONS; MODELING DISTRIBUTIONS; REGULARIZATION APPROACH; TRAINING PROCEDURES;

NETWORK ARCHITECTURE;

EID: 85047000798     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry. American Mathematical Soc., 2007.
2. Guozhong An. The effects of adding noise during backpropagation training on a generalization performance. Neural Comput., pages 643-674, 1996.
3. Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. In ICLR, 2017.
4. Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. Proceedings of Machine Learning Research. PMLR, 2017.
5. Chris M Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7:108-116, 1995.
6. Diane Bouchacourt, Pawan K Mudigonda, and Sebastian Nowozin. Disco nets: Dissimilarity coefficients networks. In Advances in Neural Information Processing Systems, pages 352-360, 2016.
7. Tong Che, Yanran Li, Athul Paul Jacob, Yoshua Bengio, and Wenjie Li. Mode regularized generative adversarial networks. arXiv preprint arXiv:1612.02136, 2016.
8. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversarial Networks. In Advances in Neural Information Processing Systems, pages 2672-2680, 2014.
9. Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13:723-773, 2012.
10. Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems, 2017.
11. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 770-778, 2016.
12. Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of Machine Learning Research, pages 448-456. PMLR, 2015.
13. Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. The International Conference on Learning Representations (ICLR), 2014.
14. Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes. The International Conference on Learning Representations (ICLR), 2013.
15. Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
16. Yujia Li, Kevin Swersky, and Richard S Zemel. Generative moment matching networks. In ICML, pages 1718-1727, 2015.
17. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision, pages 3730-3738, 2015.
18. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
19. Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. The numerics of gans. In Advances in Neural Information Processing Systems, 2017.
20. Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In International Conference on Learning Representations (ICLR), 2016.
21. Tom Minka. Divergence measures and message passing. Technical report, Microsoft Research, 2005.
22. Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in Applied Probability, 29:429-443, 1997.
23. Hariharan Narayanan and Sanjoy Mitter. Sample complexity of testing the manifold hypothesis. In Advances in Neural Information Processing Systems, pages 1786-1794, 2010.
24. XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847-5861, 2010.
25. Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems, pages 271-279, 2016.
26. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.
27. Mark D Reid and Robert C Williamson. Information, divergence and risk for binary experiments. Journal of Machine Learning Research, 12:731-817, 2011.
28. Danilo J Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning, 2014.
29. David W Scott. Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, 2015.
30. Casper Kaae Sønderby, Jose Caballero, Lucas Theis, Wenzhe Shi, and Ferenc Huszár. Amortised map inference for image super-resolution. arXiv preprint arXiv:1610.04490, 2016.
31. Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert RG Lanckriet. On integral probability metrics, phi-divergences and binary classification. arXiv preprint arXiv:0901.2698, 2009.
32. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.