f-GAN: Training generative neural samplers using variational divergence minimization

Advances in Neural Information Processing Systems

Volumn , Issue , 2016, Pages 271-279

  Nowozin, Sebastian ^a   Cseke, Botond ^a   Tomioka, Ryota ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PROBABILITY DISTRIBUTIONS;

DIVERGENCE ESTIMATION; EFFICIENT COMPUTATION; GENERATIVE MODEL; MARGINALIZATION; NETWORK WEIGHTS; PROBABILISTIC MODELS; TRAINING COMPLEXITY; TRAINING METHODS;

FEEDFORWARD NEURAL NETWORKS;

EID: 85018914753     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (35)

1. S. M. Ali and S. D. Silvey. A general class of coefficients of divergence of one distribution from another. JRSS (B), pages 131-142, 1966.
2. C. M. Bishop. Mixture density networks. Technical report, Aston University, 1994.
3. C. M. Bishop, M. Svensén, and C. K. I. Williams. GTM: The generative topographic mapping. Neural Computation, 10(1):215-234, 1998.
4. D. A. Clevert, T. Unterthiner, and S. Hochreiter. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv:1511.07289, 2015.
5. I. Csiszár and P. C. Shields. Information theory and statistics: A tutorial. Foundations and Trends in Communications and Information Theory, 1:417-528, 2004.
6. Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In NIPS, pages 2933-2941, 2014.
7. G. K. Dziugaite, D. M. Roy, and Z. Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. In UAI, pages 258-267, 2015.
8. J. Gauthier. Conditional generative adversarial nets for convolutional face generation. Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester 2014, 2014.
9. T. Gneiting and A. E. Raftery. Strictly proper scoring rules, prediction, and estimation. JASA, 102(477): 359-378, 2007.
10. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, pages 2672-2680, 2014.
11. I. J. Goodfellow. On distinguishability criteria for estimating generative models. In International Conference on Learning Representations (ICLR2015), 2015. arXiv:1412.6515.
12. A. Graves. Generating sequences with recurrent neural networks. arXiv:1308.0850, 2013.
13. A. Gretton, K. Fukumizu, C. H. Teo, L. Song, B. Schölkopf, and A. J. Smola. A kernel statistical test of independence. In NIPS, pages 585-592, 2007.
14. M. Gutmann and A. Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In AISTATS, pages 297-304, 2010.
15. J. B. Hiriart-Urruty and C. Lemaréchal. Fundamentals of convex analysis. Springer, 2012.
16. F. Huszár. How (not) to train your generative model: scheduled sampling, likelihood, adversary? arXiv:1511.05101, 2015.
17. D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014.
18. D. P. Kingma and M. Welling. Auto-encoding variational Bayes. arXiv:1402.0030, 2013.
19. H. Larochelle and I. Murray. The neural autoregressive distribution estimator. In AISTATS, 2011.
20. Y. Li, K. Swersky, and R. Zemel. Generative moment matching networks. In ICML, 2015.
21. F. Liese and I. Vajda. On divergences and informations in statistics and information theory. Information Theory, IEEE, 52(10):4394-4412, 2006.
22. D. J. C. MacKay. Bayesian neural networks and density networks. Nucl. Instrum. Meth. A, 354(1):73-80, 1995.
23. A. Makhzani, J. Shlens, N. Jaitly, and I. Goodfellow. Adversarial autoencoders. arXiv:1511.05644, 2015.
24. T. Minka. Divergence measures and message passing. Technical report, Microsoft Research, 2005.
25. X. Nguyen, M. J. Wainwright, and M. I. Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. Information Theory, IEEE, 56(11):5847-5861, 2010.
26. F. Nielsen and R. Nock. On the chi-square and higher-order chi distances for approximating f-divergences. Signal Processing Letters, IEEE, 21(1):10-13, 2014.
27. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434, 2015.
28. M. D. Reid and R. C. Williamson. Information, divergence and risk for binary experiments. Journal of Machine Learning Research, 12(Mar):731-817, 2011.
29. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In ICML, pages 1278-1286, 2014.
30. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training GANs. In NIPS, 2016.
31. J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using non-equilibrium thermodynamics. ICML, pages 2256-2265, 2015.
32. B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. Lanckriet. Hilbert space embeddings and metrics on probability measures. JMLR, 11:1517-1561, 2010.
33. L. Theis, A. v.d. Oord, and M. Bethge. A note on the evaluation of generative models. arXiv:1511.01844, 2015.
34. B. Uria, I. Murray, and H. Larochelle. RNADE: The real-valued neural autoregressive density-estimator. In NIPS, pages 2175-2183, 2013.
35. F. Yu, Y. Zhang, S. Song, A. Seff, and J. Xiao. LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv:1506.03365, 2015.