Generative models and model criticism via optimized maximum mean discrepancy

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Sutherland, Dougal J ^a,b   Tung, Hsiao Yu ^b   Strathmann, Heiko ^a   De, Soumyajit ^a   Ramdas, Aaditya ^c   Smola, Alex ^b   Gretton, Arthur ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b Carnegie Mellon University ^*  (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

STATISTICAL TESTS;

ADVERSARIAL NETWORKS; DATA DISTRIBUTION; DATA SAMPLE; DISTINGUISHABILITY; GENERATIVE MODEL; INDIVIDUAL MODELING; REFERENCE DATA;

PROBABILITY DISTRIBUTIONS;

EID: 85088227458     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (34)

1. V. Alba Fernández, M. Jiménez-Gamero, and J. Muñoz Garcia. A test for the two-sample problem based on empirical characteristic functions. Computational Statistics & Data Analysis, 52:3730-3748, 2008.
2. Wacha Bounliphone, Eugene Belilovsky, Matthew B. Blaschko, Ioannis Antonoglou, and Arthur Gretton. A test of relative similarity for model selection in generative models. In International Conference on Learning Representations, 2016. arXiv:1511.04581.
3. Olivier Breuleux, Yoshua Bengio, and Pascal Vincent. Quickly generating representative samples from an rbm-derived process. Neural Computation, 23(8):2053-2073, 2011.
4. Emily Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus. Deep generative image models using a laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems, pp. 1486-1494. 2015.
5. Gintare Karolina Dziugaite, Daniel M. Roy, and Zoubin Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. In Uncertainty in Artificial Intelligence, 2015. arXiv:1505.03906.
6. Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, and Bernhard Schölkopf. Kernel measures of conditional dependence. In Advances in Neural Information Processing Systems, 2008.
7. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014. arXiv:1406.2661.
8. Arthur Gretton, Kenji Fukumizu, Zaid Harchaoui, and Bharath K. Sriperumbudur. A fast, consistent kernel two-sample test. In Advances in Neural Information Processing Systems, pp. 673-681, 2009.
9. Arthur Gretton, Karsten M. Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alex J. Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13, 2012a.
10. Arthur Gretton, Bharath Sriperumbudur, Dino Sejdinovic, Heiko Strathmann, and Massimiliano Pontil. Optimal kernel choice for large-scale two-sample tests. In Advances in Neural Information Processing Systems, 2012b.
11. Gaël Guennebaud, Benoît Jacob, et al. Eigen v3. http://eigen.tuxfamily.org, 2010.
12. Michael U. Gutmann, Ritabrata Dutta, Samuel Kaski, and Jukka Corander. Statistical inference of intractable generative models via classification. 2014. arXiv:1407.4981.
13. Ferenc Huszar. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? 2015. arXiv:1511.05101.
14. MKL Intel. Intel math kernel library, 2003-17.
15. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. arXiv:1412.6980.
16. Yujia Li, Kevin Swersky, and Richard Zemel. Generative moment matching networks. In Uncertainty in Artificial Intelligence, 2015. arXiv:1502.02761.
17. James Robert Lloyd and Zoubin Ghahramani. Statistical model criticism using kernel two sample tests. In Advances in Neural Information Processing Systems, pp. 829-837, 2015.
18. Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in Applied Probability, 29(2):429-443, 1997.
19. XuanLong Nguyen, Martin J. Wainwright, and Michael I. Jordan. On optimal quantization rules in some problems in sequential decentralized detection. IEEE Transactions on Information Theory, 54(7):3285-3295, 2008.
20. Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems, 2016.
21. Aaaditya Ramdas, Sashank Reddi, Barnabás Póczos, Aarti Singh, and Larry Wasserman. Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing, 2015a. arXiv:1508.00655.
22. Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, and Larry Wasserman. On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions. In AAAI Conference on Artificial Intelligence, 2015b. arXiv:1406.2083.
23. Aaditya Ramdas, Aarti Singh, and Larry Wasserman. Classification accuracy as a proxy for two sample testing, 2016. arXiv:1602.02210.
24. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In Advances in Neural Information Processing Systems, 2016. arXiv:1606.03498.
25. Robert J. Serfling. Approximation Theorems of Mathematical Statistics. John Wiley & Sons, 1980.
26. Soeren Sonnenburg, Heiko Strathmann, Sergey Lisitsyn, Viktor Gal, Fernando J. Iglesias García, Wu Lin, Chiyuan Zhang, Soumyajit De, frx, tklein23, Evgeniy Andreev, JonasBehr, sploving, Parijat Mazumdar, Christian Widmer, Abhijeet Kislay, Kevin Hughes, Roman Votyakov, khalednasr, Saurabh Mahindre, Alesis Novik, Abinash Panda, Evangelos Anagnostopoulos, Liang Pang, serialhex, Alex Binder, Sanuj Sharma, Michal Uricár, Björn Esser, and Daniel Pyrathon. shogun: Shogun 4.1.0 - tajinohi no agatamori, May 2016.
27. Bharath K. Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Gert R. G. Lanckriet, and Bernhard Schölkopf. Kernel choice and classifiability for RKHS embeddings of probability distributions. In Advances in Neural Information Processing Systems, 2009.
28. Bharath K. Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Gert R. G. Lanckriet, and Bernhard Schölkopf. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11:1517-1561, 2010.
29. Bharath K. Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert R. G. Lanckriet. On the empirical estimation of integral probability metrics. Electronic Journal of Statistics, 6:1550-1599, 2012.
30. Heiko Strathmann. Adaptive Large-Scale Kernel Two-Sample Testing. M.Sc. thesis, University College London, 2012.
31. Masashi Sugiyama, Taiji Suzuki, Yuta Itoh, Takafumi Kanamori, and Manabu Kimura. Least-squares two-sample test. Neural Networks, 24(7):735-751, sep 2011.
32. Dougal J. Sutherland. Unbiased estimators for the variance of MMD estimators, 2019. arXiv:1906.02104.
33. Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. In International Conference on Learning Representations, 2016. arXiv:1511.01844.
34. Larry Wasserman. All of Nonparametric Statistics. Springer, 2006.