Revisiting classifier two-sample tests

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Lopez Paz, David ^a   Oquab, Maxime ^a,b  

^a FACEBOOK AI RESEARCH   (United States)

^b CNRS   (France)

Author keywords

[No Author keywords available]

Indexed keywords

STATISTICAL TESTS;

ADVERSARIAL NETWORKS; BINARY CLASSIFIERS; CLASSIFICATION ACCURACY; GENERATIVE MODEL; NOVEL APPLICATIONS; NULL DISTRIBUTION; PREDICTIVE UNCERTAINTY; TWO-SAMPLE TESTS;

CLASSIFICATION (OF INFORMATION);

EID: 85063956288     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (56)

1. Z. Bai and H. Saranadasa. Effect of high dimension: by an example of a two sample problem. Statistica Sinica, 1996.
2. Y. Bengio, L. Yao, and K. Cho. Bounding the test log-likelihood of generative models. arXiv, 2013.
3. W. Bounliphone, E. Belilovsky, M. B. Blaschko, I. Antonoglou, and A. Gretton. A test of relative similarity for model selection in generative models. arXiv, 2015.
4. G. E. P. Box. Sampling and bayes' inference in scientific modelling and robustness. Journal of the Royal Statistical Society, 1980.
5. K. P. Chwialkowski, A. Ramdas, D. Sejdinovic, and A. Gretton. Fast two-sample testing with analytic representations of probability measures. NIPS, 2015.
6. K. G. Dziugaite, D. M. Roy, and Z. Ghahramani. Training generative neural networks via Maximum Mean Discrepancy optimization. UAI, 2015.
7. W. Ehm. Binomial approximation to the poisson binomial distribution. Statistics & Probability Letters, 1991.
8. J. Friedman, T. Hastie, and R. Tibshirani. The elements of statistical learning. Springer, 2001.
9. J. H. Friedman. On multivariate goodness of fit and two sample testing. eConf, 2003.
10. K. Fukumizu, A. Gretton, Gert R. L., B. Schölkopf, and B. Sriperumbudur. Kernel choice and classifiability for rkhs embeddings of probability distributions. NIPS, 2009.
11. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. NIPS, 2014.
12. A. Gretton, O. Bousquet, A. Smola, and B. Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. In ALT, 2005.
13. A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. J. Smola. A kernel two-sample test. JMLR, 2012a.
14. A. Gretton, D. Sejdinovic, H. Strathmann, S. Balakrishnan, M. Pontil, K. Fukumizu, and B. Sriperumbudur. Optimal kernel choice for large-scale two-sample tests. NIPS, 2012b.
15. S. Gross and M. Wilber. Training and investigating residual nets, 2016. URL http://torch.ch/blog/2016/02/04/resnets.html.
16. M. U. Gutmann and A. Hyvärinen. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. JMLR, 2012.
17. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. CVPR, 2015.
18. G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, Amherst, 2007.
19. W. Jitkrittum, Z. Szabo, K. Chwialkowski, and A. Gretton. Interpretable Distribution Features with Maximum Testing Power. NIPS, 2016.
20. J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual Losses for Real-Time Style Transfer and Super-Resolution. ECCV, 2016.
21. T. Kanamori, T. Suzuki, and M. Sugiyama. f-divergence estimation and two-sample homogeneity test under semiparametric density-ratio models. arXiv, 2010.
22. D. Kingma and J. Ba. Adam: A method for stochastic optimization. ICLR, 2015.
23. A. N. Kolmogorov. Sulla determinazione empirica di una legge di distribuzione. Inst. Ital. Attuari, 1933.
24. N. H. Kuiper. Tests concerning random points on a circle. Nederl. Akad. Wetensch. Proc., 63, 1962.
25. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998.
26. E. L. Lehmann and J. P. Romano. Testing statistical hypotheses. Springer, 2006.
27. J. R. Lloyd and Z. Ghahramani. Statistical model criticism using kernel two sample tests. NIPS, 2015.
28. D. Lopez-Paz, K. Muandet, B. Schölkopf, and I. Tolstikhin. Towards a learning theory of cause-effect inference. In ICML, pp. 1452-1461, 2015.
29. H. B. Mann and D. R. Whitney. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics, 1947.
30. A. K. Menon and C. S. Ong. Linking losses for density ratio and class-probability estimation. ICML, 2016.
31. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. NIPS, 2013.
32. M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv, 2014.
33. S. Mohamed and B. Lakshminarayanan. Learning in Implicit Generative Models. arXiv, 2016.
34. J. M. Mooij, J. Peters, D. Janzing, J. Zscheischler, and B. Schölkopf. Distinguishing cause from effect using observational data: methods and benchmarks. JMLR, 2016.
35. S. Nowozin, B. Cseke, and R. Tomioka. f-GAN: Training generative neural samplers using variational divergence minimization. NIPS, 2016.
36. A. Odena, V. Dumoulin, and C. Olah. Deconvolution and checkerboard artifacts. http://distill.pub/2016/deconv-checkerboard/, 2016.
37. E. Olivetti, S. Greiner, and P. Avesani. Induction in neuroscience with classification: issues and solutions. In Machine Learning and Interpretation in Neuroimaging. 2012.
38. J. Pearl. Causality. Cambridge University Press, 2009.
39. F. Pereira, T. Mitchell, and M. Botvinick. Machine learning classifiers and fMRI: a tutorial overview. Neuroimage, 2009.
40. F. Pérez-Cruz. Estimation of information theoretic measures for continuous random variables. NIPS, 2009.
41. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. ICLR, 2016.
42. A. Ramdas, S. J. Reddi, B. Poczos, A. Singh, and L. Wasserman. Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing. arXiv, 2015.
43. A. Ramdas, A. Singh, and L. Wasserman. Classification accuracy as a proxy for two sample testing. arXiv, 2016.
44. S. J. Reddi, A. Ramdas, B. Póczos, A. Singh, and L. A. Wasserman. On the high dimensional power of a linear-time two sample test under mean-shift alternatives. AISTATS, 2015.
45. M. D. Reid and R. C. Williamson. Information, divergence and risk for binary experiments. JMLR, 2011.
46. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet large scale visual recognition challenge. IJCV, 2015.
47. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training GANs. NIPS, 2016.
48. N. V. Smirnov. On the estimation of the discrepancy between empirical curves of distribution for two independent samples. Bull. Math. Univ. Moscou, 1939.
49. A. J. Smola and B. Schölkopf. Learning with kernels. Citeseer, 1998.
50. Student. The probable error of a mean. Biometrika, 1908.
51. D. J. Sutherland, H.-Y. Tung, H. Strathmann, S. De, A. Ramdas, A. Smola, and A. Gretton. Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy. arXiv, 2016.
52. L. Theis, A. van den Oord, and M. Bethge. A note on the evaluation of generative models. ICLR, 2016.
53. V. Vapnik. Statistical learning theory. Wiley New York, 1998.
54. F. Wilcoxon. Individual comparisons by ranking methods. Biometrics bulletin, 1945.
55. M. Wornowizki and R. Fried. Two-sample homogeneity tests based on divergence measures. Computational Statistics, 2016.
56. F. Yu, A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao. LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. arXiv, 2015.