Generative moment matching networks

32nd International Conference on Machine Learning, ICML 2015

Volumn 3, Issue , 2015, Pages 1718-1727

  Li, Yujia ^a   Swersky, Kevin ^a   Zemel, Richard ^a,b  

^a UNIVERSITY OF TORONTO   (Canada)

^b CANADIAN INSTITUTE FOR ADVANCED RESEARCH   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; STATISTICAL TESTS; TESTING;

ADVERSARIAL NETWORKS; AUTO ENCODERS; FACE DATABASE; GENERATIVE MODEL; INDEPENDENT SAMPLES; MINIMAX PROGRAMS; MOMENT-MATCHING; STATISTICAL HYPOTHESIS TESTING;

LEARNING SYSTEMS;

EID: 84970016114     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (47)

1. Ackley, David H, Hinton, Geoffrey E, and Sejnowski, Terrence J. A learning algorithm for boltzmann machines. Cognitive science, 9(1): 147-169, 1985.
2. Bengio, Yoshua, Lamblin, Pascal, Popovici, Dan, Larochelle, Hugo, et al. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems (NIPS), 2007.
3. Bengio, Yoshua, Mesnil, Grégoire, Dauphin, Yann, and Rifai, Salah. Better mixing via deep representations. In Proceedings of the 28th International Conference on Machine Learning (ICML), 2013a.
4. Bengio, Yoshua, Yao, Li, Alain, Guillaume, and Vincent, Pascal. Generalized denoising auto-encoders as generative models. In Advances in Neural Information Processing Systems, pp. 899-907, 2013b.
5. Bengio, Yoshua, Thibodeau-Laufer, Eric, Alain, Guillaume, and Yosinski, Jason. Deep generative stochastic networks trainable by backprop. In Proceedings of the 29th International Conference on Machine Learning (ICML), 2014.
6. Cho, Kyunghyun, van Merrienboer, Bart, Gulcehre, Caglar, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua. Learning phrase representations using rnn encoderdecoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014.
7. Dziugaite, Gintare Karolina, Roy, Daniel M., and Ghahramani, Zoubin. Training generative neural networks via maximum mean discrepancy optimization. In Uncertainty in Artificial Intelligence (UAI), 2015.
8. Fang, Hao, Gupta, Saurabh, Iandola, Forrest, Srivastava, Rupesh, Deng, Li, Dollár, Piotr, Gao, Jianfeng, He, Xiaodong, Mitchell, Margaret, Piatt, John, Zitnick, C. Lawrence, and Zweig, Geoffrey. From captions to visual concepts and back. arXiv preprint arXiv:1411.4952, 2014.
9. Goodfellow, Ian, Pouget-Abadie, Jean, Mirza, Mehdi, Xu, Bing, Warde-Farley, David, Ozair, Sherjil, Courville, Aaron, and Bengio, Yoshua. Generative adversarial nets. In Advances in Neural Information Processing Systems, pp. 2672-2680, 2014.
10. Graves, Alex and Jaitly, Navdeep. Towards end-to-end speech recognition with recurrent neural networks. In Proceedings of the 31 st International Conference on Machine Learning (ICML-14), pp. 1764-1772, 2014.
11. Gretton, Arthur, Borgwardt, Karsten M, Rasch, Malte, Schölkopf, Bernhard, and Smola, Alex J. A kernel method for the two-sample-problem. In Advances in Neural Information Processing Systems (NIPS), 2007.
12. Gretton, Arthur, Borgwardt, Karsten M, Rasch, Malte J, Schölkopf, Bernhard, and Smola, Alexander. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773, 2012a.
13. Gretton, Arthur, Sejdinovic, Dino, Strathmann, Heiko, Balakrishnan, Sivaraman, Pontil, Massimiliano, Fukumizu, Kenji, and Sriperumbudur, Bharath K. Optimal kernel choice for large-scale two-sample tests. In Advances in Neural Information Processing Systems, pp. 1205-1213, 2012b.
14. Hinton, Geoffrey E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14 (8):1771-1800, 2002.
15. Hinton, Geoffrey E, Dayan, Peter, Frey, Brendan J, and Neal, Radford M. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):1158-1161, 1995.
16. Hinton, Geoffrey E., Deng, Li, Yu, Dong, Dahl, George E., Mohamed, Abdel-Rahman, Jaitly, Navdeep, Senior, Andrew, Vanhoucke, Vincent, Nguyen, Patrick, Sainath, Tara N., and Kingsbury, Brian. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process. Mag., 29(6):82-97, 2012a.
17. Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012b.
18. Kingma, Diederik P. and Welling, Max. Auto-encoding variational Bayes. In International Conference on Learning Representations, 2014.
19. Kiros, Ryan, Salakhutdinov, Ruslan, and Zemel, Richard S. Unifying visual-semantic embeddings with multimodal neural language models. arXiv preprint arXiv: 1411.2539, 2014.
20. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS), 2012.
21. Larochelle, Hugo and Murray, Iain. The neural autoregressive distribution estimator. In roceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), 2011.
22. LeCun, Yann, Bottou, Leon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
23. MacKay, David JC. Bayesian neural networks and density networks. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 354(1):73-80, 1995.
24. Magdon-Ismail, Malik and Atiya, Amir. Neural networks for density estimation. In NIPS, pp. 522-528, 1998.
25. Makhzani, Alireza and Frey, Brendan. A winner-take-all method for training sparse convolutional autoencoders. In NIPS Deep Learning Workshop, 2014.
26. Masci, Jonathan, Meier, Ueli, Cireşan, Dan, and Schmidhuber, Jürgen. Stacked convolutional auto-encoders for hierarchical feature extraction. In Artificial Neural Networks and Machine Learning-ICANN 2011, pp. 52-59. Springer, 2011.
27. Mnih, Andriy and Gregor, Karol. Neural variational inference and learning in belief networks. In International Conference on Machine Learning, 2014.
28. Nair, Vinod and Hinton, Geoffrey E. Rectified linear units improve restricted boltzmann machines. In International Conference on Machine Learning, pp. 807-814, 2010.
29. Neal, Radford M. Connectionist learning of belief networks. Artificial intelligence, 56(1):71-113, 1992.
30. Rahimi, Ali and Recht, Benjamin. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems (NIPS), 2007.
31. Ramdas, Aaditya, Reddi, Sashank J, Poczos, Barnabas, Singh, Aarti, and Wasserman, Larry. On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions. In The Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15), 2015.
32. Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning, pp. 1278-1286, 2014.
33. Rifai, Salah, Vincent, Pascal, Muller, Xavier, Glorot, Xavier, and Bengio, Yoshua. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 833-840, 2011.
34. Rifai, Salah, Bengio, Yoshua, Dauphin, Yann, and Vincent, Pascal. A generative process for sampling contractive auto-encoders. In International Conference on Machine Learning (ICML), 2012.
35. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltzmann machines. In International Conference on Artificial Intelligence and Statistics, 2009.
36. Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michaël, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations, 2014.
37. Snoek, Jasper, Larochelle, Hugo, and Adams, Ryan P. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, 2012.
38. Snoek, Jasper, Swersky, Kevin, Zemel, Richard S., and Adams, Ryan P. Input warping for Bayesian optimization of non-stationary functions. In International Conference on Machine Learning, 2014.
39. Sriperumbudur, Bharath K, Fukumizu, Kenji, Gretton, Arthur, Lanckriet, Gert RG, and Schölkopf, Bernhard. Kernel choice and classifiability for rkhs embeddings of probability distributions. In Advances in Neural Information Processing Systems, pp. 1750-1758, 2009.
40. Susskind, Joshua, Anderson, Adam, and Hinton, Geoffrey E. The toronto face dataset. Technical report, Department of Computer Science, University of Toronto, 2010.
41. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc VV. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pp. 3104-3112, 2014.
42. Szegedy, Christian, Liu, Wei, Jia, Yangqing, Sermanet, Pierre, Reed, Scott, Anguelov, Dragomir, Erhan, Du-mitru, Vanhoucke, Vincent, and Rabinovich, Andrew. Going deeper with convolutions. arXiv preprint arXiv: 1409.4842, 2014.
43. Vincent, Pascal, Larochelle, Hugo, Bengio, Yoshua, and Manzagol, Pierre-Antoine. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096-1103. ACM, 2008.
44. Vinyals, Oriol, Toshev, Alexander, Bengio, Samy, and Er-han, Dumitru. Show and tell: A neural image caption generator. arXiv preprint arXiv:1411.4555, 2014.
45. Zeiler, Matthew D, Krishnan, Dilip, Taylor, Graham W, and Fergus, Robert. Deconvolutional networks. In Computer Vision and Pattern Recognition, pp. 2528-2535. IEEE, 2010.
46. Zemel, Richard, Wu, Yu, Swersky, Kevin, Pitassi, Toni, and Dwork, Cynthia. Learning fair representations. In International Conference on Machine Learning, pp. 325-333, 2013.
47. Zhao, Ji and Meng, Deyu. Fastmmd: Ensemble of circular discrepancy for efficient two-sample test. arXiv preprint arXiv: 1405.2664, 2014.