Generalized denoising auto-encoders as generative models

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Bengio, Yoshua ^a   Yao, Li ^a   Alain, Guillaume ^a   Vincent, Pascal ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS;

DIFFERENT ATTACKS; GENERATIVE MODEL; LOG LIKELIHOOD; MATHEMATICAL JUSTIFICATION; METROPOLIS-HASTINGS; RECONSTRUCTION ERROR; SQUARED ERRORS; TRAINING PROCEDURES;

CRIME;

EID: 84899017362     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Alain, G. and Bengio, Y. (2013). What regularized auto-encoders learn from the data generating distribution. In International Conference on Learning Representations (ICLR'2013).
2. Bengio, Y. and Yao, L. (2013). Bounding the test log-likelihood of generative models. Technical report, U. Montreal, arXiv.
3. Bengio, Y., Larochelle, H., and Vincent, P. (2006a). Non-local manifold Parzen windows. In NIPS'05, pages 115-122. MIT Press.
4. Bengio, Y., Monperrus, M., and Larochelle, H. (2006b). Nonlocal estimation of manifold structure. Neural Computation, 18(10).
5. Bengio, Y., Thibodeau-Laufer, E., and Yosinski, J. (2013a). Deep generative stochastic networks trainable by backprop. Technical Report arXiv:1306.1091, Universite de Montreal.
6. Bengio, Y., Courville, A., and Vincent, P. (2013b). Unsupervised feature learning and deep learning: A review and new perspectives. IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI).
7. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy).
8. Cho, K., Raiko, T., and Ilin, A. (2013). Enhanced gradient for training restricted boltzmann machines. Neural computation, 25(3), 805-831.
9. Heckerman, D., Chickering, D. M., Meek, C., Rounthwaite, R., and Kadie, C. (2000). Dependency networks for inference, collaborative filtering, and data visualization. Journal of Machine Learning Research, 1, 49-75.
10. Hinton, G. E. (1999). Products of experts. In ICANN'1999.
11. Hinton, G. E., Osindero, S., and Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527-1554.
12. Hyvärinen, A. (2005). Estimation of non-normalized statistical models using score matching. Journal of Machine Learning Research, 6, 695-709.
13. Kingma, D. and LeCun, Y. (2010). Regularized estimation of image statistics by score matching. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 1126-1134.
14. Ranzato, M., Boureau, Y.-L., and LeCun, Y. (2008). Sparse feature learning for deep belief networks. In NIPS'07, pages 1185-1192, Cambridge, MA. MIT Press.
15. Rifai, S., Vincent, P., Muller, X., Glorot, X., and Bengio, Y. (2011). Contractive auto-encoders: Explicit invariance during feature extraction. In ICML'2011.
16. Rifai, S., Bengio, Y., Dauphin, Y., and Vincent, P. (2012). A generative process for sampling contractive auto-encoders. In ICML'2012.
17. Swersky, K., Ranzato, M., Buchman, D., Marlin, B., and de Freitas, N. (2011). On autoencoders and score matching for energy based models. In ICML'2011. ACM.
18. Vincent, P. (2011). A connection between score matching and denoising autoencoders. Neural Computation, 23(7).