Regularized estimation of image statistics by Score Matching

Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Volumn , Issue , 2010, Pages

  Kingma, Diederik P ^a   LeCun, Yann ^b  

^a UTRECHT UNIVERSITY   (Netherlands)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

IMAGE DENOISING; SAMPLING;

DENSITY MODELING; HIGH-DIMENSIONAL; HIGHER-DIMENSIONAL; IMAGE STATISTICS; MA XIMUM LIKELIHOODS; MAXIMUM-LIKELIHOOD; MODELING PARAMETERS; NATURAL IMAGE STATISTICS; SCORE MATCHING; SIMPLE MODELING;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 85162419825     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. S. Becker and Y. LeCun. Improving the convergence of back-propagation learning with second-order methods. In D. Touretzky, G. Hinton, and T. Sejnowski, editors, Proc. of the 1988 Connectionist Models Summer School, pages 29-37, San Mateo, 1989. Morgan Kaufman.
2. C. M. Bishop. Neural networks for pattern recognition. Oxford University Press, Oxford, UK, 1996.
3. A. E. Bryson and Y. C. Ho. Applied optimal control; optimization, estimation, and control. Blaisdell Pub. Co. Waltham, Massachusetts, 1969.
4. M. A. Carreira-Perpinan and G. E. Hinton. On contrastive divergence learning. In Artificial Intelligence and Statistics, 2005.
5. H. Drucker and Y. LeCun. Improving generalization performance using double backpropagation. IEEE Transactions on Neural Networks, 3(6):991-997, 1992.
6. M. Gutmann and A. Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proc. Int. Conf. on Artificial Intelligence and Statistics (AISTATS2010), 2010.
7. G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14:2002, 2000.
8. G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18(7):1527-1554, 2006.
9. G. E. Hinton, S. Osindero, M. Welling, and Y. W. Teh. Unsupervised discovery of non-linear structure using contrastive backpropagation. Cognitive Science, 30(4):725-731, 2006.
10. A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6:695-709, 2005.
11. A. Hyvärinen. Some extensions of score matching. Computational Statistics & Data Analysis, 51(5):2499-2512, 2007.
12. A. Hyvärinen. Optimal approximation of signal priors. Neural Computation, 20:3087-3110, 2008.
13. U. Köster and A. Hyvärinen. A two-layer ica-like model estimated by score matching. In J. M. de Sá, L. A. Alexandre, W. Duch, and D. P. Mandic, editors, ICANN (2), volume 4669 of Lecture Notes in Computer Science, pages 798-807. Springer, 2007.
14. U. Koster, J. T. Lindgren, and A. Hyvärinen. Estimating markov random field potentials for natural images. Proc. Int. Conf. on Independent Component Analysis and Blind Source Separation (ICA2009), 2009.
15. U. Köster, J. T. Lindgren, and A. Hyvärinen. Estimating markov random field potentials for natural images. In T. Adali, C. Jutten, J. M. T. Romano, and A. K. Barros, editors, ICA, volume 5441 of Lecture Notes in Computer Science, pages 515-522. Springer, 2009.
16. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, pages 2278-2324, 1998.
17. Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang. A tutorial on energy-based learning. In G. Bakir, T. Hofman, B. Schölkopf, A. Smola, and B. Taskar, editors, Predicting Structured Data. MIT Press, 2006.
18. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int'l Conf. Computer Vision, volume 2, pages 416-423, July 2001.
19. S. Osindero and G. E. Hinton. Modeling image patches with a directed hierarchy of markov random fields. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 1121-1128. MIT Press, Cambridge, MA, 2008.
20. M. Ranzato, Y. Boureau, and Y. LeCun. Sparse feature learning for deep belief networks. In Advances in Neural Information Processing Systems (NIPS 2007), 2007.
21. S. Roth and M. J. Black. Fields of experts. International Journal of Computer Vision, 82(2):205-229, 2009.
22. A. N. Tikhonov. On the stability of inverse problems. Dokl. Akad. Nauk SSSR, (39):176-179, 1943.
23. Y. Weiss and W. T. Freeman. What makes a good model of natural images. In CVPR 2007: Proceedings of the 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, pages 1-8, 2007.
24. M. Welling, G. E. Hinton, and S. Osindero. Learning sparse topographic representations with products of student-t distributions. In S. T. S. Becker and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 1359-1366. MIT Press, Cambridge, MA, 2003.