A theory of generative convNet

33rd International Conference on Machine Learning, ICML 2016

Volumn 6, Issue , 2016, Pages 3895-3904

  Xie, Jianwen ^a   Lu, Yang ^a   Zhu, Song Chun ^a   Wu, Ying Nian ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; FUNCTIONS; GAUSSIAN DISTRIBUTION; IMAGE PROCESSING; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD; WHITE NOISE;

CONTRASTIVE DIVERGENCE; ENERGY-BASED MODELS; GAUSSIAN WHITE NOISE; LANGEVIN DYNAMICS; MAXIMUM LIKELIHOOD LEARNING; MULTI-CATEGORY CLASSIFICATION; RANDOM FIELD MODEL; RECONSTRUCTION ERROR;

LEARNING ALGORITHMS;

EID: 84998996685     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Alain, Guillaume and Bengio, Yoshua. What regularized auto-encoders learn from the data-generating distribution. The Journal of Machine Learning Research, 15(1): 3563-3593, 2014.
2. Dai, Jifeng, Lu, Yang, and Wu, Ying Nian. Generative modeling of convolutional neural networks. In ICLR, 2015.
3. Hinton, Geoffrey E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14 (8):1771-1800, 2002.
4. Hinton, Geoffrey E., Osindero, Simon, and Teh, Yee- Whye. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554, 2006a.
5. Hinton, Geoffrey E, Osindero, Simon, Welling, Max, and Teh, Yee-Whye. Unsupervised discovery of nonlinear structure using contrastive backpropagation. Cognitive Science, 30(4):725-731, 2006b.
6. Hopfield, John J. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8): 2554-2558, 1982.
7. Hyvarinen, Aapo. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6:695-709, 2005.
8. Hyvarinen, Aapo. Connections between score matching, contrastive divergence, and pseudolikelihood for continuous-valued variables. Neural Networks, IEEE Transactions on, 18(5): 1529-1531, 2007.
9. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In NIPS, pp. 1097-1105, 2012.
10. LeCun, Yann, Bottou, Leon, Bengio, Yoshua, and Haffner, Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278- 2324, 1998.
11. LeCun, Yann, Chopra, Sumit, Hadsell, Rata, Ranzato, Mare'Aurelio, and Huang, Fu Jie. A tutorial on energy- based learning. In Predicting Structured Data. MIT Press, 2006.
12. Lee, Honglak, Grosse, Roger, Ranganath, Rajesh, and Ng, Andrew Y. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In ICML, pp. 609-616. ACM, 2009.
13. Lu, Yang, Zhu, Song-Chun, and Wu, Ying Nian. Learning FRAME models using CNN filters. In Thirtieth AAAI Conference on Artificial Intelligence, 2016.
14. Montufar, Guido F, Pascanu, Razvan, Cho, Kyunghyun, and Bengio, Yoshua. On the number of linear regions of deep neural networks. In NIPS, pp. 2924-2932, 2014.
15. Ngiam, Jiquan, Chen, Zhenghao, Koh, Pang Wei, and Ng, Andrew Y. Learning deep energy models. In ICML, 2011.
16. Olshausen, Bruno A and Field, David J. Sparse coding with an overcomplete basis set: A strategy employed by vl? Vision Research, 37(23):3311-3325, 1997.
17. Roth, Stefan and Black, Michael J. Fields of experts: A framework for learning image priors. In CVPR, volume 2, pp. 860-867. IEEE, 2005.
18. Salakhutdinov, Ruslan and Hinton, Geoffrey E. Deep boltz- mann machines. In AISTATS, 2009.
19. Swersky, Kevin, Ranzato, Mare'Aurelio, Buchman, David, Marlin, Benjamin, and Freitas, Nando. On autoencoders and score matching for energy based models. In ICML, pp. 1201-1208. ACM, 2011.
20. Teh, Yee-Whye, Welling, Max, Osindero, Simon, and Hinton, Geoffrey E. Energy-based models for sparse overcomplete representations. The Journal of Machine Learning Research, 4:1235-1260, 2003.
21. Vedaldi, A. and Lenc, K. Matconvnet - convolutional neural networks for matlab. In Proceeding of the ACM Int. Conf. on Multimedia, 2015.
22. Vincent, Pascal. A connection between score matching and denoising autoencoders. Neural Computation, 23 (7): 1661-1674, 2011.
23. Wu, Ying Nian, Zhu, Song-Chun, and Guo, Cheng-En. From information scaling of natural images to regimes of statistical models. Quarterly of Applied Mathematics, 66:81-122, 2008.
24. Xie, Jianwen, Hu, Wenze, Zhu, Song-Chun, and Wu, Ying Nian. Learning sparse FRAME models for natural image patterns. International Journal of Computer Vision, pp. 1-22, 2014.
25. Xie, Jianwen, Lu, Yang, Zhu, Song-Chun, and Wu, Ying Nian. Inducing wavelets into random fields via generative boosting. Journal of Applied and Computational Harmonic Analysis, 41:4-25, 2016.
26. Younes, Laurent. On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates. Stochastics: An International Journal of Probability and Stochastic Processes, 65(3-4): 177-228, 1999.
27. Zeiler, Matthew D and Fergus, Rob. Visualizing and understanding convolutional neural networks. In ECCV, 2014.
28. Zhu, Song-Chun and Mumford, David. Grade: Gibbs reaction and diffusion equations. In ICCV, pp. 847-854, 1998.
29. Zhu, Song-Chun, Wu, Ying Nian, and Mumford, David. Minimax entropy principle and its application to texture modeling. Neural Computation, 9(8): 1627-1660, 1997.