Deep learning with hierarchical convolutional factor analysis

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 8, 2013, Pages 1887-1901

  Chen, Bo ^a   Polatkan, Gungor ^b   Sapiro, Guillermo ^a   Blei, David ^b   Dunson, David ^c   Carin, Lawrence ^a  

^a USA   (United States)

^b Princeton University   (United States)

^c DUKE UNIVERSITY   (United States)

Author keywords

Bayesian; convolutional; deep learning; dictionary learning; factor analysis

Indexed keywords

BAYESIAN; CONVOLUTIONAL; DEEP LEARNING; DICTIONARY LEARNING; GIBBS SAMPLERS; MODEL PARAMETERS; ONLINE VERSIONS; VARIATIONAL BAYESIAN;

FACTOR ANALYSIS; MULTIVARIANT ANALYSIS;

CONVOLUTION;

EID: 84879853859     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2013.19     Document Type: Article

References (41)

1. M. Zeiler, D. Krishnan, G. Taylor, and R. Fergus, "Deconvolution Networks," Proc. IEEE Conf. Computer Vision Pattern Recognition, 2010.
2. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
3. G. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, vol. 313, no. 5786, pp. 504-507, 2006. (Pubitemid 44148451)
4. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "What Is the Best Multi-Stage Architecture for Object Recognition?" Proc. IEEE Int'l Conf. Computer Vision, 2009.
5. M. Ranzato, C. Poultney, S. Chopra, and Y. LeCun, "Efficient Learning of Sparse Representations with an Energy-Based Model," Proc. Neural Information Processing Systems, 2006.
6. P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, "Extracting and Composing Robust Features with Denoising Autoencoders," Proc. Int'l Conf. Machine Learning, 2008.
7. H. Lee, R. Grosse, R. Ranganath, and A.Y. Ng, "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations," Proc. Int'l Conf. Machine Learning, 2009.
8. H. Lee, Y. Largman, P. Pham, and A. Ng, "Unsupervised Feature Learning for Audio Classification Using Convolutional Deep Belief Networks," Proc. Neural Information Processing Systems, 2009.
9. M.R.M. Norouzi and G. Mori, "Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
10. H. Lee, C. Ekanadham, and A.Y. Ng, "Sparse Deep Belief Network Model for Visual Area V2," Proc. Neural Information Processing Systems, 2008.
11. Y. Boureau, F. Bach, Y. LeCun, and J. Ponce, "Learning Mid-Level Features for Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
12. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, "Locality-Constrained Linear Coding for Image Classification," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
13. Y.-L. Boureau, J. Ponce, and Y. LeCun, "A Theoretical Analysis of Feature Pooling in Vision Algorithms," Proc. Int'l Conf. Machine Learning, 2010.
14. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online Dictionary Learning for Sparse Coding," Proc. Int'l Conf. Machine Learning, 2009.
15. T.L. Griffiths and Z. Ghahramani, "Infinite Latent Feature Models and the Indian Buffet Process," Proc. Advances in Neural Information Processing Systems, pp. 475-482, 2005.
16. R. Thibaux and M. Jordan, "Hierarchical Beta Processes and the Indian Buffet Process," Proc. Int'l Conf. Artificial Intelligence and Statistics, 2007.
17. J. Paisley and L. Carin, "Nonparametric Factor Analysis with Beta Process Priors," Proc. Int'l Conf. Machine Learning, 2009.
18. M. Zhou, H. Chen, J. Paisley, L. Ren, G. Sapiro, and L. Carin, "Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations," Proc. Neural Information Processing Systems, 2009.
19. R. Adams, H. Wallach, and Z. Ghahramani, "Learning the Structure of Deep Sparse Graphical Models," Proc. Int'l Conf. Artificial Intelligence and Statistics, 2010.
20. B. Chen, G. Polatkan, G. Sapiro, D. Dunson, and L. Carin, "The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning," Proc. Int'l Conf. Machine Learning, 2011.
21. M. West, "Bayesian Factor Regression Models in the 'Large p, Small n' Paradigm," Bayesian Statistics 7, J.M. Bernardo, M. Bayarri, J. Berger, A. Dawid, D. Heckerman, A. Smith, and M. West, eds., pp. 723-732, Oxford Univ. Press, 2003.
22. C. Carvalho, J. Chang, J. Lucas, J.R. Nevins, Q. Wang, and M. West, "High-Dimensional Sparse Factor Modelling: Applications in Gene Expression Genomics," J. Am. Statistical Assoc., vol. 103, pp. 1438-1456, 2008.
23. M. Tipping, "Sparse Bayesian Learning and the Relevance Vector Machine," J. Machine Learning Research, vol. 1, pp. 211-244, 2001. (Pubitemid 33687203)
24. R. Adams, H. Wallach, and Z. Ghahramani, "Learning the Structure of Deep Sparse Graphical Models," Proc. Int'l Conf. Artificial Intelligence and Statistics, 2010.
25. M. Hoffman, D. Blei, and F. Bach, "Online Learning for Latent Dirichlet Allocation," Proc. Advances in Neural Information Processing Systems, 2010.
26. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge Univ. Press, 2000.
27. R.C.J. Weston and F. Ratle, "Deep Learning via Semi-Supervised Embedding," Proc. Int'l Conf. Machine Learning, 2008.
28. Y.-L. Boureau, M. Ranzato, and F.-J. Huang, "Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
29. P.O. Hoyer, "Non-Negative Matrix Factorization with Sparseness Constraints," J. Machine Learning Research, vol. 5, pp. 1457-1469, 2004.
30. D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Non-Negative Matrix Factorization," Nature, vol. 401, no. 6755 pp. 788-791, 1999.
31. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
32. H. Zhang, A.C. Berg, M. Maire, and J. Malik, "SVM-KNN: Discriminative Nearest Neighbor Classification for Visual Category Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
33. K. Yu, Y. Lin, and J. Lafferty, "Learning Image Representations from the Pixel Level via Hierarchical Sparse Coding," Proc. IEEE Conf, Computer Vision and Pattern Recognition, Dec. 2011.
34. L. Bo, X. Ren, and D. Fox, "Hierarchical Matching Pursuit for Image Classification: Architecture and Fast Algorithms," Proc. Advances in Neural Information Processing Systems, Dec. 2011.
35. M.D. Zeiler, G.W. Taylor, and R. Fergus, "Adaptive Deconvolutional Networks for Mid and High Level Feature Learning," Proc. IEEE Int'l Conf. Computer Vision, 2011.
36. Y. Boureau, F. Bach, Y. LeCun, and J. Ponce, "Learning Mid-Level Features for Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
37. K. Kavukcuoglu, P. Sermanet, Y. Boureau, K. Gregor, M. Mathieu, and Y. LeCun, "Learning Convolutional Feature Hierachies for Visual Recognition," Proc. Advances in Neural Information Processing Systems, 2010.
38. C.R. Johnson, E. Hendriks, I. Berezhnoy, E. Brevdo, S.M. Hughes, I. Daubechies, J. Li, E. Postma, and J.Z. Wang, "Image Processing for Artist Identification: Computerized Analysis of Vincent van Gogh's Brushstrokes," IEEE Signal Processing Magazine, vol. 25, no. 4, pp. 37-48, July 2008.
39. D. Marr, Vision. Freeman, 1982.
40. D.L. Donoho, "Nature vs. Math: Interpreting Independent Component Analysis in Light of Recent work in Harmonic Analysis," Proc. Int'l Workshop Independent Component Analysis and Blind Signal Separation, pp. 459-470, 2000.
41. D. Knowles and Z. Ghahramani, "Infinite Sparse Factor Analysis and Infinite Independent Components Analysis," Proc. Seventh Int'l Conf. Independent Component Analysis and Signal Separation, 2007.