Factored 3-way restricted Boltzmann machines for modeling natural images

Journal of Machine Learning Research

Volumn 9, Issue , 2010, Pages 621-628

  Ranzato, Marc'aurelio ^a   Krizhevsky, Alex ^a   Hinton, Geoffrey E ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BINARY FEATURES; CELL ARCHITECTURES; COVARIANCE STRUCTURES; DATA SETS; HAND-WRITTEN CHARACTERS; HIDDEN UNITS; LOCAL COVARIANCE; NATURAL IMAGES; PROBABILISTIC FRAMEWORK; REAL IMAGES; REAL-VALUED DATA; RESTRICTED BOLTZMANN MACHINE;

OBJECT RECOGNITION;

ARTIFICIAL INTELLIGENCE;

EID: 84862277721     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (37)

1. M. Black and A. Rangarajan. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. IJCV, 19(1):57-92, 1996.
2. M. A. Carreira-Perpignan and G. E. Hinton. On contrastive divergence learning. Artificial Intelligence and Statistics, 2005.
3. K. Fukushima and S. Miyake. Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognition, 1982.
4. S. Geman and D. Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE PAMI, 6: 721-741, 1984.
5. G. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14:1771-1800, 2002.
6. G. Hinton. Learning to represent visual input. Phil. Trans. R. Soc., 365(1537):177-184, 2010.
7. G. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, 2006.
8. G. Hinton and Y. Teh. Discovering multiple constraints that are frequently approximately satisfied. In UAI, 2001.
9. G. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554, 2006a.
10. G. E. Hinton. Boltzmann machine, 2007. Scholarpedia, 2(5):1668.
11. G. E. Hinton, S. Osindero, M. Welling, and Y. Teh. Unsupervised discovery of non-linear structure using contrastive backpropagation. Cognitive Science, 30:725-731, 2006b.
12. A. Hyvarinen, J. Karhunen, and E. Oja. Independent Component Analysis. John Wiley & Sons, 2001.
13. Y. Karklin and M. Lewicki. Emergence of complex cell properties by learning to generalize in natural scenes. Nature, 457:83-86, 2009.
14. K. Kavukcuoglu, M. Ranzato, R. Fergus, and Y. LeCun. Learning invariant features through topographic filter maps. In CVPR, 2009.
15. U. Koster and A. Hyvarinen. A two-layer ica-like model estimated by score matching. In ICANN, 2007.
16. A. Krizhevsky. Learning multiple layers of features from tiny images, 2009. MSc Thesis, Dept. of Comp. Science, Univ. of Toronto.
17. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proc. of IEEE, 86 (11):2278-2324, 1998.
18. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proc. ICML, 2009.
19. D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 2004.
20. R. Memisevic and G. Hinton. Learning to represent spatial transformations with factored higher-order boltzmann machines. Neural Comp., pages 1-20, 2010.
21. V. Mnih. Cudamat: a CUDA-based matrix class for python. Technical Report UTML TR 2009-004, Dept. Computer Science, Univ. of Toronto, 2009.
22. R. Neal. Bayesian learning for neural networks. Springer-Verlag, 1996.
23. A. Oliva and A. Torralba. Modeling the shape of the scene: a holistic representation of the spatial envelope. IJCV, 42:145-175, 2001.
24. B. A. Olshausen and D. J. Field. Sparse coding with an overcomplete basis set: a strategy employed by v1? Vision Research, 37:3311-3325, 1997.
25. S. Osindero and G. E. Hinton. Modeling image patches with a directed hierarchy of markov random fields. In NIPS, 2008.
26. S. Osindero, M. Welling, and G. E. Hinton. Topographic product models applied to natural scene statistics. Neural Comp., 18: 344-381, 2006.
27. N. Pinto, D. D. Cox, and J. J. DiCarlo. Why is real-world visual object recognition hard? PLoS Computational Biology, 4(1), 2008.
28. R. Raina, A. Battle, H. Lee, B. Packer, and A. Ng. Self-taught learning: Transfer learning from unlabeled data. In ICML, 2007.
29. M. Ranzato, F. Huang, Y. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In CVPR, 2007.
30. T. Sejnowski. Higher-order boltzmann machines. In AIP Neural networks for computing, 1986.
31. T. Serre, L.Wolf, and T. Poggio. Object recognition with features inspired by visual cortex. In CVPR, 2005.
32. E. Simoncelli and B. A. Olshausen. Natural image statistics and neural representation. Annual Review of Neuroscience, 24: 1193-1216, 2001.
33. G. W. Taylor and G. E. Hinton. Factored conditional restricted boltzmann machines for modeling motion style. In ICML, 2009.
34. Y. W. Teh, M. Welling, S. Osindero, and G. E. Hinton. Energybased models for sparse overcomplete representations. JMLR, 4:1235-1260, 2003.
35. A. Torralba, R. Fergus, andW. Freeman. 80 million tiny images: a large dataset for non-parametric object and scene recognition. PAMI, 30:1958-1970, 2008a.
36. A. Torralba, R. Fergus, and Y. Weiss. Small codes and large databases for recognition. In Computer Vision and Pattern Recognition, 2008b.
37. M. Wainwright and E. Simoncelli. Scale mixtures of gaussians and the statistics of natural images. In NIPS, 2000.