Top-down regularization of deep belief networks

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Goh, Hanlin ^a,b   Thome, Nicolas ^a   Cord, Matthieu ^a   Lim, Joo Hwee ^b,c  

^a SORBONNE UNIVERSITÉ   (France)

^b INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^c 21 01 Connexis South Tower   (France)

Author keywords

[No Author keywords available]

Indexed keywords

GLOBAL OPTIMIZATION; OBJECT RECOGNITION;

BOTTOM-UP AND TOP-DOWN; BUILDING BLOCKES; DEEP ARCHITECTURES; DEEP BELIEF NETWORKS; EFFECTIVE ALGORITHMS; LEARNING PROCEDURES; OPTIMIZATION PROCEDURES; RESTRICTED BOLTZMANN MACHINE;

ALGORITHMS;

EID: 84898949859     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (28)

1. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief networks, " Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.
2. Y. LeCun, "Une procédure d'apprentissage pour réseau a seuil asymmetrique (a learning scheme for asymmetric threshold networks), " in Cognitiva 85, 1985.
3. D. E. Rumelhart, G. E. Hinton, and R. J.Williams, "Learning representations by back-propagating errors, " Nature, vol. 323, pp. 533 - 536, October 1986.
4. Y. Bengio, "Learning deep architectures for AI, " Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
5. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks, " in NIPS, 2006.
6. M. Ranzato, C. Poultney, S. Chopra, and Y. LeCun, "Efficient learning of sparse representations with an energy-based model, " in NIPS, 2006.
7. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, "Extracting and composing robust features with denoising autoencoders, " in ICML, 2008.
8. P. Smolensky, "Information processing in dynamical systems: Foundations of harmony theory, " in Parallel Distributed Processing: Volume 1: Foundations, ch. 6, pp. 194-281, MIT Press, 1986.
9. G. E. Hinton, "Training products of experts by minimizing contrastive divergence, " Neural Computation, vol. 14, no. 8, p. 1771-1800, 2002.
10. H. Lee, C. Ekanadham, and A. Ng, "Sparse deep belief net model for visual area V2, " in NIPS, 2008.
11. H. Goh, N. Thome, and M. Cord, "Biasing restricted Boltzmann machines to manipulate latent selectivity and sparsity, " in NIPS Workshop, 2010.
12. H. Larochelle and Y. Bengio, "Classification using discriminative restricted Boltzmann machines, " in ICML, 2008.
13. N. Le Roux and Y. Bengio, "Representational power of restricted Boltzmann machines and deep belief networks, " Neural Computation, vol. 20, pp. 1631-1649, June 2008.
14. I. Sutskever and G. E. Hinton, "Learning multilevel distributed representations for high-dimensional sequences, " in AISTATS, 2007.
15. G. E. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 28, pp. 504-507, 2006.
16. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, pp. 2278-2324, November 1998.
17. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories, " CVPR Workshop, 2004.
18. R. Salakhutdinov and G. E. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure, " in AISTATS, 2007.
19. L. Deng and D. Yu, "Deep convex net: A scalable architecture for speech pattern classification, " in Interspeech, 2011.
20. D. C. Cireşan, U. Meier, and J. Schmidhuber, "Multi-column deep neural networks for image classification, " in CVPR, 2012.
21. D. Lowe, "Object recognition from local scale-invariant features, " in CVPR, 1999.
22. Y. Boureau, F. Bach, Y. LeCun, and J. Ponce, "Learning mid-level features for recognition, " in CVPR, 2010.
23. H. Goh, N. Thome, M. Cord, and J.-H. Lim, "Unsupervised and supervised visual codes with restricted Boltzmann machines, " in ECCV, 2012.
24. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, " in CVPR, 2006.
25. S. Avila, N. Thome, M. Cord, E. Valle, and A. Araújo, "Pooling in image representation: the visual codeword point of view, " Computer Vision and Image Understanding, pp. 453-465, May 2013.
26. C. Theriault, N. Thome, and M. Cord, "Extended coding and pooling in the HMAX model, " IEEE Transaction on Image Processing, 2013.
27. K. Sohn, D. Y. Jung, H. Lee, and A. Hero III, "Efficient learning of sparse, distributed, convolutional feature representations for object recognition, " in ICCV, 2011.
28. K. Sohn, G. Zhou, C. Lee, and H. Lee, "Learning and selecting features jointly with point-wise gated boltzmann machines, " in ICML, 2013.