Layer-wise analysis of deep networks with Gaussian kernels

Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Volumn , Issue , 2010, Pages

  Montavon, Grégoire ^a   Braun, Mikio L ^a   Müller, Klaus Robert ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE;

COMPLEXES STRUCTURE; GAUSSIAN KERNELS; HIGHER-LEVEL ABSTRACT REPRESENTATION; LAYER-WISE; LEARNING MACHINES; LEARNING PROBLEM; LOCAL LEARNING;

LEARNING SYSTEMS;

EID: 85161973295     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems 19, pages 153-160. MIT Press, 2007.
2. Yoshua Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
3. Mikio L. Braun. Accurate bounds for the eigenvalues of the kernel matrix. Journal of Machine Learning Research, 7:2303-2328, Nov 2006.
4. Mikio L. Braun, Joachim Buhmann, and Klaus-Robert Müller. On relevant dimensions in kernel feature spaces. Journal of Machine Learning Research, 9:1875-1908, Aug 2008.
5. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In International Conference on Machine Learning, ICML, 2008.
6. Dumitru Erhan, Yoshua Bengio, Aaron C. Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. Why does unsupervised pre-training help deep learning? Journal of Machine Learning Research, 11:625-660, 2010.
7. Ian Goodfellow, Quoc Le, Andrew Saxe, and Andrew Y. Ng. Measuring invariances in deep networks. In Advances in Neural Information Processing Systems 22, pages 646-654, 2009.
8. G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural networks. Science, 313(5786):504-507, July 2006.
9. Geoffrey E. Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning algorithm for deep belief nets. Neural Comput., 18(7):1527-1554, 2006.
10. D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of physiology, 160:106-154, January 1962.
11. Kevin Jarrett, Koray Kavukcuoglu, Marc'Aurelio Ranzato, and Yann LeCun. What is the best multistage architecture for object recognition? In Proc. International Conference on Computer Vision (ICCV'09). IEEE, 2009.
12. Hugo Larochelle, Yoshua Bengio, Jérôme Louradour, and Pascal Lamblin. Exploring strategies for training deep neural networks. J. Mach. Learn. Res., 10:1-40, 2009. ISSN 1532-4435.
13. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1):2278-2324, November 1998.
14. Hossein Mobahi, Ronan Collobert, and Jason Weston. Deep learning from temporal coherence in video. In Léon Bottou and Michael Littman, editors, Proceedings of the 26th International Conference on Machine Learning, pages 737-744, Montreal, June 2009. Omnipress.
15. Noboru Murata, Shuji Yoshizawa, and Shunichi Amari. Network information criterion - determining the number of hidden units for an artificial neural network model. IEEE Transactions on Neural Networks, 5:865-872, 1994.
16. Genevieve B. Orr and Klaus-RobertMüller, editors. Neural Networks: Tricks of the Trade, this book is an outgrowth of a 1996 NIPS workshop, volume 1524 of Lecture Notes in Computer Science, 1998. Springer.
17. M. A. Ranzato, Fu J. Huang, Y. L. Boureau, and Y. LeCun. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on, pages 1-8, 2007.
18. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 323(6088):533-536, 1986.
19. Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput., 10(5):1299-1319, 1998.
20. Jason Weston, Frédéric Ratle, and Ronan Collobert. Deep learning via semi-supervised embedding. In ICML '08: Proceedings of the 25th international conference onMachine learning, pages 1168-1175, 2008.
21. Alexander Zien, Gunnar Rätsch, Sebastian Mika, Bernhard Schölkopf, Thomas Lengauer, and Klaus-Robert Müller. Engineering support vector machine kernels that recognize translation initiation sites. Bioinformatics, 16(9):799-807, 2000.