General stochastic networks for classification

Advances in Neural Information Processing Systems

Volumn 3, Issue January, 2014, Pages 2015-2023

  Zöhrer, Matthias ^a   Pernkopf, Franz ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

COST FUNCTIONS; ECONOMIC AND SOCIAL EFFECTS; INFORMATION SCIENCE; STOCHASTIC SYSTEMS;

BASELINE MODELS; COMPETITIVE PERFORMANCE; HYBRID TRAINING; MULTIPLE NETWORKS; NETWORK TRAINING; NOISE INJECTION; STOCHASTIC NETWORKS; TRADE OFF;

NETWORK LAYERS;

EID: 84937840224     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. G. E. Hinton, S. Osindero, and Y. Teh, "A fast learning algorithm for deep belief nets." Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006.
2. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks," in Advances in Neural Information Processing Systems (NIPS), 2007, pp. 153-160.
3. P. Vincent, H. Larochelle, Y. Bengio, and P. Manzagol, "Extracting and composing robust features with denoising autoencoders," in International Conference on Machine Learning (ICML), 2008, pp. 1096-1103.
4. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms," in Advances in Neural Information Processing Systems (NIPS), 2007, pp. 801-808.
5. J. Ngiam, Z. Chen, S. A. Bhaskar, P. W. Koh, and A. Y. Ng, "Sparse filtering," in Advances in Neural Information Processing Systems (NIPS), 2011, pp. 1125-1133.
6. M. Ranzato, M. Poultney, S. Chopra, and Y. LeCun, "Efficient learning of sparse representations with an energy-based model," in Advances in Neural Information Processing Systems (NIPS), 2006, pp. 1137-1144.
7. G. E. Dahl, M. Ranzato, A. Mohamed, and G. E. Hinton, "Phone recognition with the mean-covariance restricted Boltzmann machine," in Advances in Neural Information Processing Systems (NIPS), 2010, pp. 469-477.
8. L. Deng, M. L. Seltzer, D. Yu, A. Acero, A. Mohamed, and G. E. Hinton, "Binary coding of speech spectrograms using a deep auto-encoder." in Interspeech, 2010, pp. 1692-1695.
9. F. Seide, G. Li, and D. Yu, "Conversational speech transcription using context-dependent deep neural networks." in Interspeech, 2011, pp. 437-440.
10. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems (NIPS), 2012, pp. 1097-1105.
11. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, 1998.
12. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," CoRR, vol. abs/1207.0580, 2012.
13. Y. Lecun and C. Cortes, "The MNIST database of handwritten digits," 2014. [Online]. Available: http://yann.lecun.com/exdb/mnist/
14. H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio, "An empirical evaluation of deep architectures on problems with many factors of variation," in International Conference on Machine Learning (ICML), 2007, pp. 473-480.
15. H. Larochelle, M. Mandel, R. Pascanu, and Y. Bengio, "Learning algorithms for the classification restricted Boltzmann machine," Journal of Machine Learning Research (JMLR), vol. 13, pp. 643-669, 2012.
16. Y. Bengio, L. Yao, G. Alain, and P. Vincent, "Generalized denoising auto-encoders as generative models," in Advances in Neural Information Processing Systems (NIPS), 2013, pp. 899-907.
17. Y. Bengio, E. Thibodeau-Laufer, and J. Yosinski, "Deep generative stochastic networks trainable by back-prop," CoRR, vol. abs/1306.1091, 2013.
18. L. Wan and M. Zeiler, "Regularization of neural networks using dropconnect," in International Conference on Machine Learning (ICML), 2013, pp. 109-111.
19. G. Mesnil, Y. Dauphin, X. Glorot, S. Rifai, Y. Bengio, I. J. Goodfellow, E. Lavoie, X. Muller, G. Desjardins, D. Warde-Farley, P. Vincent, A. Courville, and J. Bergstra, "Unsupervised and transfer learning challenge: a deep learning approach," in Unsupervised and Transfer Learning challenge and workshop (JMLR W& CP), 2012, pp. 97-110.
20. K. Abhishek and D. Hal, "Learning task grouping and overlap in multi-task learning," in International Conference on Machine Learning (ICML), 2012.
21. S. Ozair, L. Yao, and Y. Bengio, "Multimodal transitions for generative stochastic networks." CoRR, vol. abs/1312.5578, 2013.
22. P. Smolensky, Information processing in dynamical systems: Foundations of harmony theory. MIT Press, 1986, vol. 1, no. 1, pp. 194-281.
23. X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks," in International Conference on Artificial Intelligence and Statisitics (AISTATS), 2011, pp. 315-323.
24. G. E. Hinton, "A practical guide to training restricted boltzmann machines," in Neural Networks: Tricks of the Trade (2nd ed.), ser. Lecture Notes in Computer Science. Springer, 2012, pp. 599-619.
25. H. Larochelle, D. Erhan, A. Courville, J. Bergstra, and Y. Bengio, "Online companion for the paper an empirical evaluation of deep architectures on problems with many factors of variation," 2014. [Online]. Available: http://www.iro.umontreal.ca/~lisa/twiki/bin/view.cgi/Public/DeepVsShallowComparisonICML2007
26. R. Salakhutdinov and G. E. Hinton, "Deep boltzmann machines," in International Conference on Artificial Intelligence and Statistics (AISTATS), 2009, pp. 448-455.
27. I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "Maxout networks," in International Conference on Machine Learning (ICML), 2013, pp. 1319-1327.
28. I. J. Goodfellow, A. C. Courville, and Y. Bengio, "Joint training deep boltzmann machines for classification," CoRR, vol. abs/1301.3568, 2013.
29. D. Yu and L. Deng, "Deep convex net: A scalable architecture for speech pattern classification." in Interspeech, 2011, pp. 2285-2288.
30. S. Rifai, Y. Dauphin, P. Vincent, Y. Bengio, and X. Muller, "The manifold tangent classifier," in Advances in Neural Information Processing Systems (NIPS), 2012, pp. 2294-2302.
31. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio, "Theano: a CPU and GPU math expression compiler," in Python for Scientific Computing Conference (SciPy), 2010.
32. M. Zeiler and R. Fergus, "Stochastic pooling for regularization of deep convolutional neural networks," CoRR, vol. abs/1301.3557, 2013.