A novel sparse auto-encoder for deep unsupervised learning

2013 6th International Conference on Advanced Computational Intelligence, ICACI 2013 - Proceedings

Volumn , Issue , 2013, Pages 256-261

  Jiang, Xiaojuan ^a   Zhang, Yinghua ^a   Zhang, Wensheng ^a   Xiao, Xian ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; NEURAL NETWORKS;

AUTO ENCODERS; BENCHMARK DATABASE; BUILDING BLOCKES; CLASSIFICATION ACCURACY; DEEP NEURAL NETWORKS; KL-DIVERGENCE; PRE-TRAINING; TRAINING PROCEDURES;

LEARNING SYSTEMS;

EID: 84896333951     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICACI.2013.6748512     Document Type: Conference Paper

References (24)

1. Y. Bengio, "Learning deep architectures for AI, " Foundations and trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
2. Y. Bengio, "Representation learning: A review and new perspectives, " IEEE Trans. Pattern Anal. Mach. intell., special issue Learning Deep Architectures, 2013.
3. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks, " Advances in Neural information Processing Systems, vol. 19, no. 153, 2007.
4. Y. Bengio and Y. LeCun, "Scaling learning algorithms towards AI, " Large-Scale Kernel Machines, vol. 34, 2007.
5. R. Byrd, J. Nocedal, and R. Schnabel, "Representations of quasi-Newton matrices and their use in limited memory methods, " Mathematical Programming, vol. 63, no. 1-3, pp. 129-156, 1994
6. C. T. Chu, S. K. Kim, Y. A. Lin, Y. Y. Yu, G. Bradski, A. Y. Ng, and K. Olukotun, "Map-reduce for machine learning on multicore, " Advances in Neural Information Processing Systems, vol. 19, no. 281, 2007.
7. D. C. Ciresan, U. Meier, L. M. Gambardella, and J. Schmidhuber, "Deep big simple neural nets excel on handwritten digit recognition, " CORR, 2010.
8. G. E. Hinton and R.R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
9. G. E. Hinton, S. Osindero, and Y.W. The, "A fast learning algorithm for deep belief nets, " Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006.
10. G. Hinton, "A practical guide to training Restricted Boltzmann Machines, " Momentum, vol. 9, no. 1, 2010.
11. Y. LeCun, F. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting, " Computer Vision and Pattern Recognition, 2004.
12. Q. V. Lee, J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, and A. Y. Ng, "On optimization methods for deep learning, " Proceedings of the 28th international Conference on Machine Learning, pp. 265-272, 2011.
13. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms, " Advances in Neural information Processing Systems, pp. 801-808, 2006.
14. H. Lee, C. Ekanadham, and A. Y. Ng, "Sparse deep belief net model for visual area V2, " Advances in Neural information Processing Systems, pp. 873-880, 2007.
15. V. Nair and G. E. Hinton, "Implicit mixtures of restricted Boltzmann machines, " Advances in Neural information Processing Systems, pp. 1145-1152, 2008.
16. V. Nair and G. E. Hinton. "3D object recognition with deep belief nets, " Advances in Neural information Processing Systems, pp. 1339-1347, 2009.
17. A. Y. Ng, "Feature selection, L 1 vs. L 2 regularization, and rotational invariance, " Proceedings of the 21st international Conference on Machine learning, 2004.
18. B. Olshausen and D. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images, " Nature, vol. 381, no. 6583, pp. 607-609, 1996.
19. R. Raina, A. Madhavan, and A. Y. Ng, "Large-scale deep unsupervised learning using graphics processors, " Proceedings of the 26th international Conference on Machine learning, vol. 9, pp. 873-880, 2009.
20. P. Simard, D. Steinkraus, and J. Platt, "Best practices for convolutional neural networks applied to visual document analysis, " ICDAR, vol. 3, pp. 958-962, 2003.
21. R. Salakhutdinov and G. Hinton, "Semantic hashing, " international Journal of Approximate Reasoning, vol. 50, no. 7, pp. 969-978, 2009.
22. R. Salakhutdinov and G. Hinton, "Deep Boltzmann machines, " international Conference on Artificial intelligence and Statistics, pp. 448-455, 2009.
23. G. Taylor, G. Hinton, and S. Roweis, "Modeling human motion using binary latent variables, " Advances in Neural information Processing Systems, pp. 1345-1352, 2006.
24. G. Taylor and G. Hinton, "Factored conditional restricted Boltzmann machines for modeling motion style. " Proceedings of the 26th international Conference on Machine Learning, pp. 1025-1032, 2009.