Integrating supervised subspace criteria with restricted Boltzmann Machine for feature extraction

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2014, Pages 1622-1629

  Xie, Guo Sen ^a   Zhang, Xu Yao ^a   Zhang, Yan Ming ^a   Liu, Cheng Lin ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP NEURAL NETWORKS; DISCRIMINANT ANALYSIS; EIGENVALUES AND EIGENFUNCTIONS; GRADIENT METHODS; LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS; OPTIMIZATION;

BUILDING BLOCKES; DISCRIMINATIVE ABILITY; EIGENVALUE DECOMPOSITION; FISHER LINEAR DISCRIMINANT ANALYSIS; GRADIENT DESCENT; GRAPH CONSTRUCTION; MARGINAL FISHER ANALYSIS; RESTRICTED BOLTZMANN MACHINE;

NEURAL NETWORKS;

EID: 84908494825     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2014.6889447     Document Type: Conference Paper

References (33)

1. G. E. Hinton and T. J. Sejnowski, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Chapter 7 Learning and Relearning in Boltzmann Machines, pp. 282-317, MIT Press, USA, 1986.
2. G. E. Hinton, "Training products of experts by minimizing contrastive divergence", Neural Computation, vol. 14, pp. 1771-1800, 2002.
3. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks", Science, vol. 313, pp. 504-507, 2006.
4. R. R. Salakhutdinov and G. E. Hinton, "Deep Boltzmann machines", Proceedings of The Twelfth International Conference on Articial Intelligence and Statistics, vol. 5, pp. 448-455, 2009.
5. Y. Bengio, P. Lamblin, D. Popovici and H. Larochelle, "Greedy layerwise training of deep networks", Advances in Neural Information Processing Systems, vol. 19, pp. 153-160, 2007.
6. R. R. Salakhutdinov and G. E. Hinton, "Semantic hashing", International Journal of Approximate Reasoning, vol. 50, pp. 969-978, 2009.
7. H. Larochelle, Y. Bengio, J. Louradour and P. Lamblin, "Exploring strategies for training deep neural networks", Journal of Machine Learning Research, vol. 1, pp. 1-40, 2009.
8. M. Ranzato, C. Poultney, S. Chopra and Y. LeCun, "Efficient learning of sparse representations with an energy-based model", Advances in Neural Information Processing Systems, pp. 1137-1144, 2006.
9. H. Lee, C. Ekanadham and A. Y. Ng, "Sparse deep belief net model for visual area V2", Advances in Neural Information Processing Systems, pp. 873-880, 2008.
10. V. Nair and G. E. Hinton, "3D object recognition with deep belief nets", Advances in Neural Information Processing Systems, pp. 1339-1347, 2009.
11. H. Larochelle and Y. Bengio, "Classification using discriminative restricted Boltzmann machines", Proceedings of the 25th international conference on Machine learning, pp. 536-543, 2008.
12. A. Stuhlsatz, J. Lippel and T. Zielke, "Feature extraction with deep neural networks by a generalized discriminant analysis", IEEE Transactions on Neural Networks and Learning Systems, vol. 23, pp. 596-608, 2012.
13. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation", Neural Computation, vol. 15, pp. 1373-1396, 2003.
14. J. B. Tenenbaum, V. De Silva and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction", Science, vol. 290, pp. 2319-2323, 2000.
15. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding", Science, vol. 290, pp. 2323-2326, 2000.
16. S. Yan, D. Xu, B. Zhang, H. J. Zhang, Q. Yang and S. Lin, "Graph embedding and extensions: a general framework for dimensionality reduction", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, pp. 40-51, 2007.
17. M. K. Wong and M. Sun, "Deep learning regularized Fisher mappings", IEEE Transactions on Neural Networks, vol. 22, pp. 1668-1675, 2011.
18. J. Weston, F. Ratle, H. Mobahi and R. Collobert, Neural Networks: Tricks of the Trade. Chapter 26 Deep learning via semi-supervised embedding, pp. 639-655, Springer Berlin Heidelberg, 2012.
19. R. Salakhutdinov and G. E. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure", International Conference on Artificial Intelligence and Statistics, pp. 412-419, 2007.
20. W. Yu, G. Zeng, P. Luo, F. Zhuang, Q. He and Z. Shi, "Embedding with Autoencoder Regularization", Machine Learning and Knowledge Discovery in Databases, pp. 208-223, 2013.
21. M. Sugiyama, "Local Fisher discriminant analysis for supervised dimensionality reduction", Proceedings of the 23rd international conference on Machine learning, pp. 905-912, 2006.
22. R. A. Fisher, "The use of multiple measurements in taxonomic problems", Annals of Eugenics, vol. 7, pp. 179-188, 1936.
23. K. Fukunaga, Introduction to Statistical Pattern Recognition. Boston Academic Press, Inc, 1990.
24. P. Belhumeur, J. Hespanha and D. Kriegman, "Eigenfaces vs. fisherfaces: recognition using class specific linear projection", IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-720, 1997.
25. Y. LeCun and C. Cortes, "The MNIST database of handwritten digits", Available: http://yann. lecun. com/exdb/mnist.
26. F. Alimoglu and E. Alpaydin, "Combining multiple representations and classifiers for pen-based handwritten digit recognition", Proceedings of the Fourth International Conference on Document Analysis and Recognition, vol. 2, pp. 637-640, 1997.
27. B. M. Marlin, K. Swersky, B. Chen and N. D. Freitas, "Inductive principles for restricted Boltzmann machine learning", Proc. International Conference on Artificial Intelligence and Statistics, pp. 509-516, 2010.
28. http://people.cs.umass.edu/marlin/data.shtml.
29. D. Cai, X. F. He and J. W. Han, "SRDA: an efficient algorithm for large scale discriminant analysis", IEEE Transactions on Knowledge and Data Engineering, vol. 20, pp. 1-12, 2008.
30. D. Cai, "Spectral Regression: A Regression Framework for Efficient Regularized Subspace Learning", PhD Thesis, UIUC, Computer Science, 2009.
31. D. Cai, X. F. He, Y. X. Hu, J. W. Han and T. Huang, "Learning a Spatially Smooth Subspace for Face Recognition", Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-7, 2007.
32. G. E. Hinton, "A practical guide to training restricted Boltzmann machines", Momentum, 2010.
33. Y. Bengio, J. F. Paiement, P. Vincent, O. Delalleau, N. Le Roux and M. Ouimet, "Out-of-sample extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral clustering", Advances In Neural Information Processing Systems, vol. 16, pp. 177-184, 2004.