Deep learning regularized fisher mappings

IEEE Transactions on Neural Networks

Volumn 22, Issue 10, 2011, Pages 1668-1675

  Wong, W K ^a   Sun, Mingming ^a,b  

^a HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^b NANJING UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Deep learning architecture; feature extraction; Fisher criterion; regularization

Indexed keywords

CLASS SEPARABILITY; CLASSIFICATION TASKS; DATA SETS; DEEP LEARNING; DISCRIMINATIVE ABILITY; FEATURE EXTRACTION METHODS; FEATURE EXTRACTOR; FEATURE SPACE; FISHER CRITERION; FLEXIBLE MODEL; KERNEL METHODS; LEARNING ARCHITECTURES; LEARNING PROCEDURES; NONLOCAL; OVERFITTING; REGULARIZATION; SAMPLE SPACE; SIDE EFFECT;

FEATURE EXTRACTION; MAPPING; NETWORK ARCHITECTURE;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER PROGRAM; HUMAN; METHODOLOGY; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; HUMANS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SOFTWARE; SOFTWARE DESIGN;

EID: 80053619156     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2162429     Document Type: Article

References (37)

1. G. Baudat and F. Anouar, "Generalized discriminant analysis using a kernel approach," Neural Comput., vol. 12, no. 10, pp. 2385-2404, Oct. 2000.
2. J. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, "Face recognition using kernel direct discriminant analysis algorithms," IEEE Trans. Neural Netw., vol. 14, no. 1, pp. 117-126, Jan. 2003.
3. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," J. Mach. Learn. Res., vol. 5, pp. 27-72, Jan. 2004.
4. S.-J. Kim, A. Magnani, and S. Boyd, "Optimal kernel selection in kernel Fisher discriminant analysis," in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 465-472.
5. J. Ye and J. Chen, "Multiclass discriminant kernel learning via convex programming," J. Mach. Learn. Res., vol. 9, pp. 719-758, Apr. 2008.
6. J. Huang, P. C. Yuen, W.-S. Chen, and J. H. Lai, "Choosing parameters of kernel subspace LDA for recognition of face images under pose and illumination variations," IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 37, no. 4, pp. 847-862, Aug. 2007. (Pubitemid 47103937)
7. D. G. Tzikas, A. C. Likas, and N. P. Galatsanos, "Sparse Bayesian modeling with adaptive kernel learning," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 926-37, Jun. 2009.
8. H. Xiong, M. N. S. Swamy, and M. O. Ahmad, "Optimizing the kernel in the empirical feature space," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 460-474, Mar. 2005. (Pubitemid 40390828)
9. S. Amari and S. Wu, "Improving support vector machine classifiers by modifying kernel functions," Neural Netw., vol. 12, no. 6, pp. 783-789, Jul. 1999. (Pubitemid 29359218)
10. J. Lu, K. Plataniotis, and A. Venetsanopoulos, "Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition," Pattern Recognit. Lett., vol. 26, no. 2, pp. 181-191, Jan. 2005.
11. E. D. Vito and A. Caponnetto, "Some properties of regularized kernel methods," J. Mach. Learn. Res., vol. 5, pp. 1363-1390, Oct. 2004.
12. X. Jiang, B. Mandal, and A. Kot, "Eigenfeature regularization and extraction in face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 3, pp. 383-94, Mar. 2008. (Pubitemid 351280047)
13. G. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, Jul. 2006. (Pubitemid 44148451)
14. I. Sutskever and G. E. Hinton, "Deep, narrow sigmoid belief networks are universal approximators," Neural Comput., vol. 20, no. 11, pp. 2629-2636, Nov. 2008.
15. N. Le Roux and Y. Bengio, "Deep belief networks are compact universal approximators," Neural Comput., vol. 22, no. 8, pp. 2192-2207, Aug. 2010.
16. Y. Bengio and Y. Lecun, Scaling Learning Algorithms Toward AI. Cambridge, MA: MIT Press, 2007, pp. 1-41.
17. M. Sugiyama, "Local Fisher discriminant analysis for supervised dimensionality reduction," in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 905-912.
18. S. Amari and S. Wu, "Improving support vector machine classifiers by modifying kernel functions," Neural Netw., vol. 12, no. 6, pp. 783-789, Jul. 1999. (Pubitemid 29359218)
19. L. V. D. Maaten, "Learning a parametric embedding by preserving local structure," in Proc. 12th Int. Conf. Artif. Intell. Stat., 2009, pp. 384-391.
20. R. Salakhutdinov and G. Hinton, "Learning a nonlinear embedding by preserving class neighbourhood structure," in Proc. Artif. Intell. Stat., 2007, pp. 1-8.
21. J.Weston, F. Ratle, and R. Collobert, "Deep learning via semi-supervised embedding," in Proc. 25th Int. Conf. Mach. Learn., 2008, pp. 1168-1175.
22. R. Salakhutdinov and G. Hinton, "Using deep belief nets to learn covariance kernels for Gaussian processes," in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 1-8.
23. A. Stuhlsatz, J. Lippel, and T. Zielke, "Discriminative feature extraction with deep neural networks," in Proc. Int. Joint Conf. Neural Netw., Barcelona, Spain, Jul. 2010, pp. 18-23.
24. A. Stuhlsatz, J. Lippel, and T. Zielke, "Feature extraction for simple classification," in Proc. 20th Int. Conf. Pattern Recognit., Istanbul, Turkey, Aug. 2010, pp. 1525-1528.
25. D. Erhan, A. Courville, and P. Vincent, "Why does unsupervised pretraining help deep learning?" J. Mach. Learn. Res., vol. 11, pp. 625-660, Feb. 2010.
26. C. S. Cruz and J. R. Dorronsoro, "A nonlinear discriminant algorithm for feature extraction and data classification," IEEE Trans. Neural Netw., vol. 9, no. 6, pp. 1370-1376, Nov. 1998. (Pubitemid 128742415)
27. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Pearson Education, 1999.
28. B. Chen, H. Liu, and Z. Bao, "Optimizing the data-dependent kernel under a unified kernel optimization framework," Pattern Recognit., vol. 41, no. 6, pp. 2107-2119, Jun. 2008. (Pubitemid 351191973)
29. A. Asuncion and D. Newman. (2007). UCI Machine Learning Repository [Online]. Available: http://archive.ics.uci.edu/ml/
30. A. Martinez and R. Benavente, "The AR face database," Centre Vis. Comput., Univ. Autònoma Barcelona, Barcelona, Spain, Tech. Rep. 24, Jun. 1998.
31. K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda, and B. Scholkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001. (Pubitemid 32371478)
32. J. Yang, D. Zhang, and B. Niu, "Globally maximizing, locally minimizing: Unsupervised discriminant projection with applications to face and palm biometrics," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 4, pp. 650-664, Apr. 2007. (Pubitemid 46464405)
33. B. Chen, H. Liu, and Z. Bao, "Optimizing the data-dependent kernel under a unified kernel optimization framework," Pattern Recognit., vol. 41, no. 6, pp. 2107-2119, Jun. 2008. (Pubitemid 351191973)
34. H. Xiong, M. N. S. Swamy, and M. O. Ahmad, "Optimizing the kernel in the empirical feature space," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 460-474, Mar. 2005. (Pubitemid 40390828)
35. S.-J. Kim, A. Magnani, and S. Boyd, "Optimal kernel selection in kernel Fisher discriminant analysis," in Proc. 23rd Int. Conf. Mach. Learn., 2006, pp. 465-472.
36. J. Ye and J. Chen, "Multi-class discriminant kernel learning via convex programming," J. Mach. Learn. Res., vol. 9, pp. 719-758, Apr. 2008.
37. Y. Bengio and Y. Lecun, Scaling Learning Algorithms Toward AI. Cambridge, MA: MIT Press, 2007, pp. 1-41.