How to estimate the regularization parameter for spectral regression discriminant analysis and its kernel version?

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 24, Issue 2, 2014, Pages 211-223

  Gui, Jie ^a,b   Sun, Zhenan ^b   Cheng, Jun ^c   Ji, Shuiwang ^d   Wu, Xindong ^e  

^a INSTITUTE OF INTELLIGENT MACHINES   (China)

^b INSTITUTE OF AUTOMATION   (China)

^c SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

^d Old Dominion University   (United States)

^e NONE   (United States)

Author keywords

Kernel methods; perturbation linear discriminant analysis; regularization parameter estimation; spectral regression discriminant analysis

Indexed keywords

EID: 84894545214     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2013.2273652     Document Type: Article

References (67)

1. Y. Luo, D. Tao, B. Geng, C. Xu, and S. Maybank, "Manifold regularized multi-task learning for semi-supervised multi-label image classification, " IEEE Trans. Image Process., vol. 22, no. 2, pp. 523-536, Feb. 2013.
2. J. Yu, M. Wang, and D. Tao, "Semi supervised multiview distance metric learning for cartoon synthesis, " IEEE Trans. Image Process., vol. 21, no. 11, pp. 4636-4648, Nov. 2012.
3. J. Yu, D. Liu, D. Tao, and H. S. Seah, "On combining multiple features for cartoon character retrieval and clip synthesis, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 5, pp. 1413-1427, Oct. 2012.
4. D. Tao, L. Liang, L. Jin, and Y Gao, "Similar handwritten chinese character recognition using discriminative locality alignment manifold learning, " in Proc. Int. Conf. Doc. Anal. Recognit., 2011, pp. 1012-1016.
5. D. Tao and L. Jin, "Discriminative information preservation for face recognition, " Neurocomput., vol. 91, no. 15, pp. 11-20, 2012.
6. D. Tao, L. Jin, W. Liu, and X. Li, "Hessian regularized support vector machines for mobile image annotation on the cloud, " IEEE Trans. Multimedia, vol. 15, no. 4, pp. 833-844, Jun. 2013.
7. X. Tian, D. Tao, and Y Rui, "Sparse transfer learning for interactive video search reranking, " in Proc. ACM Trans. Multimedia Comput., Commun., Appl., vol. 8, no. 3, 2012, pp. 1-19.
8. X. Tian, D. Tao, X.-S. Hua, and X. Wu, "Active reranking for web image search, " IEEE Trans. Image Process., vol. 19, no. 3, pp. 805-820, Mar. 2010.
9. X. Wang, Z. Li, and D. Tao, "Subspaces indexing model on grassmann manifold for image search, " IEEE Trans. Image Process., vol. 20, no. 9, pp. 2627-2635, Sep. 2011.
10. C. Keßler, "What is the difference? a cognitive dissimilarity measure for information retrieval result sets, " Knowl. Inf. Syst., vol. 30, no. 2, pp. 319-340, 2012.
11. J. Gui, S.-L. Wang, and Y.-K. Lei, "Multi-step dimensionality reduction and semi-supervised graph-based tumor classification using gene expression data, " Artif. Intell. Med., vol. 50, no. 3, pp. 181-191, 2010.
12. X. Kong and S. Y Philip, "gMLC: A multi-label feature selection framework for graph classification, " Knowl. Inf. Syst., vol. 31, no. 2, pp. 281-305, 2012.
13. T. Zhou and D. Tao, "Double shrinking sparse dimension reduction, " IEEE Trans. Image Process., vol. 22, no. 1, pp. 244-257, Jan. 2013.
14. B. Geng, D. Tao, C. Xu, L. Yang, and X.-S. Hua, "Ensemble manifold regularization, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 34, no. 6, pp. 1227-1233, Jun. 2012.
15. J. C. Gomez, E. Boiy, and M.-F. Moens, "Highly discriminative statistical features for email classification, " Knowl. Inf. Syst., vol. 31, no. 1, pp. 23-53, 2012.
16. W. Bian and D. Tao, "Constrained empirical risk minimization framework for distance metric learning, " IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 8, pp. 1194-1205, Aug. 2012.
17. N. Lavesson and S. Axelsson, "Similarity assessment for removal of noisy end user license agreements, " Knowl. Inf. Syst., vol. 32, no. 1, pp. 167-189, 2012.
18. W. Yang, C. Sun, and L. Zhang, "A multi-manifold discriminant analysis method for image feature extraction, " Pattern Recognit., vol. 44, no. 8, pp. 1649-1657, 2011.
19. T. de Vries, S. Chawla, and M. E. Houle, "Density-preserving projections for large-scale local anomaly detection, " Knowl. Inf. Syst., vol. 32, no. 1, pp. 25-52, 2012.
20. S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, "Graph embedding and extensions: A general framework for dimensionality reduction, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 40-51, Jan. 2007. (Pubitemid 46415944)
21. D. Xu, S. Lin, S. Yan, and X. Tang, "Rank-one projections with adaptive margins for face recognition, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 5, pp. 1226-1236, Oct. 2007. (Pubitemid 47541886)
22. X. Wang and X. Tang, "A unified framework for subspace face recognition, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 26, no. 9, pp. 1222-1228, Sep. 2004.
23. J. Yang, D. Zhang, and J. Yang, "Constructing PCA baseline algorithms to reevaluate ICA-based face-recognition performance, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 4, pp. 1015-1021, Aug. 2007. (Pubitemid 47103950)
24. D. Tao, X. Li, X. Wu, and S. Maybank, "Geometric mean for subspace selection, " IEEE Trans. Pattern Anal. Mach. Intell, vol. 31, no. 2, pp. 260-274, Feb. 2009.
25. N. Bouguila and D. Ziou, "A countably infinite mixture model for clustering and feature selection, " Knowl. Inf. Syst., vol. 33, no. 2, pp. 351-370, 2012.
26. W. Zuo, D. Zhang, J. Yang, and K. Wang, "BDPCA plus LDA: a novel fast feature extraction technique for face recognition, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 4, pp. 946-953, Aug. 2006. (Pubitemid 44178163)
27. T. Zhang, K. Huang, X. Li, J. Yang, and D. Tao, "Discriminative orthogonal neighborhood-preserving projections for classification, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 1, pp. 253-263, Feb. 2010.
28. J. Gui, Z. Sun, W. Jia, R. Hu, Y. Lei, and S. Ji, "Discriminant sparse neighborhood preserving embedding for face recognition, " Pattern Recognit., vol. 45, no. 8, pp. 2884-2893, 2012.
29. P. Belhumeur, J. Hespanha, and D. Kriegman, "Eigenfaces vs. fish-erfaces: Recognition using class specific linear projection, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, July 1997. (Pubitemid 127762263)
30. H. Li, T. Jiang, and K. Zhang, "Efficient and robust feature extraction by maximum margin criterion, " IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 157-165, Jan. 2006. (Pubitemid 43263948)
31. J. Liu, S. Chen, X. Tan, and D. Zhang, "Comments on efficient and robust feature extraction by maximum margin criterion, " IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1862-1864, Nov. 2007. (Pubitemid 350148434)
32. X. Li, S. Lin, S. Yan, and D. Xu, "Discriminant locally linear embedding with high-order tensor data, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 2, pp. 342-352, Apr. 2008. (Pubitemid 351479557)
33. Y. Fu and T. S. Huang, "Image classification using correlation tensor analysis, " IEEE Trans. Image Process., vol. 17, no. 2, pp. 226-234, Feb. 2008. (Pubitemid 351279250)
34. Y. Fu, S. Yan, and T. S. Huang, "Correlation metric for generalized feature extraction, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 12, pp. 2229-2235, Dec. 2008.
35. J. Gui, W. Jia, L. Zhu, S.-L. Wang, and D.-S. Huang, "Locality preserving discriminant projections for face and palmprint recognition, " Neurocomput., vol. 73, no. 13, pp. 2696-2707, 2010.
36. Y. Pang, D. Tao, Y. Yuan, and X. Li, "Binary two-dimensional PCA, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 4, pp. 1176-1180, Aug. 2008.
37. J. Li and D. Tao, "Simple exponential family PCA, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 3, pp. 485-497, Mar. 2013.
38. J. Li and D. Tao, "On preserving original variables in bayesian PCA with application to image analysis, " IEEE Trans. Image Process., vol. 21, no. 12, pp. 4830-4843, Dec. 2012.
39. X. He and P. Niyogi, "Locality preserving projections, " in Proc. Neural Inf. Process. Syst., vol. 16, pp. 153-160, 2003.
40. X. He, S. Yan, Y. Hu, P. Niyogi, and H. Zhang, "Face recognition using laplacianfaces, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 3, pp. 328-340, Mar. 2005. (Pubitemid 40377326)
41. N. Guan, D. Tao, Z. Luo, and B. Yuan, "NeNMF: An optimal gradient method for nonnegative matrix factorization, " IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2882-2898, Jun. 2012.
42. N. Guan, D. Tao, Z. Luo, and J. Shawe-Taylor, "MahNMF: Manhattan non-negative matrix factorization, " arXiv preprint arXiv:1207.3438, 2012.
43. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Non-negative patch alignment framework, " IEEE Trans. Neural Netw., vol. 22, no. 8, pp. 1218-1230, Aug. 2011.
44. M. Belkin and P. Niyogi, "Semi-supervised learning on riemannian manifolds, " Mach. Learn., vol. 56, no. 1, pp. 209-239, 2004.
45. D. Zhang, Z. Zhou, and S. Chen, "Semi-supervised dimensionality reduction, " in Proc. SIAM Int. Conf. Data Min., 2007, pp. 629-634.
46. D. Cai, X. He, and J. Han, "Semi-supervised discriminant analysis, " in Proc. Int. Conf. Comput. Vision, 2007, pp. 1-7.
47. Y. Song, F. Nie, C. Zhang, and S. Xiang, "A unified framework for semi-supervised dimensionality reduction, " Pattern Recognit., vol. 41, no. 9, pp. 2789-2799, 2008.
48. F. Nie, D. Xu, X. Li, and S. Xiang, "Semisupervised dimensionality reduction and classification through virtual label regression, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 99, pp. 1-11, Jun. 2011.
49. C. S. Dhir, J. Lee, and S.-Y. Lee, "Extraction of independent discriminant features for data with asymmetric distribution, " Knowl. Inf. Syst., vol. 30, no. 2, pp. 359-375, 2012.
50. J. Yang, S. Yan, and T. Huang, "Ubiquitously supervised subspace learning, " IEEE Trans. Image Process., vol. 18, no. 2, pp. 241-249, Feb. 2009.
51. J. Ye, T. Li, T. Xiong, and R. Janardan, "Using uncorrelated discriminant analysis for tissue classification with gene expression data, " IEEE/ACM TCBB, vol. 1, no. 4, pp. 181-190, 2004. (Pubitemid 40325595)
52. W. Ayadi, M. Elloumi, and J.-K. Hao, "Bicfinder: A biclustering algorithm for microarray data analysis, " Knowl. Inf. Syst., vol. 30, no. 2, pp. 341-358, 2012.
53. D. Tao, X. Li, X. Wu, and S. Maybank, "General tensor discriminant analysis and gabor features for gait recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1700-1715, Oct. 2007.
54. D. Cai, X. He, and J. Han, "Speed up kernel discriminant analysis, " VLDB J., vol. 20, no. 1, pp. 21-33, 2011.
55. D. Cai, X. He, and J. Han, "SRDA: An efficient algorithm for large-scale discriminant analysis, " IEEE Trans. Knowl. Data Eng., vol. 20, no. 1, pp. 1-12, Jan. 2008.
56. D. Cai, X. He, and J. Han, "Efficient kernel discriminant analysis via spectral regression, " in Proc. Int. Conf. Data Min., 2007, pp. 427-432.
57. J. van Gemert, C. Veenman, A. Smeulders, and J. Geusebroek, "Visual word ambiguity, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1271-1283, July 2010.
58. W. Chen, C. Shan, and G. De Haan, "Optimal regularization parameter estimation for spectral regression discriminant analysis, " IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 12, pp. 1921-1926, Dec. 2009.
59. W. Zheng, J. Lai, P. Yuen, and S. Li, "Perturbation LDA: Learning the difference between the class empirical mean and its expectation, " Pattern Recognit., vol. 42, no. 5, pp. 764-779, 2009.
60. J. Friedman, "Regularized discriminant analysis, " J. Amer. Statist. Asso., vol. 84, no. 405, pp. 165-175, 1989.
61. J. Ye, "Least squares linear discriminant analysis, " in Proc. Int. Conf. Mach. Learn., 2007, pp. 1087-1093. (Pubitemid 47275175)
62. J. Ye and T. Xiong, "Computational and theoretical analysis of null space and orthogonal linear discriminant analysis, " J. Mach. Learn. Res., vol. 7, pp. 1183-1204, 2006. (Pubitemid 44015300)
63. S. Ji and J. Ye, "Generalized linear discriminant analysis: A unified framework and efficient model selection, " IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1768-1782, Oct. 2008.
64. G. Baudat and F. Anouar, "Generalized discriminant analysis using a kernel approach, " Neural Comput., vol. 12, no. 10, pp. 2385-2404, 2000.
65. J. Hull, "A database for handwritten text recognition research, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 16, no. 5, pp. 550-554, 1994.
66. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
67. J. Gui, Z. Sun, and T. Tan, "Regularization parameter estimation for spectral regression discriminant analysis based on perturbation theory, " in Proc. Int. Conf. Pattern Recognit., pp. 401-404, 2012.