Unsupervised Feature Selection via Nonnegative Spectral Analysis and Redundancy Control

IEEE Transactions on Image Processing

Volumn 24, Issue 12, 2015, Pages 5343-5355

  Li, Zechao ^a   Tang, Jinhui ^a  

^a NANJING UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

Author keywords

Constrained Redundancy; Feature Selection; Nonnegative Spectral Clustering; Row Sparsity

Indexed keywords

ALGORITHMS; CLUSTERING ALGORITHMS; IMAGE PROCESSING; ITERATIVE METHODS; OPTIMIZATION; PATTERN RECOGNITION; REDUNDANCY; SPECTRUM ANALYSIS;

DISCRIMINATIVE FEATURES; HIGH DIMENSIONAL FEATURE; ITERATIVE ALGORITHM; OPTIMIZATION PROBLEMS; PATTERN RECOGNITION PROBLEMS; ROW-SPARSITY; SPECTRAL CLUSTERING; UNSUPERVISED FEATURE SELECTION;

FEATURE EXTRACTION;

EID: 84943804725     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2479560     Document Type: Article

References (52)

1. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
2. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York, NY, USA: Wiley, 2001.
3. L. Wolf and A. Shashua, "Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach," J. Mach. Learn. Res., vol. 6, pp. 1855-1887, Jan. 2005.
4. J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, and V. Vapnik, "Feature selection for SVMs," in Proc. Adv. Neural Inf. Process. Syst., 2000, pp. 668-674.
5. X. He, D. Cai, and P. Niyogi, "Laplacian score for feature selection," in Proc. Adv. Neural Inf. Process. Syst., 2005, pp. 507-514.
6. W. Jiang, G. Er, Q. Dai, and J. Gu, "Similarity-based online feature selection in content-based image retrieval," IEEE Trans. Image Process., vol. 15, no. 3, pp. 702-712, Mar. 2006.
7. F. Nie, H. Huang, X. Cai, and C. H. Ding, "Efficient and robust feature selection via joint 2,1-norms minimization," in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 1813-1821.
8. Z. Zhao and H. Liu, "Spectral feature selection for supervised and unsupervised learning," in Proc. Int. Conf. Mach. Learn., 2007, pp. 1151-1157.
9. Z. Zhu, Y.-S. Ong, and M. Dash, "Markov blanket-embedded genetic algorithm for gene selection," Pattern Recognit., vol. 40, no. 11, pp. 3236-3248, Nov. 2007.
10. K.-Q. Shen, C.-J. Ong, X.-P. Li, and E. P. V. Wilder-Smith, "Feature selection via sensitivity analysis of SVM probabilistic outputs," Mach. Learn., vol. 70, no. 1, pp. 1-20, Jan. 2008.
11. D. Cai, C. Zhang, and X. He, "Unsupervised feature selection for multicluster data," in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2010, pp. 333-342.
12. J.-B. Yang and C.-J. Ong, "Feature selection using probabilistic prediction of support vector regression," IEEE Trans. Neural Netw. Learn. Syst., vol. 22, no. 6, pp. 954-962, Jun. 2011.
13. L. Song, A. Smola, A. Gretton, J. Bedo, and K. Borgwardt, "Feature selection via dependence maximization," J. Mach. Learn. Res., vol. 13, no. 1, pp. 1393-1434, Jan. 2012.
14. L. Zhang, C. Chen, J. Bu, and X. He, "A unified feature and instance selection framework using optimum experimental design," IEEE Trans. Image Process., vol. 21, no. 5, pp. 2379-2388, May 2012.
15. Z. Zhao, L. Wang, H. Liu, and J. Ye, "On similarity preserving feature selection," IEEE Trans. Knowl. Data Eng., vol. 25, no. 3, pp. 619-632, Mar. 2013.
16. Z. Li, J. Liu, Y. Yang, X. Zhou, and H. Lu, "Clustering-guided sparse structural learning for unsupervised feature selection," IEEE Trans. Knowl. Data Eng., vol. 26, no. 9, pp. 2138-2150, Sep. 2014.
17. H. C. Peng, F. Long, and C. Ding, "Feature selection based on mutual information criteria of max-dependency, max-relevance, and minredundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
18. Z. Zhao, L. Wang, and H. Liu, "Efficient spectral feature selection with minimum redundancy," in Proc. Nat. Conf. Artif. Intell., 2010, pp. 673-678.
19. R. Chakraborty and N. R. Pal, "Feature selection using a neural framework with controlled redundancy," IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 1, pp. 35-50, Jan. 2015.
20. P. Mitra, C. A. Murthy, and S. K. Pal, "Unsupervised feature selection using feature similarity," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 301-312, Mar. 2002.
21. H. Zeng and Y.-M. Cheung, "Feature selection and kernel learning for local learning-based clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1532-1547, Aug. 2011.
22. Y. Yang, H. T. Shen, Z. Ma, Z. Huang, and X. Zhou, "2,1-norm regularized discriminative feature selection for unsupervised learning," in Proc. Int. Joint Conf. Artif. Intell., 2011, pp. 1589-1954.
23. Z. Ma, F. Nie, Y. Yang, J. R. R. Uijlings, N. Sebe, and A. G. Hauptmann, "Discriminating joint feature analysis for multimedia data understanding," IEEE Trans. Multimedia, vol. 14, no. 6, pp. 1662-1672, Dec. 2012.
24. A. Jain and D. Zongker, "Feature selection: Evaluation, application, and small sample performance," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 2, pp. 153-158, Feb. 1997.
25. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Jan. 2003.
26. Y.-M. Cheung and H. Zeng, "Local kernel regression score for selecting features of high-dimensional data," IEEE Trans. Knowl. Data Eng., vol. 21, no. 12, pp. 1798-1802, Dec. 2009.
27. J. G. Dy and C. E. Brodley, "Feature selection for unsupervised learning," J. Mach. Learn. Res., vol. 5, pp. 845-889, Jan. 2004.
28. M. H. C. Law, M. A. T. Figueiredo, and A. K. Jain, "Simultaneous feature selection and clustering using mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1154-1166, Sep. 2004.
29. C. Constantinopoulos, M. K. Titsia, and A. Likas, "Bayesian feature and model selection for Gaussian mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 6, pp. 1013-1018, Jun. 2006.
30. Z. Li, Y. Yang, J. Liu, X. Zhou, and H. Lu, "Unsupervised feature selection using nonnegative spectral analysis," in Proc. Nat. Conf. Artif. Intell. (AAAI), 2012, pp. 1026-1032.
31. L. Du, Z. Shen, X. Li, P. Zhou, and Y.-D. Shen, "Local and global discriminative learning for unsupervised feature selection," in Proc. IEEE Int. Conf. Data Mining, Dec. 2013, pp. 131-140.
32. M. Qian and C. Zhai, "Robust unsupervised feature selection," in Proc. Int. Joint Conf. Artif. Intell., 2014, pp. 1621-1627.
33. L. Shi, L. Du, and Y.-D. Shen, "Robust spectral learning for unsupervised feature selection," in Proc. IEEE Int. Conf. Data Mining, Dec. 2014, pp. 977-982.
34. Z. Li, J. Liu, J. Tang, and H. Lu, "Robust structured subspace learning for data representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 10, pp. 2085-2098, Oct. 2015.
35. J. Tang, Z. Li, M. Wang, and R. Zhao, "Neighborhood discriminant hashing for large-scale image retrieval," IEEE Trans. Image Process., vol. 24, no. 9, pp. 2827-2840, Sep. 2015.
36. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy," J. Mach. Learn. Res., vol. 5, no. 10, pp. 1205-1224, 2004.
37. G. Qu, S. Hariri, and M. Yousif, "A new dependency and correlation analysis for features," IEEE Trans. Knowl. Data Eng., vol. 17, no. 9, pp. 1199-1207, Sep. 2005.
38. L. Zhou, L. Wang, and C. Shen, "Feature selection with redundancyconstrained class separability," IEEE Trans. Neural Netw. Learn. Syst., vol. 21, no. 5, pp. 853-858, May 2010.
39. Q. Song, J. Ni, and G. Wang, "A fast clustering-based feature subset selection algorithm for high-dimensional data," IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 1-14, Jan. 2013.
40. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
41. A. Y. Ng, M. I. Jordan, and Y. Weiss, "On spectral clustering: Analysis and an algorithm," in Proc. Adv. Neural Inf. Process. Syst., 2001, pp. 849-856.
42. R. Chartrand, "Exact reconstruction of sparse signals via nonconvex minimization," IEEE Signal Process. Lett., vol. 14, no. 10, pp. 707-710, Oct. 2007.
43. X. Chen, F. Xu, and Y. Ye, "Lower bound theory of nonzero entries in solutions of 2-p minimization," SIAM J. Sci. Comput., vol. 32, no. 5, pp. 2832-2852, 2010.
44. L. Wang, S. Chen, and Y. Wang, "A unified algorithm for mixed 2,p-minimizations and its application in feature selection," Comput. Optim. Appl., vol. 58, no. 2, pp. 409-421, Jun. 2014.
45. H. Kuhn and A. Tucker, "Nonlinear programming," in Proc. Berkeley Symp. Math. Statist. Probab., 1951, pp. 481-492.
46. Z. Yang, T. Hao, O. Dikmen, X. Chen, and E. Oja, "Clustering by nonnegative matrix factorization using graph random walk," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1079-1087.
47. Data for MATLAB Hackers. [Online]. Available: http://cs.nyu.edu/~roweis/data.html, accessed Oct. 16, 2012.
48. F. S. Samaria and A. C. Harter, "Parameterisation of a stochastic model for human face identification," in Proc. IEEE Workshop Appl. Comput. Vis., Dec. 1994, pp. 138-142.
49. M. J. Lyons, J. Budynek, and S. Akamatsu, "Automatic classification of single facial images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 12, pp. 1357-1362, Dec. 1999.
50. Y. Yang, D. Xu, F. Nie, S. Yan, and Y. Zhuang, "Image clustering using local discriminant models and global integration," IEEE Trans. Image Process., vol. 19, no. 10, pp. 2761-2773, Oct. 2010.
51. S. A. Nene, S. K. Nayar, and H. Murase, "Columbia object image library (COIL-20)," Dept. Comput. Sci., Columbia Univ., New York, NY, USA, Tech. Rep. CUCS-005-96, 1996.
52. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories," in Proc. IEEE Comput. Vis. Pattern Recognit. Workshop, Jun. 2004, pp. 178-186.