FREL: A stable feature selection algorithm

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 7, 2015, Pages 1388-1402

  Li, Yun ^a   Si, Jennie ^b   Zhou, Guojing ^a   Huang, Shasha ^a   Chen, Songcan ^c  

^a NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b Arizona State University   (United States)

^c NANJING UNIVERSITY OF AERONAUTICS AND ASTRONAUTICS   (China)

Author keywords

Energy based learning; ensemble; feature selection; feature weighting; uniform weighting stability

Indexed keywords

ALGORITHMS; STABILITY;

ENERGY-BASED; ENSEMBLE; FEATURE SELECTION ALGORITHM; FEATURE WEIGHTING; HIGH DIMENSIONALITY; IMPLEMENTATION STRATEGIES; SMALL SAMPLE SIZE PROBLEMS; STABILITY PROPERTIES;

FEATURE EXTRACTION;

EID: 85027920848     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2341627     Document Type: Article

References (43)

1. Z. Zhao, "Spectral feature selection for mining ultrahigh dimensional data, " Ph.D. dissertation, Dept. School Comput., Informat., Decision Syst. Eng., Arizona State Univ., Phoenix, AZ, USA, 2010.
2. H. Liu and L. Yu, "Toward integrating feature selection algorithms for classification and clustering, " IEEE Trans. Knowl. Data Eng., vol. 17, no. 4, pp. 494-502, Apr. 2005.
3. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection, " J. Mach. Learn. Res., vol. 31, pp. 1157-1182, Jan. 2003.
4. Y. Saeys, T. Abeel, and Y. van de Peer, "Robust feature selection using ensemble feature selection techniques, " in Proc. 25th Eur. Conf. Mach. Learn. Knowl. Discovery Databases, 2008, pp. 313-325.
5. S. Loscalzo, L. Yu, and C. Ding, "Consensus group stable feature selection, " in Proc. 15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2009, pp. 567-575.
6. T. Abeel, T. Helleputte, Y. van de Peer, P. Dupont, and Y. Saeys, "Robust biomarker identification for cancer diagnosis with ensemble feature selection methods, " Bioinformatics, vol. 26, no. 3, pp. 392-398, 2010.
7. Y. Li, S. Gao, and S. Chen, "Ensemble feature weighting based on local learning and diversity, " in Proc. 26th AAAI Conf. Artif. Intell., 2012, pp. 1019-1025.
8. A. Woznica, P. Nguyen, and A. Kalousis, "Model mining for robust feature selection, " in Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2012, pp. 913-921.
9. Y. Han and L. Yu, "A variance reduction framework for stable feature selection, " in Proc. 10th Int. Conf. Data Mining, Dec. 2010, pp. 206-215.
10. L. Yu, Y. Han, and M. E. Berens, "Stable gene selection from microarray data via sample weighting, " IEEE/ACM Trans. Comput. Biol. Bioinform., vol. 9, no. 1, pp. 262-272, Jan./Feb. 2012.
11. L. Yu, C. Ding, and S. Loscalzo, "Stable feature selection via dense feature groups, " in Proc. 14th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2008, pp. 803-811.
12. Z. He and W. Yu, "Stable feature selection for biomarker discovery, " Comput. Biol. Chem., vol. 34, no. 4, pp. 215-225, 2010.
13. Y. LeCun, S. Chopra, R. Hadsell, M. A. Ranzato, and F. J. Huang, A Tutorial on Energy-Based Model. Cambridge, MA, USA: MIT Press, 2006.
14. A. S. Das, M. Datar, A. Garg, and S. Rajaram, "Google news personalization: Scalable online collaborative filtering, " in Proc. 16th Int. Conf. World Wide Web, 2007, pp. 271-280.
15. S. Dudoit, J. Fridlyand, and T. P. Spee, "Comparison of discrimination methods for the classification of tumors using gene expression data, " J. Amer. Statist. Assoc., vol. 97, no. 457, pp. 77-87, 2002.
16. Y. Sun, S. Todorovic, and S. Goodison, "Local-learning-based feature selection for high-dimensional data analysis, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1610-1626, Sep. 2010.
17. P. Domingos, "A few useful things to know about machine learning, " Commun. ACM, vol. 55, no. 10, pp. 78-87, 2012.
18. B. Wang and H.-D. Chiang, "Elite: Ensemble of optimal input-pruned neural networks using TRUST-TECH, " IEEE Trans. Neural Netw., vol. 22, no. 1, pp. 96-109, Jan. 2011.
19. S. Tang, Y.-T. Zheng, Y. Wang, and T.-S. Chua, "Sparse ensemble learning for concept detection, " IEEE Trans. Multimedia, vol. 14, no. 1, pp. 43-54, Feb. 2012.
20. S. Gutta, J. R. J. Huang, P. Jonathon, and H. Wechsler, "Mixture of experts for classification of gender, ethnic origin, and pose of human faces, " IEEE Trans. Neural Netw., vol. 11, no. 4, pp. 948-960, Jul. 2000.
21. L. I. Kuncheva, J. J. Rodriguez, C. O. Plumpton, D. E. J. Linden, and S. J. Johnston, "Random subspace ensembles for fMRI classification, " IEEE Trans. Med. Imag., vol. 29, no. 2, pp. 531-542, Feb. 2010.
22. L. Breiman, "Bagging predictors, " Mach. Learn., vol. 26, no. 2, pp. 123-140, Aug. 1996.
23. O. Bousquet and A. Elisseeff, "Stability and generalization, " J. Mach. Learn. Res., vol. 2, pp. 499-526, Jan. 2002.
24. A. Elisseeff, T. Evgeniou, and M. Pontil, "Stability of randomized learning algorithm, " J. Mach. Learn. Res., vol. 6, pp. 55-79, Jan. 2005.
25. S. Agarwal and P. Niyogi, "Generalization bounds for ranking algorithm via algorithmic stability, " J. Mach. Learn. Res., vol. 10, pp. 441-474, Feb. 2009.
26. H. Xu, C. Caramanis, and S. Mannor, "Sparse algorithms are not stable: A no-free-lunch theorem, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 187-193, Jan. 2012.
27. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. Boston, MA, USA: Kluwer, 2004.
28. M. Tan, I. W. Tsang, and L. Wang, "Minimax sparse logistic regression for very high-dimensional feature selection, " IEEE Trans. Neural Netw. Learn. Syst., vol. 24, no. 10, pp. 1609-1622, Oct. 2013.
29. T. R. Golub et al., "Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring, " Science, vol. 286, no. 5439, pp. 531-537, 1999.
30. D. Singh et al., "Gene expression correlates of clinical prostate cancer behavior, " Cancer Cell, vol. 1, no. 2, pp. 203-209, 2002.
31. K. Kira and L. Rendell, "A practical approach to feature selection, " in Proc. 9th Int. Workshop Mach. Learn., 1992, pp. 249-256.
32. I. Kononenko, "Estimating attributes: Analysis and extensions of RELIEF, " in Proc. Eur. Conf. Mach. Learn., 1994, pp. 171-182.
33. M. Robnik-Sikonja and I. Kononenko, "Theoretical and empirical analysis of ReliefF and RReliefF, " Mach. Learn., vol. 53, nos. 1-2, pp. 23-69, 2003.
34. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York, NY, USA: Wiley, 2001.
35. T. G. Dietterich, "Machine learning research: Four current directions, " AI Mag., vol. 18, no. 4, pp. 97-136, 1997.
36. K. Crammer, R. Gilad-Bachrach, A. Navot, and N. Tishby, "Margin analysis of the LVQ algorithm, " in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2002, pp. 462-469.
37. Y. Li and B.-L. Lu, "Feature selection based on loss margin of nearest neighbor classification, " Pattern Recognit., vol. 42, no. 9, pp. 1914-1921, 2009.
38. A. Kalousis, J. Prados, and M. Hilario, "Stability of feature selection algorithms: A study on high-dimensional spaces, " Knowl. Inf. Syst., vol. 12, no. 1, pp. 95-116, 2007.
39. C. C. Chang and C. J. Lin. (2002). Libsvm: A Library for Support Vector Machines. [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/papers/libsvm.ps.gz
40. S. A. van de Geer, "High-dimensional generalized linear models and the lasso, " Ann. Statist., vol. 36, no. 2, pp. 614-645, 2008.
41. S. N. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu, "A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers, " Statist. Sci., vol. 27, no. 4, pp. 538-557, 2012.
42. S. N. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu, "A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers, " Dept. EECS., Univ. California, Berkeley, Berkeley, CA, USA, Tech. Rep. 797, 2010.
43. S. N. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu, "A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers, " in Advances in Neural Information Processing Systems. Red Hook, NY, USA: Curran & Associates, Inc., 2009, pp. 1348-1356.