Feature selection for classification: A review

Data Classification: Algorithms and Applications

Volumn , Issue , 2014, Pages 37-64

  Tang, Jiliang ^a   Alelyani, Salem ^a   Liu, Huan ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85054068951     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/b17320     Document Type: Chapter

References (81)

1. S. Alelyani, J. Tang, and H. Liu. Feature selection for clustering: A review. Data Clustering: Algorithms and Applications, Editors: Charu Aggarwal and Chandan Reddy, CRC Press, 2013.
2. S. Alelyani, L. Wang, and H. Liu. The effect of the characteristics of the dataset on the selection stability. In Proceedings of the 23rd IEEE International Conference on Tools with Artificial Intelligence, 2011.
3. F. R. Bach. Consistency of the group Lasso and multiple kernel learning. The Journal of Machine Learning Research, 9:1179–1225, 2008.
4. H. D. Bondell and B. J. Reich. Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with oscar. Biometrics, 64(1):115–123, 2008.
5. P. S. Bradley and O. L. Mangasarian. Feature selection via concave minimization and support vector machines. Proceedings of the Fifteenth International Conference on Machine Learning (ICML ’98), 82–90. Morgan Kaufmann, 1998.
6. E. Candes and T. Tao. The Dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 35(6):2313–2351, 2007.
7. H. Y. Chuang, E. Lee, Y.T. Liu, D. Lee, and T. Ideker. Network-based classification of breast cancer metastasis. Molecular Systems Biology, 3(1), 2007.
8. M. Dash and H. Liu. Feature selection for classification. IntelligentData Analysis, 1(1-4):131-156, 1997.
9. C. Domeniconi and D. Gunopulos. Local feature selection for classification. Computational Methods, page 211, 2008.
10. R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification. Wiley-Interscience, 2012.
11. R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification. New York: John Wiley & Sons, 2nd edition, 2001.
12. J.G. Dy and C.E. Brodley. Feature subset selection and order identification for unsupervised learning. In In Proc. 17th International Conference on Machine Learning, pages 247–254. Morgan Kaufmann, 2000.
13. J.G. Dy and C.E. Brodley. Feature selection for unsupervised learning. The Journal ofMachine Learning Research, 5:845–889, 2004.
14. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456):1348–1360, 2001.
15. N. L. C. Talbot G. C. Cawley, and M. Girolami. Sparse multinomial logistic regression via Bayesian l1 regularisation. In Neural Information Processing Systems, 2006.
16. Q. Gu, Z. Li, and J. Han. Towards feature selection in network. In International Conference on Information and Knowledge Management, 2011.
17. Q. Gu, Z. Li, and J. Han. Generalized Fisher score for feature selection. arXiv preprint arXiv:1202.3725, 2012.
18. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. The Journal of Machine Learning Research, 3:1157–1182, 2003.
19. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines. Machine learning, 46(1-3):389–422, 2002.
20. M.A. Hall and L.A. Smith. Feature selection for machine learning: Comparing a correlationbased filter approach to the wrapper. In Proceedings of the Twelfth International Florida Artificial Intelligence Research Society Conference, volume 235, page 239, 1999.
21. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2001.
22. T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6):607–616, 1996.
23. J. Huang, J. L. Horowitz, and S. Ma. Asymptotic properties of bridge estimators in sparse high-dimensional regression models. The Annals of Statistics, 36(2):587–613, 2008.
24. J. Huang, S. Ma, and C. Zhang. Adaptive Lasso for sparse high-dimensional regression models. Statistica Sinica, 18(4):1603, 2008.
25. J. Huang, T. Zhang, and D. Metaxas. Learning with structured sparsity. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 417–424. ACM, 2009.
26. I. Inza, P. Larrañaga, R. Blanco, and A. J. Cerrolaza. Filter versus wrapper gene selection approaches in dna microarray domains. Artificial intelligence in Medicine, 31(2):91–103, 2004.
27. L. Jacob, G. Obozinski, and J. Vert. Group Lasso with overlap and graph Lasso. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 433–440. ACM, 2009.
28. G. James, P. Radchenko, and J. Lv. Dasso: Connections between the Dantzig selector and Lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(1):127-142, 2009.
29. R. Jenatton, J. Audibert, and F. Bach. Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research, 12:2777–2824, 2011.
30. R. Jenatton, J. Mairal, G. Obozinski, and F. Bach. Proximal methods for sparse hierarchical dictionary learning. Journal of Machine Learning Research, 12:2297–2334, 2011.
31. D. Jensen and J. Neville. Linkage and autocorrelation cause feature selection bias in relational learning. In International Conference on Machine Learning, pages 259–266, 2002.
32. S. Kim and E. Xing. Statistical estimation of correlated genome associations to a quantitative trait network. PLoS Genetics, 5(8):e1000587, 2009.
33. S. Kim and E. Xing. Tree-guided group Lasso for multi-task regression with structured sparsity. In Proceedings of the 27th International Conference on Machine Learing, Haifa, Israel, 2010.
34. K. Kira and L. Rendell. A practical approach to feature selection. In Proceedings of the Ninth International Workshop on Machine Learning, pages 249–256. Morgan Kaufmann Publishers Inc., 1992.
35. K. Knight and W. Fu. Asymptotics for Lasso-type estimators. Annals of Statistics, 78(5):1356-1378, 2000.
36. R. Kohavi and G.H. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1-2):273–324, 1997.
37. D. Koller and M. Sahami. Toward optimal feature selection. In Proceedings of the International Conference on Machine Learning, pages 284–292, 1996.
38. H. Liu and H. Motoda. Feature selection for knowledge discovery and data mining, volume 454. Springer, 1998.
39. H. Liu and H. Motoda. ComputationalMethods of Feature Selection. Chapman and Hall/CRC Press, 2007.
40. H. Liu and L. Yu. Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4):491, 2005.
41. J. Liu, S. Ji, and J. Ye. Multi-task feature learning via efficient l 2, 1-norm minimization. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 339–348. AUAI Press, 2009.
42. J. Liu and J. Ye. Moreau-Yosida regularization for grouped tree structure learning. Advances in Neural Information Processing Systems, 187:195–207, 2010.
43. Z. Liu, F. Jiang, G. Tian, S. Wang, F. Sato, S. Meltzer, and M. Tan. Sparse logistic regression with lp penalty for biomarker identification. Statistical Applications in Genetics and Molecular Biology, 6(1), 2007.
44. B. Long, Z.M. Zhang, X. Wu, and P.S. Yu. Spectral clustering for multi-type relational data. In Proceedings of the 23rd International Conference on Machine Learning, pages 585–592. ACM, 2006.
45. B. Long, Z.M. Zhang, and P.S. Yu. A probabilistic framework for relational clustering. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 470–479. ACM, 2007.
46. S. Ma and J. Huang. Penalized feature selection and classification in bioinformatics. Briefings in Bioinformatics, 9(5):392–403, 2008.
47. S.A. Macskassy and F. Provost. Classification in networked data: A toolkit and a univariate case study. The Journal of Machine Learning Research, 8:935–983, 2007.
48. M. Masaeli, J G Dy, and G. Fung. From transformation-based dimensionality reduction to feature selection. In Proceedings of the 27th International Conference on Machine Learning, pages 751–758, 2010.
49. J. McAuley, J Ming, D. Stewart, and P. Hanna. Subband correlation and robust speech recognition. IEEE Transactions on Speech and Audio Processing, 13(5):956–964, 2005.
50. L. Meier, S. De Geer, and P. Bühlmann. The group Lasso for logistic regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(1):53–71, 2008.
51. P. Mitra, C. A. Murthy, and S. Pal. Unsupervised feature selection using feature similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24:301–312, 2002.
52. H. Peng, F. Long, and C. Ding. Feature selection based on mutual information:Criteria ofmaxdependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1226–1238, 2005.
53. S. Perkins and J. Theiler. Online feature selection using grafting. In Proceedings of the International Conference on Machine Learning, pages 592–599, 2003.
54. J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
55. J. R. Quinlan. Induction of decision trees. Machine Learning, 1(1):81–106, 1986.
56. M. Robnik Sikonja and I. Kononenko. Theoretical and empirical analysis of ReliefF and RReliefF. Machine Learning, 53(1-2):23–69, 2003.
57. Y. Saeys, I. Inza, and P. Larrañaga. A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19):2507–2517, 2007.
58. T. Sandler, P. Talukdar, L. Ungar, and J. Blitzer. Regularized learning with networks of features. Neural Information Processing Systems, 2008.
59. P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93, 2008.
60. M. R. Sikonja and I. Kononenko. Theoretical and empirical analysis of ReliefF and RReliefF. Machine Learning, 53:23–69, 2003.
61. L. Song, A. Smola, A. Gretton, K. Borgwardt, and J. Bedo. Supervised feature selection via dependence estimation. In Proceedings of the 24th International Conference on Machine Learning, pages 823–830, 2007.
62. J. Tang and H. Liu. Feature selection with linked data in socialmedia. In SDM, pages 118–128, 2012.
63. B. Taskar, P. Abbeel, M.F. Wong, and D. Koller. Label and link prediction in relational data. In Proceedings of the IJCAI Workshop on Learning Statistical Models from Relational Data. Citeseer, 2003.
64. R. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288, 1996.
65. R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight. Sparsity and smoothness via the fused Lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1):91–108, 2005.
66. R. Tibshirani and P. Wang. Spatial smoothing and hot spot detection for cgh data using the fused Lasso. Biostatistics, 9(1):18–29, 2008.
67. J. Wang, P. Zhao, S. Hoi, and R. Jin. Online feature selection and its applications. IEEE Transactions on Knowledge and Data Engineering, pages 1–14, 2013.
68. J. Weston, A. Elisseff, B. Schoelkopf, and M. Tipping. Use of the zero norm with linear models and kernel methods. Journal of Machine Learning Research, 3:1439–1461, 2003.
69. D. Wettschereck, D. Aha, and T. Mohri. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artificial Intelligence Review, 11:273–314, 1997.
70. X. Wu, K. Yu, H. Wang, and W. Ding. Online streaming feature selection. In Proceedings of the 27th International Conference on Machine Learning, pages 1159–1166, 2010.
71. Z. Xu, R. Jin, J. Ye, M. Lyu, and I. King. Discriminative semi-supervised feature selection via manifold regularization. In IJCAI’09: Proceedings of the 21th International Joint Conference on Artificial Intelligence, 2009.
72. S. Yang, L. Yuan, Y. Lai, X. Shen, P. Wonka, and J. Ye. Feature grouping and selection over an undirected graph. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 922–930. ACM, 2012.
73. J. Ye and J. Liu. Sparse methods for biomedical data. ACM SIGKDD Explorations Newsletter, 14(1):4–15, 2012.
74. L. Yu and H. Liu. Feature selection for high-dimensional data: A fast correlation-based filter solution. In International Conference on Machine Learning, 20:856, 2003.
75. M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (StatisticalMethodology), 68(1):49–67, 2006.
76. P. Zhao and B. Yu. On model selection consistency of Lasso. Journal of Machine Learning Research, 7:2541–2563, 2006.
77. Z. Zhao and H. Liu. Semi-supervised feature selection via spectral analysis. In Proceedings of SIAM International Conference on Data Mining, 2007.
78. D. Zhou, J. Huang, and B. Schölkopf. Learning from labeled and unlabeled data on a directed graph. In International Conference on Machine Learning, pages 1036–1043. ACM, 2005.
79. J. Zhou, J. Liu, V. Narayan, and J. Ye. Modeling disease progression via fused sparse group Lasso. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1095–1103. ACM, 2012.
80. H. Zou. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476):1418–1429, 2006.
81. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2):301–320, 2005.