Feature selection for high dimensional data: An evolutionary filter approach

Journal of Computer Science

Volumn 7, Issue 5, 2011, Pages 800-820

  Yahya, Anwar Ali ^a   Osman, Addin ^a   Ramli, Abd Rahman ^b   Balola, Adlan ^a  

^a NAJRAN UNIVERSITY   (Saudi Arabia)

^b UNIVERSITY PUTRA MALAYSIA   (Malaysia)

Author keywords

Evaluation function; Feature selection; Filter approach; Genetic algorithm; High dimensional data; Machine Learning (ML); Mutation operator; Natural language processing; Proposed approach; Search algorithm

Indexed keywords

EID: 80053080403     PISSN: 15493636     EISSN: None     Source Type: Journal    
DOI: 10.3844/jcssp.2011.800.820     Document Type: Article

References (94)

1. Allen, J. 1997. Draft of DAMSL: Dialog Act Markup in Several Layers. National Institute informatics. http://ci.nii.ac.jp/naid/10022006078/
2. Almuallim, H. and T.G. Dietterich, 1991. Learning with many irrelevant features. Proceedings of Ninth National Conference on Artificial intelligence, (AI'91), AAAI Press, USA., pp: 547-552. http://portal.acm.org/citation.cfm?id=1865761
3. Andernach, J.A., H.W.L. Doest, H.J.A. Akker, G.F. Hoeven and J. Schaake et al., 1995. Language analysis for dialogue management in a theatre information and booking system. Proceedings of the 5th International Conference on Artificial Intelligence, (ICAI'95), University of Montpellier, Montpellier, France, pp: 351-362. http://eprints.ewi.utwente.nl/9564/
4. Arbor, A., J.H. Michigan, S.B. Hong and S.B. Cho, 2006. Efficient huge-scale feature selection with speciated genetic algorithm. Patt. Recogn. Lett. 27: 143-150. DOI: 10.1016/J.PATREC.2005.07.009
5. Austin, J.L., 1962. How to do Things with Words. 1st Edn., Harvard University Press, Boston, pp: 166.
6. Baldi, P. and S.A. Brunak, 2001. Bioinformatics: The Machine Learning Approach. MIT Press, Cambridge, Mass, ISBN: 10: 9780262025065, pp: 476.
7. Ben-Bassat, M., 1982. Classification Pattern Recognition and Reduction of Dimensionality. Handbook of statistics 2: Classification, Pattern Recognition and Reduction of Dimensionality, Krishnaiah, P.R. and L.N. Kanal (Eds.). North-Holland Pub. Co., Amsterdam, pp: 773-791. ISBN: 044486217X
8. Biesiada, J. and W. Duch, 2005. Feature selection for high-dimensional data: A kolmogorov-smirnov correlation-based filter. Comput. Recogn. Syst., 30: 95-103. DOI: 10.1007/3-540-32390-2_9
9. Bins, J. and B.A. Draper, 2001. Feature selection from huge feature sets. Proceedings of the 8th International Conferences Computer Vision, July, 7-14, IEEE Xplore Press, Vancouver, BC, Canada, pp: 159-165. DOI: 10.1109/ICCV.2001.937619
10. Blum, A. and P. Langley, 1997. Selection of relevant features and examples in machine learning. Art. Intell. 97: 245-271. DOI: 10.1016/S0004-3702(97)00063-5
11. Bunt, H., 1994. Context and dialogue control. Think, 3: 19-31. http://citeseer.ist.psu.edu/viewdoc/summary?doi= 10.1.1.39.1926
12. Caballero, R.E. and P.A. Estevez, 1998. A niching genetic algorithm for selecting features for neural network classifiers. Proceeding of the 8th International Conference of Artificial Neural Networks, Springer-Verlag, New York, pp: 311-316.
13. Cantu-Paz, E., 2004. Feature subset selection, class separability and genetic algorithms. Genetic Evol. Comput. 3102: 959-970. DOI: 10.1007/978-3-540-24854-5_96
14. Das, S., 2001. Filters, wrappers and a boosting-based hybrid for feature selection. Proceedings of the 18th International Conference on Machine Learning, (ICML'01) Morgan Kaufmann Publishers Inc. San Francisco, CA, USA., pp: 74-81. http://portal.acm.org/citation.cfm?id=658297
15. Dash, M. and H. Liu, 1997. Feature selection for classifications: Intelligent data analysis. Int. J., 1: 131-156. DOI: 10.1016/S1088-467x(97)00008-5
16. Dash, M. and H. Liu, 2003. Consistency-based search in feature selection. Art. Intell., 151: 155-176. DOI: 10.1016/S0004-3702(03)00079-1
17. Davidor, Y., 1991a. A Genetic Algorithm Applied to Robot Trajectory Generation. In: Handbook of Genetic Algorithms, Davis, L. (Eds.). Van Nostr and Reinhold, New York, pp: 144-165. ISBN: 0442001738
18. Davidor, Y., 1991b. Genetic Algorithms and Robotics: A Heuristic Strategy for Optimization. World Scientific, Singapore, Teaneck, NJ., ISBN: 9810202172, pp: 164.
19. Duch, W., 2006. Filter Methods. In: Feature Extraction, Foundations and Applications, Guyon, I., S. Gunn, M. Nikravesh and L.A. Zadeh (Eds.). Springer, Berlin, Heidelberg, New York, pp: 89-117. ISBN: 10: 3540354875
20. Eads, D., D. Hill, S. Davis, S.J. Perkins and J. Ma et al., 2002. Genetic algorithms and support vector machines for time series classification. Proceedings of 5th Conference on the Application and Science of Neural Networks, Fuzzy Systems and Evolutionary Computation, SPIE, USA., pp: 74-85. DOI: 10.1117/12.453526
21. Fatourechi, M., G.E. Birch and R.K. Ward, 2007. Application of a hybrid wavelet feature selection method in the design of a self-paced brain interface system. J. Neuro. Eng. Rehabilit., 4: 1-11. DOI: 10.1186/1743-0003-4-11
22. Fishel, M., 2007. Machine learning techniques in dialogue act recognition. Estonian Science Foundation. http://math.ut.ee/~fishel/doc/publ/darec.ery07.pdf
23. Fogel, L.J., A.J. Owens and M.J. Walsh, 1966. Artificial Intelligence Through Simulated Evolution. Wiley, New York, pp: 170.
24. Frohlich, H., O. Chapelle and B. Scho{double acute}lkopf, 2004. Feature selection for support vector machines using genetic algorithms. Int. J. Art. Intell. Tools, 13: 791-800. doi:10.1142/S0218213004001818
25. Gheyas, I.A. and L.S. Smith, 2010. Feature subset selection in large dimensionality domains. Patt. Recogn., 43: 5-13. DOI: 10.1016/J.PATCOG.2009.06.009
26. Goldberg, D.E., 1989. Genetic Algorithms in Search, Optimization and Machine Learning. 1st Edn., Addison-Wesley Public Co., New York, ISBN: 0201157675, pp: 412.
27. Goldberg, D.E., K. Deb and B. Korb, 1990. Messy genetic algorithms revisited: Studiesin in mixed size and scale. Complex Syst., 4: 415-444. http://www.complex-systems.com/pdf/04-4-4.pdf
28. Guo, B., R. I. Damper, S. R. Gunn and J. D. B. Nelson, 2008. A fast separability-based feature selection method for high-dimensional remotely-sensed image classification. Patt. Recogn., 41: 1653-1662. DOI: 10.1016/J.PATCOG.2007.11.007
29. Hall, M., 1999. Correlation-based feature selection for machine learning. PhD Thesis, The University of Waikato. http://www.cs.waikato.ac.nz/~ml/publications/1999/99MH-Thesis.pdf
30. Hall, M.A., 2000. Correlation-Based Feature Selection for Discrete and Numeric Class Machine Learning. Proceeding of the 17th International Conferences on Machine Learning, (ICML'00), Morgan Kaufmann Publishers Inc. San Francisco, CA, USA., pp: 359-366. http://portal.acm.org/citation.cfm?id=657793
31. Harvey, I., 1995. The artificial evolution of adaptive behaviour. D. Phil. Thesis, School of Cognitive and Computing Sciences, University of Sussex.
32. Hastie, T., R. Tibshirani and J.H. Friedman, 2001. The Elements of Statistical Learning. Springer, New York, ISBN: 0387952845, pp: 533.
33. Haupt, R.L. and S.E. Haupt, 2004. Practical Genetic Algorithms. 2nd Edn., Wiley-IEEE, New York, ISBN: 0471455652, pp: 253.
34. Hirschberg, J. and D. Litman, 1993. Empirical studies on the disambiguation of cue phrases. Comput. Linguist., 19: 501-530. http://portal.acm.org/citation.cfm?id=972490
35. Holland, J.H. and J.H. Holland, 1975. Adaptation in Natural and Artificial Systems: An introductory analysis with applications to biology, control, and artificial intelligence. The University of Michigan Press, Ann Arbor, ISBN: 0472084607, pp: 183.
36. Hong, J.H. and S.B. Cho, 2006. Efficient huge-scale feature selection with speciated genetic algorithm. Patt. Recogn. Lett., 27: 143-150. DOI: 10.1016/J.PATREC.2005.07.009
37. John, G.H., R. Kohavi and K. Pfleger, 1994. Irrelevant Feature and the Subset Selection Problem. Proceeding of the 11th International Conference on Machine Learning, Morgan Kaufmann Publishers, San Francisco, CA., pp: 121-129. http://citeseerx.ist.psu.edu/viewdoc/download?doi= 10.1.1.99.6141&rep=rep1&type=pdf
38. Jurafsky, D., 2004. Pragmatics and Computational Linguistics, In: Handbook of Pragmatics, Horn, L. and G. Ward (Eds.). Wiley-Blackwell, Malden, MA., pp: 578-604. ISBN: 10: 0631225471
39. Jurafsky, D., E. Shriberg, B. Fox and T. Curl, 1998. Lexical, prosodic and syntactic cues for dialog acts. Proceedings of ACL/Coling '98 Workshop on Discourse Relations and Discourse Markers, Montreal, Quebec, Canada, pp: 114-120. http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.14.209
40. Kats, H., 2006. Classification of user utterances in question answering dialogues, Master's Thesis, University of Twente, Netherlands.
41. Keizer, S. and R. op den Akker, 2007. Dialogue act recognition under uncertainty using bayesian networks. Nat. Language Eng., 13: 287-316. DOI: 10.1017/S1351324905004067
42. Kelly, J.D. and L. Davis, 1991. Hybridising the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm. In ICGA, pp: 377-383.
43. Kira, K. and L. Rendell, 1992. The feature selection problem: Traditional methods and a new algorithm. Proceedings of the National Conference on Artificial Intelligence, (NCAI'92), John Wiley and Sons Ltd., USA., pp: 1-29. http://direct.bl.uk/bld/PlaceOrder.do?UIN=000587 606&ETOC=EN&from=searchengine
44. Kohavi, R. and G. John, 1997. Wrappers for feature subset selection. Art. Intell., 97: 273-324. DOI: 10.1016/S0004-3702(97)00043-X
45. Koza, J.R., 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, ISBN: 0262111705, pp: 833.
46. Lal, T.N., O. Chapelle, J. Weston and A. Elisseeff, 2006. Embedded Methods. In: Feature Extraction: Foundations and Applications, Guyon, I., S. Gunn, M. Nikravesh and L.A. Zadeh (Eds.). Springer, Berlin, Germany, pp: 137-165. ISBN: 10: 3540354875
47. Langley, P., 1994. Selection of relevant features in machine learning. Proceedings of the AAAI Fall Symposium Relevance, (FSR'94), AAAI Press, USA., pp: 140-144.http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.54.9852
48. Lanzi, P., 1997. Fast feature selection with genetic algorithms: A filter approach. Proceedings of IEEE International Conference on Evolutionary Computation, Apr. 13-16, IEEE Xplore Press, Indianapolis, IN., USA., pp: 537-540. DOI: 10.1109/ICEC.1997.592369
49. Lecocke, M. and K. Hess, 2007. An empirical study of univariate and genetic algorithm-based feature selection in binary classification with microarray data. Can. Inf., 2: 313-327. PMCID: PMC2675488
50. Lee, S.H., H.I. Koo and N.I. Cho, 2010. Image segmentation algorithms based on the machine learning of features. Patt. Recogn. Lett, 31: 2325-2336. DOI: 10.1016/j.patrec.2010.07.004
51. Lesch, S., 2005. Classification of Multidimensional Dialogue Acts using Maximum Entropy DIploma Thesis. Saarl and University, Postfach 151150, D-66041 Saarbrucken, Germany.
52. Li, L., C.R. Weinberg, T.A. Darden and L.G. Pedersen, 2001. Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method. Bioinformatics, 17: 1131-1142. DOI: 10.1093/BIOINFORMATICS/17.12.1131
53. Li, Y., Z. F. Wu and J. M. Liu, 2004. Feature selection for high-dimensional data using two-level filter. Proceedings of the third International Conference on Machine Learning and Cybernetics, Aug. 26-29, IEEE Xplore Press, USA., pp: 1711-1716. DOI: 10.1109/ICMLC.2004.1382051
54. Liu, H. and H. Motoda, 1998. Feature Selection for Knowledge Discovery and Data Mining. Springer, New York, ISBN: 079238198X, pp: 214.
55. Liu, H. and L. Yu, 2005. Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. Knowledge Data Eng., 17: 491-502. DOI: 10.1109/TKDE.2005.66
56. Liu, W., M. Wang and Y. Zhong, 1995. Selecting features with genetic algorithm in handwritten digit recognition. Proceedings of the International IEEE Conference on Evolutionary Computation, Nov. 29-1 Dec., IEEE Xplore Press, Perth, WA, Australia, pp: 396-399. DOI: 10.1109/ICEC.1995.489180
57. Lu, J., T. Zhao and Y. Zhang, 2008. Feature selection based-on genetic algorithm for image annotation. Knowledge-Based Syst., 21: 887-891. DOI: 10.1016/J.KNOSYS.2008.03.051
58. Marquez, L., 2000. Machine Learning and Natural Language Processing. Technical Report LSI-00-45-R, Departament de Llenguatges i Sistemes Informatics, Universitat Politecnica de Catalunya.
59. Mitchell, M., 1996. An Introduction to Genetic Algorithms. MIT Press, Cambridge, ISBN: 0262133164, pp: 205.
60. Morariu, D.I., L.N. Vintan and V. Tresp, 2006. Evolutionary feature selection for text documents using the SVM. Proc. Worl Acad. Sci. Tech., 15: 215-221. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.100.8408&rep=rep1&type=pdf
61. Moser, A. and M.N. Murty, 2000. On the scalability of genetic algorithms to very large-scale feature selection. Real-World Appl. Evolut. Comput., 1803: 309-311. DOI: 10.1007/3-540-45561-2_8
62. Ozdemir, M., M.J. Embrechts, F. Arciniegas, C.M. Breneman and L. Lockwood et al., 2001. Feature selection for in-silico drug design using genetic algorithms and neural networks. IEEE Mountain Workshop on Soft Computing in Industrial Applications, June 25-27, IEEE Xplore Press, Blacksburg, VA, USA., pp: 53-57. 10.1109/SMCIA.2001.936728
63. Punch, W.F., E.D. Goodman, M. Pei, L. Chia-Shun and P. Hovl and et al., 1993. Further research on feature selection and classification using genetic algorithms. Proceedings of the 5th International Conference on Genetic Algorithms, (ICGA'93), Morgan Kaufmann Publishers Inc. San Francisco, CA, USA., 557-564. http://portal.acm.org/citation.cfm?id=657588
64. Saeys, Y., I. Inza and P. Larraaga, 2007. A review of feature selection techniques in bioinformatics. Bioinformatics, 23: 2507-2517. DOI: 10.1093/bioinformatics/btm344
65. Sajn, L. and M. Kukar, 2010. Image processing and machine learning for fully automated probabilistic evaluation of medical images. Comput. Methods Programs Biomed., DOI: 10.1016/J.CMPB.2010.06.021
66. Samuel, K., S. Carberry and K. Vijay-Shanker, 1999. Automatically selecting useful phrases for dialogue act tagging. Proceedings of 4th Conference of the Pacific Association for Computational Linguistics, (PACLING'99), Waterloo, Ontario, Canada, 1-14. http://adsabs.harvard.edu/abs/1999cs........6016S
67. Sanchez-Ferrero, G.V. and J.I. Arribas, 2007. A statistical-genetic algorithm to select the most significant features in mammograms. Proceedings of the 12th International Conference on Computer Analysis of Images and Patterns, (CAIP'07), Springer-Verlag Berlin, Heidelberg, pp: 189-196. http://portal.acm.org/citation.cfm?id=1770931
68. Schtz, M., 1997. Other Operators. Gene Duplication and Deletion. In: Handbook of Evolutionary Computation C3, Bck, T., Z. Fogel and D.B. Michalewicz, Hrsg (Eds.). Institute of Physics Publishing, New York, pp: 8-15. ISBN: 0750303921
69. Searle, J.R., 1975. A Taxonomy of Illocutionary Acts. In: Language, Mind and Knowledge, Minnesota Studies in the Philosophy of Science, Gunderson, K. (Eds.). U of Minnesota Press, Minnesota, pp: 344-369. ISBN: 0816657793
70. Shahla, N., M.E. Basiri, N. Ghasem-Aghaee and M.H. Aghdam, 2009. A novel ACOGA hybrid algorithm for feature selection in protein function prediction. Expert Syst. Appl., 36:12086-12094 DOI: 10.1016/J.ESWA.2009.04.023
71. Siedlecki, W. and J. Sklansky, 1988. On automatic feature selection. Int. J. Patt. Recog. Art. Intell., 2: 197-220. DOI: 10.1142/S0218001488000145
72. Silla, C.N., G.L. Pappa, A.A. Freitas and C.A.A. Kaestner, 2004. Automatic text summarization with genetic algorithm-based attribute selection. Adv. Art. Intell. IBERAMIA, 3315: 305-314. DOI: 10.1007/978-3-540-30498-2_31
73. Smith, S.F., 1980. A learning system based on genetic adaptive algorithms. Doctoral Dissertation Thesis. University of Pittsburgh Pittsburgh, PA, USA. http://portal.acm.org/citation.cfm?id=909835
74. Sridhar, V.K.R., S. Bangalore and S. Narayanan, 2009. Combining lexical, syntactic and prosodic cues for improved online dialog act tagging. Comput. Speech Language, 23: 407422. DOI: 10.1016/j.csl.2008.12.001
75. Vafaie, H. and K. De Jong, 1992. Genetic algorithms as a tool for feature selection in machine learning. Proceedings of the 4th International Conference on Tools with Artificial Intelligence, Nov. 10-13, Arlington, VA. USA., pp: 200-203. DOI: 10.1109/TAI.1992.246402
76. Vafaie, H. and K. De Jong, 1995. Genetic algorithms as a tool for restructuring feature space representations. Proceedings of the Seventh International Conference on Tools with Artificial Intelligence, Nov. 5-8, Henidon, VA. USA., pp: 8-11. DOI: 10.1109/TAI.1995.479372
77. Verbree, A.T., R.J. Rienks and D.K.J. Heylen, 2006. Dialogue act tagging using smart feature selection: Results on multiple corpora. Proceedings of the International IEEE Spoken Language Technology Workshop, Dec. 10-13, IEEE Xplore Press, Palm Beach, pp: 10-13. DOI: 10.1109/SLT.2006.326819
78. Vose, M.D., 1999. The Simple Genetic Algorithm: Foundation and Theory, MIT, Press, Cambridge, Mass. London, ISBN: 026222058X, pp: 251.
79. Webb, N., M. Hepple and Y. Wilks, 2005. Dialogue act classification based on Intra-utterance features. University of Sheffield, UK. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.80.8356&rep=rep1&type=pdf
80. William, H., 2004. Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning, Inform. Sci. Int. J., 163: 103-122. 10.1016/j.ins.2003.03.019
81. Xing, E.P., M.I. Jordan and R.M. Karp, 2001. Feature selection for high-dimensional genomic microarray data. Proceeding of the 8th International Conference on Machine Learning, (ICML'01), Morgan Kaufmann Publishers Inc. San Francisco, CA, USA., pp: 601-608.
82. Yahya, A. A, R. Mahmod, F. Ahmad and N. Sullaiman, 2006. Dynamic Bayesian networks for intention recognition in conversational agent. Proceedings of the 3rd International Conference on Artificial Intelligence in Engineering and Technology (iCAiET2006), University Malaysia Sabah, Sabah, Malaysia.
83. Yahya, A.A., A.R. Ramli and T. Perumal, 2009. Dynamic bayesian networks with ranking-based feature selection for dialogue act recognition. Proceedings of the ATCi Advanced Technology Congress 2009, Nov. 3rd -5th, PWTC, Kuala Lumpur.
84. Yang, A., D. Li and L. Zhu, 2011. An improved genetic algorithm for optimal feature subset selection from multi-character feature set. Expert Syst. Appl. Int. J., 38: 2733-2740. DOI: 10.1016/j.eswa.2010.08.063
85. Yang, J. and V. Honavar, 1998. Feature subset selection using a genetic algorithm. IEEE Intell. Syst., 13: 44-49. DOI: 10.1109/5254.671091
86. Yu, E. and S. Cho, 2003. GA-SVM wrapper approach for feature subset selection in keystroke dynamics identity verification. Proceedings of the IEEE International Joint Conference on Neural Networks, July 20-24, IEEE Xplore Press, pp: 2253-2257. DOI: 10.1109/IJCNN.2003.1223761
87. Yu, L. and H. Liu, 2003. Feature selection for highdimensional data: A fast correlation-based filter solution. Proceedings of the International Conference on Machine Learning (ICML-03), AAAI Press, Washington, DC., pp: 856-863. http://www.aaai.org/Papers/ICML/2003/ICML03-111.pdf
88. Zebulum, R.S., M. Vellasco and M.A. Pacheco, 2000. Variable length representation in evolutionary electronics. Evol. Comput. J. 8: 93-120. DOI: 10.1162/106365600568112
89. Zhang, C.K. and H. Hu, 2005. An effective feature selection scheme via genetic algorithm using mutual information. Fuzzy Syst. Knowl. Discovery, 3614: 73-80. DOI: 10.1007/11540007_10
90. Zhang, L., J. Wang, Y. Zhao and Z. Yang, 2003. A novel hybrid feature selection algorithm: Using ReliefF estimation for GA-Wrapper search. Proceedings of IEEE International Conference on Machine Learning and Cybernetics. Nov. 2-5, IEEE Xplore Press, USA., pp: 380-384. DOI: 10.1109/ICMLC.2003.1264506
91. Zhang, P., B. Verma and K. Kumar, 2004. A neuralgenetic algorithm for feature selection and breast abnormality classification in digital mammography. IEEE, 3: 2303-2308. DOI: 10.1109/IJCNN.2004.1380985
92. Zheng, H. and Y. Zhang, 2008. Feature selection for high dimensional data in astronomy. Adv. Space Res., 41: 1960-1964. DOI: 10.1016/J.ASR.2007.08.033
93. Zhuo, L., J. Zheng, F. Wang, X. Li and B. Ai et al., 2008. A genetic algorithm based wrapper feature selection method for classification of hyperspectral images using support vector machine. Remote Sens. Spatial Inform. Sci., 7147: 397-402. http://www.isprs.org/proceedings/XXXVII/congres s/7_pdf/3_WG-VII-3/32.pdf
94. Zhu, H., L. Jiao and J. Pan, 2006. Multi-population genetic algorithm for feature selection. Adv. Nat. Comput., 4222: 480-487. DOI: 10.1007/11881223_59