A Multiobjective Genetic Programming-Based Ensemble for Simultaneous Feature Selection and Classification

IEEE Transactions on Cybernetics

Volumn 46, Issue 2, 2016, Pages 499-510

  Nag, Kaustuv ^a   Pal, Nikhil R ^b  

^a JADAVPUR UNIVERSITY   (India)

^b INDIAN STATISTICAL INSTITUTE   (India)

Author keywords

Classification; ensemble; feature selection (FS); genetic programming (GP)

Indexed keywords

ALGORITHMS; C (PROGRAMMING LANGUAGE); FEATURE EXTRACTION; GENETIC ALGORITHMS; GENETIC PROGRAMMING; SIZE DETERMINATION; TREES (MATHEMATICS);

BINARY CLASSIFICATION PROBLEMS; CLASSIFICATION METHODS; FEATURE SELECTION AND CLASSIFICATION; FEATURES SELECTION; INTEGRATED ALGORITHM; MULTIOBJECTIVE GENETIC PROGRAMMING; MUTATION OPERATIONS; RELEVANT FEATURES;

CLASSIFICATION (OF INFORMATION);

ALGORITHM; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; DNA MICROARRAY; FEMALE; GENE EXPRESSION PROFILING; GENETICS; HUMAN; MALE; METABOLISM; NEOPLASM; PROCEDURES;

ALGORITHMS; FEMALE; GENE EXPRESSION PROFILING; HUMANS; MALE; MODELS, GENETIC; NEOPLASMS; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS; PATTERN RECOGNITION, AUTOMATED;

EID: 84924560326     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2015.2404806     Document Type: Article

References (90)

1. P. G. Espejo, S. Ventura, and F. Herrera, "A survey on the application of genetic programming to classification, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 40, no. 2, pp. 121-144, Mar. 2010.
2. J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA, USA: MIT Press, 1992.
3. J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs. Cambridge, MA, USA: MIT Press, 1994.
4. J. R. Koza, "Genetic programming as a means for programming computers by natural selection, " Stat. Comput., vol. 4, no. 2, pp. 87-112, 1994.
5. J. R. Koza, F. H. Bennett, III, and O. Stiffelman, Genetic Programming as a Darwinian Invention Machine. Berlin, Germany: Springer, 1999.
6. P. J. Rauss, J. M. Daida, and S. Chaudhary, "Classification of spectral imagery using genetic programming, " in Proc. Genet. Evol. Comput. Conf. (GECCO), Las Vegas, NV, USA, 2000, pp. 726-733.
7. S. A. Stanhope and J. M. Daida, "Genetic programming for automatic target classification and recognition in synthetic aperture radar imagery, " in Evolutionary Programming VII. Berlin, Germany: Springer, 1998, pp. 735-744.
8. I. De Falco, A. D. Cioppa, and E. Tarantino, "Discovering interesting classification rules with genetic programming, " Appl. Soft Comput., vol. 1, no. 4, pp. 257-269, 2002.
9. C. C. Bojarczuk, H. S. Lopes, and A. A. Freitas, "Genetic programming for knowledge discovery in chest-pain diagnosis, " IEEE Eng. Med. Biol. Mag., vol. 19, no. 4, pp. 38-44, Jul./Aug. 2000.
10. H. Gray, R. Maxwell, I. Martinez-Perez, C. Arus, and S. Cerdan, "Genetic programming for classification of brain tumours from nuclear magnetic resonance biopsy spectra, " in Proc. 1st Annu. Conf. Genet. Program., 1996, p. 28-31.
11. D. Hope, E. Munday, and S. Smith, "Evolutionary algorithms in the classification of mammograms, " in Proc. IEEE Symp. Comput. Intell. Image Signal Process. (CIISP), Honolulu, HI, USA, 2007, pp. 258-265.
12. G. Wilson and M. Heywood, "Introducing probabilistic adaptive mapping developmental genetic programming with redundant mappings, " Genet. Program. Evol. Mach., vol. 8, no. 2, pp. 187-220, 2007.
13. J. Kishore, L. M. Patnaik, V. Mani, and V. Agrawal, "Application of genetic programming for multicategory pattern classification, " IEEE Trans. Evol. Comput., vol. 4, no. 3, pp. 242-258, Sep. 2000.
14. J.-Y. Lin, H.-R. Ke, B.-C. Chien, and W.-P. Yang, "Designing a classifier by a layered multi-population genetic programming approach, " Pattern Recognit., vol. 40, no. 8, pp. 2211-2225, 2007.
15. D. P. Muni, N. R. Pal, and J. Das, "A novel approach to design classifiers using genetic programming, " IEEE Trans. Evol. Comput., vol. 8, no. 2, pp. 183-196, Apr. 2004.
16. M. Zhang and P. Wong, "Genetic programming for medical classification: A program simplification approach, " Genet. Program. Evol. Mach., vol. 9, no. 3, pp. 229-255, 2008.
17. M. Zhang and W. Smart, "Using Gaussian distribution to construct fitness functions in genetic programming for multiclass object classification, " Pattern Recognit. Lett., vol. 27, no. 11, pp. 1266-1274, 2006.
18. D. P. Muni, N. R. Pal, and J. Das, "Genetic programming for simultaneous feature selection and classifier design, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 106-117, Feb. 2006.
19. K.-H. Liu and C.-G. Xu, "A genetic programming-based approach to the classification of multiclass microarray datasets, " Bioinformatics, vol. 25, no. 3, pp. 331-337, 2009.
20. T. G. Dietterich, "Ensemble methods in machine learning, " in Multiple Classifier Systems. Berlin, Germany: Springer, 2000, pp. 1-15.
21. L. K. Hansen and P. Salamon, "Neural network ensembles, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 10, pp. 993-1001, Oct. 1990.
22. U. Bhowan, M. Johnston, M. Zhang, and X. Yao, "Evolving diverse ensembles using genetic programming for classification with unbalanced data, " IEEE Trans. Evol. Comput., vol. 17, no. 3, pp. 368-386, Jun. 2013.
23. W. B. Langdon and R. Poli, Foundations of Genetic Programming. Berlin, Germany: Springer, 2002.
24. K. Neshatian, M. Zhang, and P. Andreae, "A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming, " IEEE Trans. Evol. Comput., vol. 16, no. 5, pp. 645-661, Oct. 2012.
25. P. Nordin, "A compiling genetic programming system that directly manipulates the machine code, " Advances in Genetic Programming, vol. 1. Cambridge, MA, USA: MIT Press, 1994, pp. 311-331.
26. M. ONeill and C. Ryan, "Grammatical evolution, " IEEE Trans. Evol. Comput., vol. 5, no. 4, pp. 349-358, Aug. 2001.
27. J. F. Miller and P. Thomson, "Cartesian genetic programming, " in Genetic Programming. Berlin, Germany: Springer, 2000, pp. 121-132.
28. R. Kohavi and G. H. John, "Wrappers for feature subset selection, " Artif. Intell., vol. 97, no. 1, pp. 273-324, 1997.
29. Y.-C. Chen, N. R. Pal, and I.-F. Chung, "An integrated mechanism for feature selection and fuzzy rule extraction for classification, " IEEE Trans. Fuzzy Syst., vol. 20, no. 4, pp. 683-698, Aug. 2012.
30. N. Pal and K. Chintalapudi, "A connectionist system for feature selection, " Neural Parallel Sci. Comput., vol. 5, no. 3, pp. 359-382, 1997.
31. D. Chakraborty and N. R. Pal, "Selecting useful groups of features in a connectionist framework, " IEEE Trans. Neural Netw., vol. 19, no. 3, pp. 381-396, Mar. 2008.
32. I. Guyon, J. Weston, S. Barnhill, and V. Vapnik, "Gene selection for cancer classification using support vector machines, " Mach. Learn., vol. 46, nos. 1-3, pp. 389-422, 2002.
33. H. Jabeen and A. R. Baig, "Review of classification using genetic programming, " Int. J. Eng. Sci. Technol., vol. 2, no. 2, pp. 94-103, 2010.
34. M. Muharram and G. D. Smith, "Evolutionary constructive induction, " IEEE Trans. Knowl. Data Eng., vol. 17, no. 11, pp. 1518-1528, Nov. 2005.
35. K. Neshatian and M. Zhang, "Genetic programming and class-wise orthogonal transformation for dimension reduction in classification problems, " in Genetic Programming. Berlin, Germany: Springer, 2008, pp. 242-253.
36. H. Guo and A. K. Nandi, "Breast cancer diagnosis using genetic programming generated feature, " Pattern Recognit., vol. 39, no. 5, pp. 980-987, 2006.
37. H. Guo, L. B. Jack, and A. K. Nandi, "Feature generation using genetic programming with application to fault classification, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 35, no. 1, pp. 89-99, Feb. 2005.
38. K. Krawiec, "Genetic programming-based construction of features for machine learning and knowledge discovery tasks, " Genet. Program. Evol. Mach., vol. 3, no. 4, pp. 329-343, 2002.
39. X. Tan, B. Bhanu, and Y. Lin, "Fingerprint classification based on learned features, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 35, no. 3, pp. 287-300, Aug. 2005.
40. M. G. Smith and L. Bull, "Genetic programming with a genetic algorithm for feature construction and selection, " Genet. Program. Evol. Mach., vol. 6, no. 3, pp. 265-281, 2005.
41. Y. Lin and B. Bhanu, "Evolutionary feature synthesis for object recognition, " IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 35, no. 2, pp. 156-171, May 2005.
42. J. R. Koza, "Concept formation and decision tree induction using the genetic programming paradigm, " in Parallel Problem Solving from Nature. Berlin, Germany: Springer, 1991, pp. 124-128.
43. G. Folino, C. Pizzuti, and G. Spezzano, "Genetic programming and simulated annealing: A hybrid method to evolve decision trees, " in Genetic Programming. Berlin, Germany: Springer, 2000, pp. 294-303.
44. J. Eggermont, "Evolving fuzzy decision trees with genetic programming and clustering, " in Genetic Programming. Berlin, Germany: Springer, 2002, pp. 71-82.
45. H. Zhao, "A multi-objective genetic programming approach to developing pareto optimal decision trees, " Decis. Support Syst., vol. 43, no. 3, pp. 809-826, 2007.
46. T. M. Khoshgoftaar and Y. Liu, "A multi-objective software quality classification model using genetic programming, " IEEE Trans. Rel., vol. 56, no. 2, pp. 237-245, Jun. 2007.
47. A. Tsakonas, "A comparison of classification accuracy of four genetic programming-evolved intelligent structures, " Inf. Sci., vol. 176, no. 6, pp. 691-724, 2006.
48. S. Sakprasat and M. C. Sinclair, "Classification rule mining for automatic credit approval using genetic programming, " in Proc. IEEE Congr. Evol. Comput. (CEC), Singapore, 2007, pp. 548-555.
49. C. Qing-Shan, Z. De-Fu, W. Li-Jun, and C. Huo-Wang, "A modified genetic programming for behavior scoring problem, " in Proc. IEEE Symp. Comput. Intell. Data Min. (CIDM), Honolulu, HI, USA, 2007, pp. 535-539.
50. E. Carreno, G. Leguizamón, and N. Wagner, "Evolution of classification rules for comprehensible knowledge discovery, " in Proc. IEEE Congr. Evol. Comput. (CEC), Singapore, 2007, pp. 1261-1268.
51. R. R. Mendes, F. B. de Voznika, A. A. Freitas, and J. C. Nievola, "Discovering fuzzy classification rules with genetic programming and co-evolution, " in Principles of Data Mining and Knowledge Discovery. Berlin, Germany: Springer, 2001, pp. 314-325.
52. A. L. Garcia-Almanza and E. P. Tsang, "Evolving decision rules to predict investment opportunities, " Int. J. Autom. Comput., vol. 5, no. 1, pp. 22-31, 2008.
53. U. Bhowan, M. Johnston, and M. Zhang, "Developing new fitness functions in genetic programming for classification with unbalanced data, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 406-421, Apr. 2012.
54. W. Fu, M. Johnston, and M. Zhang, "Low-level feature extraction for edge detection using genetic programming, " IEEE Trans. Cybern., vol. 44, no. 8, pp. 1459-1472, Aug. 2014.
55. P. Wang, K. Tang, T. Weise, E. Tsang, and X. Yao, "Multiobjective genetic programming for maximizing ROC performance, " Neurocomputing, vol. 125, pp. 102-118, Feb. 2014.
56. P. Wang et al., "Convex hull-based multi-objective genetic programming for maximizing receiver operating characteristic performance, " IEEE Trans. Evol. Comput., 2014, Doi: 10.1109/TEVC.2014.2305671.
57. P. A. Whigham and G. Dick, "Implicitly controlling bloat in genetic programming, " IEEE Trans. Evol. Comput., vol. 14, no. 2, pp. 173-190, Apr. 2010.
58. S. Luke and L. Panait, "A comparison of bloat control methods for genetic programming, " Evol. Comput., vol. 14, no. 3, pp. 309-344, 2006.
59. R. Poli, "A simple but theoretically-motivated method to control bloat in genetic programming, " in Genetic Programming. Berlin, Germany: Springer, 2003, pp. 204-217.
60. M. J. Streeter, "The root causes of code growth in genetic programming, " in Genetic Programming. Berlin, Germany: Springer, 2003, pp. 443-454.
61. S. Luke and L. Panait, "Fighting bloat with nonparametric parsimony pressure, " in Parallel Problem Solving from NaturePPSN VII. Berlin, Germany: Springer, 2002, pp. 411-421.
62. F. Fernández-de Vega, G. G. Gil, J. A. G. Pulido, and J. L. Guisado, "Control of bloat in genetic programming by means of the island model, " in Parallel Problem Solving from Nature-PPSN VIII. Berlin, Germany: Springer, 2004, pp. 263-271.
63. F. Fernandez, G. Galeano, and J. Gomez, "Comparing synchronous and asynchronous parallel and distributed genetic programming models, " in Genetic Programming. Berlin, Germany: Springer, 2002, pp. 326-335.
64. M. Castelli, L. Vanneschi, and S. Silva, "Semantic search-based genetic programming and the effect of intron deletion, " IEEE Trans. Cybern., vol. 44, no. 1, pp. 103-113, Jan. 2014.
65. D. Rochat, M. Tomassini, and L. Vanneschi, "Dynamic size populations in distributed genetic programming, " in Genetic Programming. Berlin, Germany: Springer, 2005, pp. 50-61.
66. F. Fernandez, L. Vanneschi, and M. Tomassini, "The effect of plagues in genetic programming: A study of variable-size populations, " in Genetic Programming. Berlin, Germany: Springer, 2003, pp. 317-326.
67. A. Song, Q. Shi, and W. Yin, "Understanding of GP-evolved motion detectors, " IEEE Comput. Intell. Mag., vol. 8, no. 1, pp. 46-55, Feb. 2013.
68. J. Luna, J. Romero, C. Romero, and S. Ventura, "On the use of genetic programming for mining comprehensible rules in subgroup discovery, " IEEE Trans. Cybern., vol. 44, no. 12, pp. 2329-2341, Dec. 2014.
69. S. Pang, D. Kim, and S. Y. Bang, "Membership authentication in the dynamic group by face classification using SVM ensemble, " Pattern Recognit. Lett., vol. 24, no. 1, pp. 215-225, 2003.
70. H. Ishibuchi and T. Yamamoto, "Evolutionary multiobjective optimization for generating an ensemble of fuzzy rule-based classifiers, " in Genetic and Evolutionary Computation-GECCO. Berlin, Germany: Springer, 2003, pp. 1077-1088.
71. P. Yang, Y. H. Yang, B. B. Zhou, and A. Y. Zomaya, "A review of ensemble methods in bioinformatics, " Current Bioinformat., vol. 5, no. 4, pp. 296-308, 2010.
72. K. Nag, T. Pal, and N. Pal, "ASMiGA: An archive-based steadystate micro genetic algorithm, " IEEE Trans. Cybern., vol. 45, no. 1, pp. 40-52, Jan. 2015.
73. R. Enache, "Steady state evolutionary algorithms, " Honda Res. Inst. Europe GmbH, Offenbach am Main, Germany, Tech. Rep. HRI-EU 04-02, Jan. 2004.
74. J.-H. Hong and S.-B. Cho, "Gene boosting for cancer classification based on gene expression profiles, " Pattern Recognit., vol. 42, no. 9, pp. 1761-1767, 2009.
75. K. Nag and T. Pal, "A new archive based steady state genetic algorithm, " in Proc. IEEE Congr. Evol. Comput. (CEC), Brisbane, QLD, Australia, 2012, pp. 1-7.
76. S. Kernell, "Presidential popularity and negative voting: An alternative explanation of the midterm congressional decline of the presidents party, " Amer. Polit. Sci. Rev., vol. 71, no. 1, pp. 44-66, 1977.
77. M. Fang, H. Takauj, S. Kaneko, and H. Watanabe, "Robust optical flow estimation for underwater image, " in Proc. Int. Symp. Optomechatronic Technol. (ISOT), Istanbul, Turkey, 2009, pp. 185-190.
78. M. Fang, H. Takauji, and S. Kaneko, "Rapid computation of robust optical flow by efficient complementary voting, " in Proc. World Autom. Congr. (WAC), Kobe, Japan, 2010, pp. 1-6.
79. M. P. Fiorina and K. A. Shepsle, "Is negative voting an artifact?" Amer. J. Polit. Sci., vol. 33, pp. 423-439, May 1989.
80. J. Durillo, A. Nebro, and E. Alba, "The jMetal framework for multiobjective optimization: Design and architecture, " in Proc. IEEE Congr. Evol. Comput. (CEC), Barcelona, Spain, Jul. 2010, pp. 4138-4325.
81. J. J. Durillo and A. J. Nebro, "jMetal: A java framework for multi-objective optimization, " Adv. Eng. Softw., vol. 42, pp. 760-771, 2011. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0965997811001219
82. Q. Song, J. Ni, and G. Wang, "A fast clustering-based feature subset selection algorithm for high-dimensional data, " IEEE Trans. Knowl. Data Eng., vol. 25, no. 1, pp. 1-14, Jan. 2013.
83. L. Yu and H. Liu, "Feature selection for high-dimensional data: A fast correlation-based filter solution, " in Proc. 20th Int. Conf. Mach. Learn., vol. 20. Washington, DC, USA, 2003, pp. 856-863.
84. L. Yu and H. Liu, "Efficient feature selection via analysis of relevance and redundancy, " J. Mach. Learn. Res., vol. 5, pp. 1205-1224, Dec. 2004.
85. M. A. Hall, "Correlation-based feature selection for machine learning, " Ph.D. dissertation, Dept. Comput. Sci., Univ. Waikato, Hamilton, New Zealand, 1999.
86. M. Robnik-Šikonja and I. Kononenko, "Theoretical and empirical analysis of ReliefF and RreliefF, " Mach. Learn., vol. 53, nos. 1-2, pp. 23-69, 2003.
87. M. Dash, H. Liu, and H. Motoda, "Consistency based feature selection, " in Knowledge Discovery and Data Mining. Current Issues and New Applications. Berlin, Germany: Springer, 2000, pp. 98-109.
88. H. Almuallim and T. G. Dietterich, "Learning boolean concepts in the presence of many irrelevant features, " Artif. Intell., vol. 69, no. 1, pp. 279-305, 1994.
89. M. Friedman, "The use of ranks to avoid the assumption of normality implicit in the analysis of variance, " J. Amer. Stat. Assoc., vol. 32, no. 200, pp. 675-701, 1937.
90. M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings, " Ann. Math. Stat., vol. 11, no. 1, pp. 86-92, 1940.