IFS-CoCo: Instance and feature selection based on cooperative coevolution with nearest neighbor rule

Pattern Recognition

Volumn 43, Issue 6, 2010, Pages 2082-2105

  Derrac, Joaquín ^a   García, Salvador ^b   Herrera, Francisco ^a  

^a UNIVERSITY OF GRANADA   (Spain)

^b UNIVERSITY OF JAÉN   (Spain)

Author keywords

Cooperative coevolution; Evolutionary algorithms; Feature selection; Instance selection; Nearest neighbor

Indexed keywords

CLASSIFICATION TASKS; COMPUTATIONAL PROBLEM; COOPERATIVE COEVOLUTION; EVOLUTIONARY APPROACH; EVOLUTIONARY MODELS; FEATURE SELECTION; INSTANCE SELECTION; NEAREST NEIGHBOR CLASSIFICATION; NEAREST NEIGHBOR RULE; NEAREST NEIGHBORS; NON-PARAMETRIC STATISTICAL TESTS;

DATA REDUCTION; STATISTICAL TESTS; STATISTICS;

EVOLUTIONARY ALGORITHMS;

EID: 76749096459     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2009.12.012     Document Type: Article

References (92)

1. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Tools and Techniques. second ed. (2005), Morgan Kaufmann, Los Altos, CA
2. X. Wu, V. Kumar (Eds.), The Top Ten Algorithms in Data Mining, Chapman & Hall, CRC, London, Boca Raton, 2009.
3. Pyle D. Data Preparation for Data Mining (1999), Morgan Kaufmann, San Francisco
4. Wang-Manoranjan S., and Xu D.C. Efficient data reduction in multimedia data. Applied Intelligence 25 (2006) 359-374
5. Kolesnikov A., and Frantib P. Data reduction of large vector graphics. Pattern Recognition 38 (2005) 381-394
6. Kim S.W., and Oomenn B.J. On using prototype reduction schemes to optimize dissimilarity-based classification. Pattern Recognition 40 11 (2007) 2946-2957
7. Cano J.R., García S., and Herrera F. Subgroup discovery in large size data sets preprocessed using stratified instance selection for increasing the presence of minority classes. Pattern Recognition Letters 29 (2008) 2156-2164
8. S.W. Kim, B.J. Oomenn, On using prototype reduction schemes to enhance the computation of volume-based inter-class overlap measures, Pattern Recognition 42(11) (2009) 2695-2704.
9. Cover T.M., and Hart P.E. Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13 (1967) 21-27
10. Perner P. Prototype-based classification. Applied Intelligence 28 (2008) 238-246
11. H. Liu, H. Motoda (Eds.), Instance Selection and Construction for Data Mining, Springer, New York, 2001.
12. H. Liu, H. Motoda (Eds.), Computational Methods of Feature Selection, Chapman & Hall, CRC, London, Boca Raton, 2007.
13. Cano J.R., Herrera F., and Lozano M. Using evolutionary algorithms as instance selection for data reduction in KDD: An experimental study. IEEE Transactions on Evolutionary Computation 7 (2003) 561-575
14. Eiben A.E., and Smith J.E. Introduction to Evolutionary Computing (2003), Springer, Berlin
15. García S., Cano J.R., and Herrera F. A memetic algorithm for evolutionary prototype selection: a scaling up approach. Pattern Recognition 41 8 (2008) 2693-2709
16. Kuncheva L.I. Editing for the k-nearest neighbors rule by a genetic algorithm. Pattern Recognition Letters 16 (1995) 809-814
17. Inza I., Larraaga P., and Sierra B. Feature subset selection by Bayesian networks: a comparison with genetic and sequential algorithms. International Journal of Approximate Reasoning 27 (2001) 143-164
18. Oh I., Lee J., and Moon B. Hybrid Genetic Algorithms for Feature Selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 (2004) 1424-1437
19. D. Whitley, C. Guerra-Salcedo, Genetic search for feature subset selection: a comparison between CHC and GENESIS, in: Proceedings of the Third Annual Conference on Genetic Programming, Wisconsin, 1998, pp. 504-509.
20. Freitas A.A. Data Mining and Knowledge Discovery with Evolutionary Algorithms (2002), Springer, New York
21. Ghosh A., and Jain L.C. Evolutionary Computation in Data Mining (2005), Springer, Berlin
22. Bandyopadhyay S., and Santanu S. A genetic approach for efficient outlier detection in projected space. Pattern Recognition 41 (2008) 1338-1349
23. Potter M.A., and De Jong K.A. Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evolutionary Computation 8 (2000) 1-29
24. Wolpert D.H., and Macready W.G. Coevolutionary free lunches. IEEE Transactions on Evolutionary Computation 9 (2005) 721-735
25. Garcia-Pedrajas N., and Ortiz-Boyer D. A cooperative constructive method for neural networks for pattern recognition. Pattern Recognition 40 1 (2007) 80-98
26. Wiegand R.P., and Jansen T. The cooperative coevolutionary (1 + 1) EA. Evolutionary Computation 12 (2004) 405-434
27. Wilcoxon F. Individual comparisons by rankings methods. Biometrics 1 (1945) 80-83
28. Liu H., and Motoda H. On issues of instance selection. Data Mining and Knowledge Discovery 6 2 (2002) 115-130
29. Cano J.R., Herrera F., and Lozano M. Evolutionary stratified training set selection for extracting classification rules with trade-off precision-interpretability. Data and Knowledge Engineering 60 (2007) 90-100
30. Kim K. Artificial neural networks with evolutionary instance selection for financial forecasting. Expert Systems with Applications 30 (2006) 519-526
31. Wilson D.R., and Martinez T.R. Reduction techniques for instance-based learning algorithms. Machine Learning 38 (2000) 257-286
32. Hart P.E. The condensed nearest neighbor rule. IEEE Transactions on Information Theory 18 (1968) 515-516
33. Wilson D.L. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man and Cybernetics 3 (1972) 408-421
34. Marchiori E. Hit miss networks with applications to instance selection. Journal of Machine Learning Research 9 (2008) 997-1017
35. J.A. Olvera-López, J.A. Carrasco-Ochoa, J.F. Martínez-Trinidad, A new fast prototype selection method based on clustering, Pattern Analysis and Applications (2009), in press, doi:10.1007/s10044-008-0142-x.
36. Bezdek J.C., and Kuncheva L.I. Nearest prototype classifier designs: an experimental study. International Journal of Intelligent Systems 16 (2001) 1445-1473
37. N. Jankowski, M. Grochowski, Comparison of instances selection algorithms I. Algorithms survey, in: Lecture Notes in Computer Science, vol. 3070, Springer, Berlin, 2004, pp. 598-603.
38. Kim S.W., and Oomenn B.J. A brief taxonomy and ranking of creative prototype reduction schemes. Pattern Analysis and Applications 6 (2003) 232-244
39. Kohavi R., and John G. Wrappers for feature selection. Artificial Intelligence 97 (1997) 273-324
40. Guyon I., and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 3 (2003) 1157-1182
41. Saeys Y., Inza I., and Larranaga P. A review of feature selection techniques in bioinformatics. Bioinformatics 19 (2007) 2507-2517
42. Liu H., and Motoda H. Feature Selection for Knowledge Discovery and Data Mining (1998), Springer, New York
43. Li Y., and Lu B.L. Feature selection based on loss-margin of nearest neighbor classification. Pattern Recognition 42 9 (2009) 1914-1921
44. Stracuzzi D.J., and Utgoff P.E. Randomized variable elimination. Journal of Machine Learning Research 5 (2004) 1331-1362
45. Shie J., and Chen S. Feature subset selection based on fuzzy entropy measures for handling classification problems. Applied Intelligence 28 (2008) 69-82
46. Liu H., and Yu L. Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering 17 3 (2005) 1-12
47. Kuncheva L.I., and Jain L.C. Nearest neighbor classifier: simultaneous editing and descriptor selection. Pattern Recognition Letters 20 (1999) 1149-1156
48. H. Ishibuchi, T. Nakashima, M. Nii, Genetic-algorithm-based instance and feature selection, in: H. Liu, H. Motoda (Eds.), Instance Selection and Construction for Data Mining, 2001, pp. 95-112.
49. J. Teixeira, R.A. Ferreira, G.A. Lima, A novel approach for integrating feature and instance selection, in: International Conference on Machine Learning and Cybernetics, Kunming, 2008, pp. 374-379.
50. Kirkpatrick S., Gelatt C.D., and Vecchi M.P. Optimization by simulated annealing. Science 4598 (1983) 671-680
51. Ahn H., and Kim K. Bankruptcy prediction modeling with hybrid case-based reasoning and genetic algorithms approach. Applied Soft Computing 9 (2009) 599-607
52. Eshelman L.J. The CHC adaptative search algorithm: how to have safe search when engaging in nontraditional genetic recombination. In: Rawlins G.J.E. (Ed). Foundations of Genetic Algorithms (1991) 265-283
53. Gil-Pita R., and Yao X. Evolving edited k-nearest neighbor classifiers. International Journal of Neural Systems 18 6 (2008) 1-9
54. Ishibuchi H., and Nakashima T. Evolution of reference sets in nearest neighbor classification. Lecture Notes in Computer Science vol. 1585 (1999) 82-89
55. B. Sierra, E. Lazkano, I. Inza, M. Merino, P. Larraaga, J. Quiroga, Prototype selection and feature subset selection by estimation of distribution algorithms. A case study in the survival of cirrhotic patients treated with TIPS, in: Lecture Notes in Artificial Intelligence, vol. 2101, Springer, Berlin, 2001, pp. 20-29.
56. Bala J., De Jong K.A., Huang J., Vafaie H., and Wechsler H. Using learning to facilitate the evolution of features for recognizing visual concepts. Evolutionary Computation 4 3 (1997) 297-311
57. Casillas J., Cordon O., Del Jesus M.J., and Herrera F. Genetic feature selection in a fuzzy rule-based classification system learning process for high-dimensional problems. Information Sciences 136 (2001) 135-157
58. Gonzalez A., and Perez R. Selection of relevant features in a fuzzy genetic learning algorithm. IEEE Transactions on Systems, Man and Cybernetics 31 3 (2001) 417-425
59. Rokach L. Genetic algorithm-based feature set partitioning for classification problems. Pattern Recognition 41 (2008) 1676-1700
60. Siedlecki W., and Sklansky J. A note on genetic algorithm for large-scale feature selection. Pattern Recognition Letters 10 (1989) 335-347
61. Wang C., and Huang Y. Evolutionary-based feature selection approaches with new criteria for data mining: a case study of credit approval data. Expert Systems with Applications 36 (2009) 5900-5908
62. Zhang P., Verma B., and Kumar K. Neural vs. statistical classifier in conjunction with genetic algorithm based feature selection. Pattern Recognition Letters 26 7 (2005) 909-919
63. Ho S., Liu C., and Liu S. Design of an optimal nearest neighbor classifier using an intelligent genetic algorithm. Pattern Recognition Letters 23 (2002) 1495-1503
64. Ros F., Guillaume S., Pintore M., and Chretien J.R. Hybrid genetic algorithm for dual selection. Pattern Analysis and Applications 11 (2008) 179-198
65. Price P.W. Biological Evolution (1998), Saunders College Publishing
66. R.P. Wiegand, An analysis of cooperative coevolutionary algorithms, Ph.D. Thesis, George Mason University, Fairfax, Virginia, 2003.
67. Rosin C.D., and Belew R.K. New Methods for competitive coevolution. Evolutionary Computation 15 (1997) 1-29
68. L. Panait, R.P. Wiegand, S. Luke, Improving coevolutionary search for optimal multiagent behaviors, in: International Joint Conferences on Artificial Intelligence, Acapulco, 2003, pp. 653-658.
69. L. Panait, S. Luke, J.F Harrison, Archive-based cooperative coevolutionary algorithms, in: Genetic and Evolutionary Computation Conference, GECCO'06, Seattle, 2006, pp. 345-352.
70. R.P. Wiegand, J. Sarma, Spatial embedding and loss of gradient in cooperative coevolutionary algorithms, Parallel Problem Solving from Nature VIII, Birmingham, 2004, pp. 912-921.
71. E. Popovici, K.A. De Jong, Sequential versus parallel cooperative coevolutionary algorithms for optimization, IEEE Congress on Evolutionary Computation, Vancouver, 2006, pp. 1610-1617.
72. R.P. Wiegand, L. Liles, K.A. De Jong, An empirical analysis of collaboration methods in cooperative coevolutionary algorithms, in: Genetic and Evolutionary Computation Conference, GECCO'01, San Francisco, 2001, pp. 1235-1242.
73. Hofbauer J., and Sigmund K. Evolutionary games and population dynamics (1998), Cambridge University Press, Cambridge
74. Wolpert D.H., and Macready W.G. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1 1 (1997) 67-82
75. C.S. Travis, D.R. Tauritz, A no-free-lunch framework for coevolution, in: Genetic and Evolutionary Computation Conference, GECCO'08, Atlanta, 2008, pp. 371-378.
76. A. Asuncion, D.J. Newman, UCI repository of machine learning databases, 2007, URL: 〈 http://www.ics.uci.edu/∼mlearn/MLRepository.html〉.
77. Alpaydin E. Introduction to Machine Learning (2004), MIT Press, Cambridge, MA
78. Lim T.S., Loh W.Y., and Shih Y.S. A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Machine Learning 40 3 (2000) 203-228
79. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 1 (1960) 37-46
80. Ben-David A. A lot of randomness is hiding in accuracy. Engineering Applications of Artificial Intelligence 20 7 (2007) 875-885
81. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
82. García S., and Herrera F. An extension on "Statistical comparisons of classifiers over multiple data sets" for all pairwise comparisons. Journal of Machine Learning Research 9 (2008) 2677-2694
83. Sheskin D.J. Handbook of Parametric and Nonparametric Statistical Procedures (1997), CRC Press, Boca Raton
84. Zar J.H. Biostatistical Analysis (1999), Prentice-Hall, Englewood Cliffs, London
85. Brighton H., and Mellish C. Advances in instance selection for instance-based learning algorithms. Data Mining and Knowledge Discovery 6 2 (2002) 153-172
86. Kira K., and Rendell L. A practical approach to feature selection. In: Sleeman P., and Edwards P. (Eds). Proceedings of the Ninth International Conference on Machine Learning (ICML-92) (1992), Morgan Kaufmann, Los Altos, CA 249-256
87. H. Liu, R. Setiono, Feature selection and classification: a probabilistic wrapper approach, in: Ninth International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, Fukuoka, Japan, 1996, pp. 419-424.
88. Ho T.K., and Basu M. Complexity measures of supervised classification problems. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 3 (2002) 289-300
89. Singh S. Multiresolution estimates of classification complexity. IEEE Transactions on Pattern Analysis and Machine Intelligence 25 12 (2003) 1534-1539
90. Cano J.R., Herrera F., and Lozano M. Stratification for scaling up evolutionary prototype selection. Pattern Recognition Letters 26 (2005) 953-963
91. Haro-García A., and García-Pedrajas N. A divide-and-conquer recursive approach for scaling up instance selection algorithms. Data Mining and Knowledge Discovery 18 (2009) 392-418
92. Paredes R., and Vidal E. Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Transactions on Pattern Analysis and Machine Intelligence 28 7 (2006) 1100-1110