Genetic algorithm-based feature set partitioning for classification problems

Pattern Recognition

Volumn 41, Issue 5, 2008, Pages 1676-1700

  Rokach, Lior ^a  

^a BEN GURION UNIVERSITY OF THE NEGEV   (Israel)

Author keywords

Ensemble learning; Feature selection; Feature set partitioning; Genetic algorithm

Indexed keywords

FEATURE EXTRACTION; GENETIC ALGORITHMS; IMAGE CODING; LEARNING SYSTEMS; PROBLEM SOLVING; SET THEORY;

ENSEMBLE LEARNING; FEATURE SET PARTITIONING; VAPNIK-CHERVONENKIS DIMENSIONS;

IMAGE CLASSIFICATION;

EID: 38349121661     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2007.10.013     Document Type: Article

References (58)

1. Jimenez L.O., and Landgrebe D.A. Supervised classification in high-dimensional space: geometrical, statistical, and asymptotical properties of multivariate data. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 28 (1998) 39-54
2. Fukunaga K. Introduction to Statistical Pattern Recognition (1990), Academic Press, San Diego, CA
3. Hwang J., Lay S., and Lippman A. Nonparametric multivariate density estimation: a comparative study. IEEE Trans. Signal Process. 42 (1994) 2795-2810
4. Bellman R. Adaptive Control Processes: A Guided Tour (1961), Princeton University Press, Princeton, NJ
5. Guyon I., Gunn S., Nikravesh M., and Zadeh L. Feature Extraction, Foundations and Applications, Series Studies in Fuzziness and Soft Computing (2006), Physica-Verlag, Springer, Wurzburg, Berlin
6. Opitz D., and Maclin R. Popular ensemble methods: an empirical study. J. Artif. Res. 11 (1999) 169-198
7. Geman S., Bienenstock E., and Doursat R. Neural networks and the bias/variance dilemma. Neural Comput. 4 (1995) 1-58
8. Tumer K., and Ghosh J. Linear and order statistics combiners for pattern classification. In: Sharkey A. (Ed). Combining Artificial Neural Nets (1999), Springer, Berlin 127-162
9. Breiman L. Bagging predictors. Mach. Learn. 24 (1996) 123-140
10. Freund Y., and Schapire R. Experiments with a new boosting algorithm. Machine Learning. Proceedings of the 13th International Conference (1996), Morgan Kaufmann, San Francisco 148-156
11. Tumer K., and Oza C.N. Input decimated ensembles. Pattern Anal. Appl. 6 (2003) 65-77
12. Bryll R., Gutierrez-Osuna R., and Quek F. Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets. Pattern Recognition 36 (2003) 1291-1302
13. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20 (1998) 832-844
14. Bay S. Nearest neighbor classification from multiple feature subsets. Intelligent Data Anal. 3 (1999) 191-209
15. Tsymbal A., and Puuronen S. Ensemble feature selection with the simple Bayesian classification in medical diagnostics. Proceedings of the 15th IEEE Symposium on Computer-Based Medical Systems CBMS'2002, Maribor, Slovenia (2002), IEEE Computer Soc. Press, Silver Spring, MD 225-230
16. Wu Q.X., Bell D., and McGinnity M. Multi-knowledge for decision making. J. Knowl. Inf. Syst. 7 (2005) 246-266
17. Bao Y., and Ishii N. Combining multiple K-nearest neighbor classifiers for text classification by reducts. Proceedings of the 5th International Conference on Discovery Science, Lecture Notes in Computer Science vol. 2534 (2002), Springer, Berlin 340-347
18. Hu Q.H., Yu D.R., and Wang M.Y. Constructing rough decision forests. In: Slezak D., et al. (Ed). RSFDGrC 2005, Lecture Notes in Artificial Intelligence vol. 3642 (2005), Springer, Berlin 147-156
19. Cunningham P., and Carney J. Diversity versus quality in classification ensembles based on feature selection. In: de Mántaras R.L., and Plaza E. (Eds). Proceedings of the ECML 2000, Barcelona, Spain, Lecture Notes in Computer Science vol. 1810 (2000), Springer, Berlin 109-116
20. Zenobi G., and Cunningham P. Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In: De Readt L., and Flach P. (Eds). Proceedings of the ECML 2001, Lecture Notes in Artificial Intelligence vol. 2167 (2001), Springer, Berlin 576-587
21. Tsymbal A., Pechenizkiy M., and Cunningham P. Diversity in search strategies for ensemble feature selection. Inf. Fusion 6 (2005) 83-98
22. Gunter S., and Bunke H. Feature selection algorithms for the generation of multiple classifier systems. Pattern Recognition Lett. 25 (2004) 1323-1336
23. Rokach L. Decomposition methodology for classification tasks-a meta decomposer framework. Pattern Anal. Appl. 9 (2006) 257-271
24. Kusiak A. Decomposition in data mining: an industrial case study. IEEE Trans. Electron. Packag. Manuf. 23 (2000) 345-353
25. F.J. Provost, V. Kolluri, A survey of methods for scaling up inductive learning algorithms, in: Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, 1997.
26. Sharkey A. On combining artificial neural nets. Connection Sci. 8 (1996) 299-313
27. Chan P.K., and Stolfo S.J. On the accuracy of meta-learning for scalable data mining. J. Intelligent Inf. Syst. 8 (1997) 5-28
28. Gama J. A linear-Bayes classifier. In: Monard C. (Ed). Advances on Artificial Intelligence, Proceedings of the SBIA 2000, Lecture Notes in Artificial Intelligence vol. 1952 (2000), Springer, Berlin 269-279
29. K. Tumer, J. Ghosh, Error correlation and error reduction in ensemble classifiers, connection science, Combining Artificial Neural Networks: Ensemble Approaches 8 (1996) 385-404 (special issue).
30. Cherkauer K.J. Human expert-level performance on a scientific image analysis task by a system using combined artificial neural networks. In: Chan P. (Ed). Working Notes, Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms Workshop, 13th National Conference on Artificial Intelligence (1996), AAAI Press, Portland, OR 15-21
31. Chen K., Wang L., and Chi H. Methods of combining multiple classifiers with different features and their applications to text-independent speaker identification. Int. J. Pattern Recognition Artif. Intell. 11 (1997) 417-445
32. Liao Y., and Moody J. Constructing heterogeneous committees via input feature grouping. In: Solla S.A., Leen T.K., and Muller K.-R. (Eds). Advances in Neural Information Processing Systems vol. 12 (2000), MIT Press, Cambridge, MA
33. Rokach L., and Maimon O. Feature set decomposition for decision trees. J. Intelligent Data Anal. 9 (2005) 131-158
34. Freitas A. Evolutionary algorithms for data mining. In: Maimon O., and Rokach L. (Eds). The Data Mining and Knowledge Discovery Handbook (2005), Springer, Berlin 435-467
35. Opitz D. Feature selection for ensembles. Proceedings of the 16th National Conference on Artificial Intelligence (1999), AAAI 379-384
36. Louis S.J., and Rawlins G.J.E. Predicting convergence time for genetic algorithms. In: Whitley L.D. (Ed). Foundations of Genetic Algorithms vol. 2 (1993), Morgan Kaufmann, Los Altos, CA 141-161
37. Rand W.M. Objective criteria for the evaluation of clustering methods. J. Am. Statist. Assoc. 66 (1971) 846-850
38. Sharpe P.K., and P Glover R. Efficient GA based techniques for classification. Appl. Intell. 11 (1999) 277-284
39. Kudo M., and Sklansky J. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33 (2000) 25-41
40. Hsu W.H. Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning. Inf. Sci. 163 (2004) 103-122
41. Opitz D., and Shavlik J. Actively searching for an effective neural-network ensemble. Connection Sci. 8 (1996) 337-353
42. W.H. Hsu, M. Welge, J. Wu, T. Yang, Genetic algorithms for selection and partitioning of features in large-scale data mining problems, in: Proceedings of the Joint AAAI-GECCO Workshop on Data Mining with Evolutionary Algorithms, Orlando, FL, July 1999.
43. Wolpert D.H. The relationship between PAC, the statistical physics framework, the Bayesian framework, and the VC framework. In: Wolpert D.H. (Ed). The Mathematics of Generalization, The SFI Studies in the Sciences of Complexity (1995), Addison-Wesley, Reading, MA 117-214
44. Y. Mansour, D. McAllester, Generalization bounds for decision trees, in: Proceedings of the 13th Annual Conference on Computer Learning Theory, San Francisco, Morgan Kaufmann, Los Altos, CA, 2000, pp. 69-80.
45. Fröhlich H., Chapelle O., and Schölkopf B. Feature selection for support vector machines using genetic algorithms. Int. J. Artif. Intell. Tools 13 (2004) 791-800
46. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, Los Altos, CA
47. Duda R., and Hart P. Pattern Classification and Scene Analysis (1973), Wiley, New-York
48. Almuallim H., and Dietterichm T.G. Learning Boolean concepts in the presence of many irrelevant features. Artif. Intell. 69 (1994) 279-306
49. Schlimmer J.C. Efficiently inducing determinations: a complete and systematic search algorithm that uses optimal pruning. Proceedings of the 1993 International Conference on Machine Learning (1993), Morgan Kaufmann, San Mateo, CA 284-290
50. Langley P., and Sage S. Induction of selective Bayesian classifiers. Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence (1994), Morgan Kaufmann, Seattle, WA 399-406
51. Last M., and Maimon M. A compact and accurate model for classification. IEEE Trans. Knowl. Data Eng. 16 (2004) 203-215
52. Chawla N.V., Hall L.O., Bowyer K.W., and Kegelmeyer W.P. Learning ensembles from bites: a scalable and accurate approach. J. Mach. Learn. Res. 5 (2004) 421-451
53. Merz C.J., and Murphy P.M. UCI repository of Machine Learning Databases (1998), Department of Information and Computer Science, University of California, Irvine, CA
54. Demsar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7 (2006) 1-30
55. Holden S.B., and Niranjan M. On the practical applicability of VC dimension bounds. Neural Comput. 7 (1995) 1265-1288
56. Kearns M., Li M., and Valiant L. Learning Boolean formulas. J. ACM 41 (1994) 1298-1328
57. Wallace C.S. MML inference of predictive trees, graphs and nets. In: Gammerman A. (Ed). Computational Learning and Probabilistic Reasoning (1996), Wiley, New York 43-66
58. Schmitt M. On the complexity of computing and learning with multiplicative neural networks. Neural Comput. 14 (2002) 241-301