Diversity in search strategies for ensemble feature selection

Information Fusion

Volumn 6, Issue 1, 2005, Pages 83-98

  Tsymbal, Alexey ^a   Pechenizkiy, Mykola ^b   Cunningham, Pádraig ^a  

^a TRINITY COLLEGE DUBLIN   (Ireland)

^b UNIVERSITY OF JYVÄSKYLÄ   (Finland)

Author keywords

Dynamic integration of classifiers; Ensemble diversity; Ensemble of classifiers; Feature selection; Search strategy

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA MINING; GENETIC ALGORITHMS; INTEGRATION; METRIC SYSTEM; PERSONNEL TRAINING;

DYNAMIC INTEGRATION OF CLASSIFIERS; ENSEMBLE DIVERSITY; ENSEMBLE OF CLASSIFIERS; FEATURE SELECTION; SEARCH STRATEGY;

LEARNING SYSTEMS;

EID: 10444238133     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.inffus.2004.04.003     Document Type: Article

References (35)

1. D.W. Aha, R.L. Bankert, A comparative evaluation of sequential feature selection algorithms, in: D. Fisher, H. Lenz (Eds.), Proceedings of 5th International Workshop on Artificial Intelligence and Statistics, 1995, pp. 1-7.
2. R.E. Banfield, L.O. Hall, K.W. Bowyer, W.P. Kegelmeyer, A new ensemble diversity measure applied to thinning ensembles, in: T. Windeatt, F. Roli (Eds.), Multiple Classifier Systems, 4th International Workshop, MCS 2003, LNCS 2709, Springer, 2003, pp. 306-316.
3. E. Bauer, R. Kohavi, An empirical comparison of voting classification algorithms: bagging, boosting, and variants, Machine Learning 36 (1,2) (1999) 105-139.
4. C.L. Blake, E. Keogh, C.J. Merz, UCI repository of machine learning databases [http://www.ics.uci.edu/~mlearn/MLRepository.html], Dept. of Information and Computer Science, University of California, Irvine, CA, 1999.
5. C. Brodley, T. Lane, Creating and exploiting coverage and diversity, in: Proceedings of AAAI-96 Workshop on Integrating Multiple Learned Models, Portland, OR, 1996, pp. 8-14.
6. J. Cohen, A coefficient of agreement for nominal scales, Educational and Psychological Measurement 20 (1960) 37-46.
7. L.P. Cordelia, P. Foggia, C. Sansone, F. Tortorella, M. Vento, Reliability parameters to improve combination strategies in multiexpert systems, Pattern Analysis and Applications 2 (3) (1999) 205-214.
8. P. Cunningham, J. Carney, Diversity versus quality in classification ensembles based on feature selection, in: R.L. deMántaras, E. Plaza (Eds.), Proceedings of ECML 2000 11th European Conference on Machine Learning, Barcelona, Spain, LNCS 1810, Springer, 2000, pp. 109-116.
9. T.G. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization, Machine Learning 40 (2) (2000) 139-157.
10. T.G. Dietterich, Machine learning research: four current directions, AI Magazine 18 (4) (1997) 97-136.
11. P. Domingos, M. Pazzani, On the optimality of the simple Bayesian classifier under zero-one loss, Machine Learning 29 (2,3) (1997) 103-130.
12. C. Ferri, J. Hernández-Orallo, M.J. Ramírez-Quintana, From ensemble methods to comprehensible models, in: S. Lange, K. Satoh, C.H. Smith (Eds.), Proceedings of DS 2002, 5th International Conference on Discovery Science, LNCS 2534, Springer, 2002, pp. 165-177.
13. G. Giacinto, F. Roli, Methods for dynamic classifier selection, in: Proceedings of ICIAP '99, 10th International Conference on Image Analysis and Processing, IEEE CS Press, 1999, pp. 659-664.
14. T.K. Ho, The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (8) (1998) 832-844.
15. R. Kohavi, Wrappers for performance enhancement and oblivious decision graphs, Dept. of Computer Science, Stanford University, Stanford, USA, PhD Thesis, 1995.
16. R. Kohavi, D. Sommerfield, J. Dougherty, Data mining using MLC++: a machine learning library in C++, Tools with Artificial Intelligence, IEEE CS Press, 1996, pp. 234-245.
17. M. Kudo, J. Sklansky, Comparison of algorithms that select features for pattern classifiers, Pattern Recognition 33 (1) (2000) 24-41.
18. L.I. Kuncheva, Genetic algorithm for feature selection for parallel classifiers, Information Processing Letters 46 (1993) 163-168.
19. L.I Kuncheva, L.C. Jain, Designing classifier fusion systems by genetic algorithms, IEEE Transactions on Evolutionary Computation 4 (4) (2000) 327-336.
20. L.I. Kuncheva, C.J. Whitaker, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Machine Learning 51 (2) (2003) 181-207.
21. D.D. Margineantu, T.G. Dietterich, Pruning adaptive boosting, in: Proc. 14th Int. Conf. on Machine Learning, Morgan Kaufmann, 1997, pp. 211-218.
22. L.S. Oliveira, R. Sabourin, F. Bortolozzi, C.Y. Suen, A methodology for feature selection using multi-objective genetic algorithms for handwritten digit string recognition, International Journal of Pattern Recognition and Artificial Intelligence 17 (6) (2003) 903-930.
23. D. Opitz, Feature selection for ensembles, in: Proc. 16th National Conf. on Artificial Intelligence, AAAI Press, 1999, pp. 379-384.
24. N. Oza, K. Turner, Dimensionality reduction through classifier ensembles, Computational Sciences Division, NASA Ames Research Center, Technical report NASA-ARC-IC-1999-126, 1999.
25. S. Puuronen, V. Terziyan, A. Tsymbal, A dynamic integration algorithm for an ensemble of classifiers, in: Z.W. Ras, A. Skowron (Eds.), Foundations of Intelligent Systems: 11th Int. Symp. ISMIS'99, Warsaw, Poland, LNAI 1609, Springer, 1999, pp. 592-600.
26. J. Reunanen, Overfitting in making comparisons between variable selection methods (Special Issue on Variable and Feature Selection), Journal of Machine Learning Research 3 (2003) 1371-1382.
27. S.L. Salzberg, On comparing classifiers: a critique of current research and methods, Data Mining and Knowledge Discovery 1 (1999). 1-12.
28. C. Schaffer, Selecting a classification method by cross-validation, Machine Learning 13 (1993) 135-143.
29. A.J.C. Sharkey, On combining artificial neural nets, Connection Science, Special Issue on Combining Artificial Neural Networks: Ensemble Approaches 8 (3,4) (1996) 299-314.
30. D.B. Skalak, The sources of increased accuracy for two proposed boosting algorithms, in: AAAI-96 Workshop on Integrating Multiple Models for Improving and Scaling Machine Learning Algorithms (in conjunction with AAAI-96), Portland, Oregon, USA, 1996, pp. 120-125.
31. M. Skurichina, R.P.W. Duin, Bagging and the random subspace method for redundant feature spaces, in: J. Kittler, F. Roli (Eds.), Proceedings of 2nd International Workshop on Multiple Classifier Systems MCS 2001, Cambridge, UK, 2001, pp. 1-10.
32. A. Tsymbal, P. Cunningham, M. Pechinizkiy, S. Puuronen, Search strategies for ensemble feature selection in medical diagnostics, in: M. Krol, S. Mitra, D.J. Lee (Eds.), Proceedings of 16th IEEE Symposium on Computer-Based Medical Systems CBMS'2003, The Mount Sinai School of Medicine, New York, NY, IEEE CS Press, 2003, pp. 124-129.
33. A. Tsymbal, S. Puuronen, D. Patterson, Ensemble feature selection with the simple Bayesian classification, Information Fusion 4 (2) (2003) 87-100.
34. A. Tsymbal, S. Puuronen, I. Skrypnyk, Ensemble feature selection with dynamic integration of classifiers, in: Int. ICSC Congress on Computational Intelligence Methods and Applications CIMA'2001, Bangor, Wales, UK, 2001, pp. 558-564.
35. G. Zenobi, P. Cunningham, Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error, in: L.D. Raedt, P.A. Flach (Eds.), Proc. ECML 2001 12th European Conf. On Machine Learning, LNCS 2167, Springer, 2001, pp. 576-587.