A discrete particle swarm optimization method for feature selection in binary classification problems

European Journal of Operational Research

Volumn 206, Issue 3, 2010, Pages 528-539

  Unler, Alper ^a   Murat, Alper ^a  

^a Wayne State University   (United States)

Author keywords

Binary classification; Feature selection; Logistic regression; Metaheuristics; Particle swarm optimization

Indexed keywords

ADAPTIVE FEATURE SELECTION; BINARY CLASSIFICATION; BINARY CLASSIFICATION PROBLEMS; CLASSIFICATION ACCURACY; COMPUTATIONAL PERFORMANCE; DATA SETS; DISCRETE PARTICLE SWARM OPTIMIZATION; DISCRETE PARTICLE SWARM OPTIMIZATION METHOD; FEATURE SELECTION; FEATURE SUBSET; FEATURE SUBSET SELECTION; LOGISTIC REGRESSION; LOGISTIC REGRESSION MODELS; LOGISTIC REGRESSIONS; META HEURISTICS; PSO ALGORITHMS; SCATTER SEARCH ALGORITHM;

FEATURE EXTRACTION; LEARNING ALGORITHMS; LOGISTICS; OPTICAL COMMUNICATION; REGRESSION ANALYSIS; TABU SEARCH;

PARTICLE SWARM OPTIMIZATION (PSO);

EID: 77951139898     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2010.02.032     Document Type: Article

References (57)

1. Alpaydin E. Introduction to Machine Learning (2004), The MIT Press ISBN 0-262-01211-1
2. Amaldi E., and Kann V. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science 209 1-2 (1998) 237-260
3. Baumgartner C., Böhm C., Baumgartner D., Marini G., Weinberger K., Olgemöller B., Liebl B., and Roscher A. Supervised machine learning techniques for the classification of metabolic disorders in newborns. Bioinformatics 20 17 (2004) 2985-2996
4. Blum A.L., and Langley P. Selection of relevant features and examples in machine learning. Artificial Intelligence 97 1-2 (1997) 245-271
5. Bradley P.S., Mangasarian O.L., and Street W.N. Feature selection via mathematical programming. INFORMS Journal on Computing 10 2 (1998) 209-217
6. Casado S. Different metaheuristic strategies to solve the feature selection problem. Pattern Recognition Letters 30 5 (2009) 525-534
7. Clerc M., and Kennedy J. The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6 1 (2002) 58-73
8. Cover T.M., and Van Campenhout J.M. On the possible orderings in the measurement selection problem. IEEE Transactions on Systems, Man and Cybernetics 7 9 (1997) 657-661
9. Dash M., and Liu H. Feature selection for classification. Intelligent Data Analysis 1 (1997) 131-156
10. Debuse J.C.W., and Rayward-Smith V.J. Feature subset selection within a simulated annealing data mining algorithm. Journal of Intelligent Information Systems 9 1 (1997) 57-81
11. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
12. Dietterich T.G. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10 (1998) 1895-1924
13. Escalante H.J., Montes M., and Sucar L.E. Particle swarm model selection. Journal of Machine Learning Research 10 (2009) 405-440
14. Fayyad U., and Uthurusamy R. Evolving data into mining solutions for insights. Communications of the ACM 45 8 (2002) 28-31
15. Gadat S., and Younes L. A stochastic algorithm for feature selection in pattern recognition. Journal of Machine Learning Research 8 (2007) 509-547
16. García F.C., García M., Melián B., Moreno J.A., and Moreno M. Solving feature subset selection problem by a parallel scatter search. European Journal of Operational Research 169 2 (2006) 477-489
17. García-Nieto, J., Alba, E., Jourdan, L., Talbi, E-G., 2007. A comparison of PSO and GA approaches for gene selection and classification of microarray data. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO) 2007, ACM, p. 427.
18. Guyon I., and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 3 (2003) 1157-1182
19. Guyon I., Gunn S., Ben-Hur A., and Dror G. Result analysis of the NIPS 2003 feature selection challenge. Advances in Neural Information Processing Systems vol. 17 (2005), MIT Press, Cambridge, MA 45-552
20. Jain A.K., and Chandrasekaran B. Dimensionality and sample size considerations in pattern recognition in practice. In: Krishnaiah P.R., and Kanal L.N. (Eds). Handbook of Statistics. vol. 2 (1982), North-Holland, Amsterdam 835-855
21. Kennedy, J., Eberhart, R., 1995. Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Network, pp. 1942-1948.
22. Kennedy J., and Eberhart R. A discrete binary version of the particle swarm algorithm. Proceedings of the International Conference on Systems, Man and Cybernetics vol. 5 (1997), IEEE Press 4104-4108
23. Kiang M.Y. A comparative assessment of classification methods. Decision Support Systems 35 4 (2003) 441-454
24. Kohavi R., and John G. Wrappers for feature subset selection. Artificial Intelligence - Special Issue on Relevance 97 1-2 (1997) 273-324
25. Liu H., and Motada H. Feature Extraction, Construction and Selection: A Data Mining Perspective (1998), Kluwer
26. Liu H., and Yu L. Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering 17 4 (2005) 491-502
27. Marill T., and Green D.M. On the effectiveness of receptors in recognition systems. IEEE Transactions on Information Theory 9 1 (1963) 11-17
28. Meiri R., and Zahavi J. Using simulated annealing to optimize the feature selection problem in marketing applications. European Journal of Operational Research 171 3 (2006) 842-858
29. Minka, T., 2003. A Comparison of Numerical Optimizers for Logistic Regression. Technical Report, 758, Carnegie Mellon University.
30. Mitchell T.M. Machine learning and data mining. Communications of the ACM 42 11 (1999) 30-36
31. Narendra P.M., and Fukunaga K. A branch and bound algorithm for feature subset selection. IEEE Transactions on Computers 26 9 (1977) 917-922
32. NIPS, 2003. iN: Neural Information Processing Conference Feature Extraction Workshop. .
33. Olafsson S., and Yang J. Intelligent partitioning for feature selection. INFORMS Journal on Computing 17 3 (2005) 339-355
34. Olafsson S., Li X., and Wu S. Operations research and data mining. European Journal of Operational Research 187 3 (2008) 1429-1448
35. O'Reilly C.A. Variations in decision makers' use of information sources: the impact of quality and accessibility of information. Academy of Management Journal 25 4 (1982) 756-771
36. Pacheco J., Casado S., Núñez L., and Gomez O. Analysis of new variable selection methods for discriminant analysis. Computational Statistics and Data Analysis 51 3 (2006) 1463-1748
37. Pacheco J., Casado S., and Núñez L. Use of VNS and TS in classification: Variable selection and determination of the linear discrimination function coefficients. IMA Journal of Management Mathematics 18 2 (2007) 191-206
38. Pacheco J., Casado S., and Núñez L. A variable selection method based on tabu search for logistic regression models. European Journal of Operational Research 199 2 (2009) 506-511
39. Pudil P., Novovicová J., and Kittler J. Floating search methods in feature selection. Pattern Recognition Letters 15 11 (1994) 1119-1125
40. Raymer M.L., Punch W.F., Goodman E.D., Kuhn L.A., and Jain A.K. Dimensionality reduction using genetic algorithms. IEEE Transactions On Evolutionary Computation 4 2 (2000) 164-171
41. Reinartz T. A unifying view on instance selection. Data Mining and Knowledge Discovery 6 2 (2002) 191-210
42. Reunanen J. Overfitting in making comparisons between variable selection methods. Journal of Machine Learning Research 3 (2003) 1371-1382
43. Robles V., Bielza C., Larrañaga P., González S., and Ohno-Machado L. Optimizing logistic regression coefficients for discrimination and calibration using estimation of distribution algorithms. TOP 16 2 (2008) 345-366
44. Salzberg S.L. On comparing classifiers: pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery 1 3 (1997) 317-328
45. Shi, Y., Eberhart, R., 1998. Parameter selection in particle swarm optimization. In: Proceedings of 7th Annual Conference on Evolutionary Programming, pp. 591-600.
46. Sikora R., and Piramuthu S. Framework for efficient feature selection in genetic algorithm based data mining. European Journal of Operational Research 180 2 (2007) 723-737
47. Somol P., Pudil P., and Kittler J. Fast branch and bound algorithms for optimal feature selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 26 7 (2004) 900-912
48. Stearns, S.D., 1976. On selecting feature for pattern classifiers. In: Proceedings of the 3rd International Conference on Pattern Recognition, Coronado, CA, pp. 71-75.
49. Talbi, E-G., Jourdan, L., Garcia-Nieto, J., Alba, E., 2008. Comparison of population based metaheuristics for feature selection: Application to microarray data classification. In: Proceedings of AICCSA 2008.
50. UCI Machine Learning Repository. Center for Machine Learning and Intelligent Systems. .
51. Vinterbo S., and Ohno-Machado L. A genetic algorithm to select variables in logistic regression: Example in the domain of myocardial infarction. Journal of the American Medical Informatics Association 6 (1999) 984-988
52. Wang X., Yang J., Teng X., Xia W., and Jensen R. Feature selection based on rough sets and particle swarm optimization. Pattern Recognition Letters 28 4 (2007) 459-471
53. Whitney A.W. A direct method of nonparametric measurement selection. IEEE Transactions on Computers 20 9 (1971) 1100-1103
54. Winker P., and Gilli M. Applications of optimization heuristics to estimation and modeling problems. Computational Statistics and Data Analysis 47 2 (2004) 211-223
55. Yang J., and Honavar V. Feature subset selection using a genetic algorithm. Intelligent Systems and their Applications, IEEE 13 2 (1998) 44-49
56. Yang J., and Olafsson S. Optimization-based feature selection with adaptive instance sampling. Computers & Operations Research 33 11 (2006) 3088-3106
57. Zhang H., and Sun G. Feature selection using tabu search method. Pattern Recognition 35 3 (2002) 701-711