Creating diverse nearest-neighbour ensembles using simultaneous metaheuristic feature selection

Pattern Recognition Letters

Volumn 31, Issue 11, 2010, Pages 1470-1480

  Tahir, Muhammad Atif ^a,b   Smith, Jim ^a,b  

^a UNIVERSITY OF SURREY   (United Kingdom)

^b UNIVERSITY OF THE WEST OF ENGLAND   (United Kingdom)

Author keywords

1NN classifier; Ensemble classifiers; Feature selection; Tabu Search

Indexed keywords

BENCHMARK DATASETS; CLASSIFICATION ACCURACY; CLASSIFIER ENSEMBLES; COMPUTING DISTANCES; DATA MINING PROBLEMS; DISTANCE FUNCTIONS; DISTANCE METRICS; ENSEMBLE CLASSIFIERS; ENSEMBLE METHODS; ENSEMBLE TECHNIQUES; ENSEMBLES OF CLASSIFIERS; ERROR RATE; EXPLORATORY DATA ANALYSIS; FEATURE SELECTION; FEATURE SETS; FEATURE VECTORS; INDIVIDUAL CLASSIFIERS; LOCAL SEARCH; MACHINE-VISION; METAHEURISTIC; NEAREST-NEIGHBOUR; PERFORMANCE IMPROVEMENTS; REAL-WORLD; SOLUTION SPACE; UCI MACHINE LEARNING REPOSITORY;

BENCHMARKING; CLASSIFIERS; DATA REDUCTION; FACE RECOGNITION; TABU SEARCH; VECTORS;

FEATURE EXTRACTION;

EID: 77953138030     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2010.01.030     Document Type: Article

References (46)

1. Alpaydin E. Voting over multiple condensed nearest neighbors. Artif. Intell. Rev. 11 (1997) 115-132
2. Amaldi E., and Kann V. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theor. Comput. Sci. 209 (1998) 237-260
3. Bao, Y., Ishii, N., Du, X., 2004. Cobmining multiple k-nearest neighbor classifiers using different distance functions. In: Lecture Notes in Computer Science (LNCS 3177), 5th Internat. Conf. on Intelligent Data Engineering and Automated Learning (IDEAL 2004), Exeter, UK.
4. Bay, S.D., 1998. Combining nearest neighbor classifiers through multiple feature subsets. In: Proc. 15th Internat. Conf. on Machine Learning, pp. 37-45.
5. Blake, C., Keogh, E., Merz, C.J., xxxx. UCI Repository of Machine Learning Databases. University of California, Irvine.
6. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
7. Breiman L. Random forests. Machine Learning 45 1 (2001) 5-32
8. Cover T.M., and Hart P.E. Nearest neighbor pattern classification. IEEE Trans. Inform. Theory 13 1 (1967) 21-27
9. Cunningham P., and Carney J. Diversity versus quality in classification ensembles based on feature selection. Proc. 11th European Conf. on Machine Learning, Barcelona, Spain, LNCS vol. 1810 (2000), Springer 109-116
10. Davies S., and Russell S. NP-completeness of searches for smallest possible feature sets. Proc. AAAI Fall Symposium on Relevance (1994), AAAI Press 37-39
11. Domeniconi C., Peng J., and Gunopulos D. Locally adaptive metric nearest-neighbor classification. IEEE Trans. Pattern Anal. Machine Intell. 24 9 (2002) 1281-1285
12. Duda R., and Hart P. Pattern Classification and Scene Analysis (1973), Wiley, New York
13. Freund, Y., Schapire, R.E., 1996. Experiments with a new boosting algorithm. In: Proc. Internat. Conf. on Machine Learning, pp. 148-156.
14. Glover F. Tabu search I. ORSA J. Comput. 1 3 (1989) 190-206
15. Glover F. Tabu search II. ORSA J. Comput. 2 1 (1990) 4-32
16. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Machine Intell. 20 8 (1998) 832-844
17. Jain A.K., and Zongker D. Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Machine Intell. 19 2 (1997) 153-158
18. Jain A.K., Duin R.P.W., and Mao J. Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Machine Intell. 22 1 (2000) 4-37
19. Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. Proc. 14th Internat. Joint Conf. on Artificial Intelligence vol. 2(12) (1995), Morgan Kaufmann, San Mateo 1137-1143
20. Kohonen K. Self-organizing Maps (1997), Springer-Verlag New York, Inc., Secaucus, NJ
21. Korycinski, D., Crawford, M., Barnes, J.W., Ghosh, J., 2003. Adaptive feature selection for hyperspectral data analysis using a binary hierarchical classifier and Tabu search. In: Proc. IEEE Internat. Geoscience and Remote Sensing Symposium, IGARSS.
22. Kudo M., and Sklansky J. Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33 (2000) 25-41
23. Kuncheva L.I. Diversity in multiple classifier systems (editorial). Inform. Fusion 6 1 (2005) 3-4
24. Michie D., Spiegelhalter D.J., and Taylor C.C. Machine Learning, Neural and Statistical Classification (1994), Ellis Horwood
25. Okun, O. Proosalut, H., 2005. Multiple views in ensembles of nearest neihbor classifiers. In: Proc. ICML Workshop on Learning with Multiple Views, Bonn, Germany, pp. 51-58.
26. Paredes R., and Vidal E. Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans. Pattern Anal. Machine Intell. 28 7 (2006) 1100-1110
27. Pudil P., Novovicova J., and Kittler J. Floating search methods in feature selection. Pattern Recognition Lett. 15 (1994) 1119-1125
28. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann
29. Raudys S., and Jain A. Small sample effects in statistical pattern recognition: Recommendations for practitioners. IEEE Trans. Pattern Anal. Machine Intell. 13 3 (1991) 252-264
30. Raymer M.L., et al. Dimensionality reduction using genetic algorithms. IEEE Trans. Evolut. Comput. 4 2 (2000) 164-171
31. Sait S.M., and Youssef H. General Iterative Algorithms for Combinatorial Optimization (1999), IEEE Computer Society
32. Sannen D., et al. An on-line interactive self-adaptive image classification framework. Proc. Sixth Internat. Conf. Computer Vision Systems, LNCS vol. 5008/2008 (2008), Springer 171-180
33. Serpico S.B., and Bruzzone L. A new search algorithm for feature selection in hyperspectral remote sensing images. IEEE Trans. Geosci. Remote Sens. 39 7 (2001) 1360-1367
34. Siedlecki W., and Sklansy J. A note on genetic algorithms for large-scale feature selection. Pattern Recognition Lett. 10 11 (1989) 335-347
35. Smith, J.E., Fogarty, T.C., Johnson, I.R., 1994. Genetic feature selection for clustering and classification. In: Proc. IEE Colloquium on Genetic Algorithms in Image Processing and Vision, IEE Digest 1994/193.
36. Tahir M.A., and Bouridane A. Novel round-robin tabu search algorithm for prostate cancer classification and diagnosis using multispectral imagery. IEEE Trans. Inform. Technol. Biomed. 10 4 (2006) 782-793
37. Tahir, M.A., Smith, J., 2006. Improving nearest neighbor classifier using tabu search and ensemble distance metrics. In: Proc. IEEE Internat. Conf. Data Mining (ICDM).
38. Tahir M.A., Bouridane A., and Kurugollu F. Simultaneous feature selection and feature weighting using hybrid tabu search/K-nearest neighbor classifier. Pattern Recognition Lett. (2007) 28
39. Tsymbal A., Puuronen S., and Patterson D.W. Ensemble feature selection with the simple Bayesian classification. Inform. Fusion 4 2 (2003) 87-100
40. Tsymbal, A., Pechenizkiy, M., Cunningham, P., 2004. Diversity in random subspacing ensembles. In: Lecture Notes in Computer Science (LNCS 3181), 6th Internat. Conf. on Data Warehousing and Knowledge Discovery, Zaragoza, Spain.
41. Wang J., Neskovic P., and Cooper L. Improving nearest neighbor rule with a simple adaptive distance measure. Pattern Recognition Lett. 28 (2007) 207-213
42. Whitney A.W. A direct method of nonparametric measurement selection. IEEE Trans. Comput. 20 9 (1971) 1100-1103
43. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Tools and Techniques. second ed. (2005), Morgan Kaufmann, San Francisco
44. Wolpert D.H. Stacked generalization. Neural Networks 5 (1992) 241-259
45. Yu S., Backer S.D., and Scheunders P. Genetic feature selection combined with composite fuzzy nearest neighbor classifiers for hyperspectral satellite imagery. Pattern Recognition Lett. 23 (2002) 183-190
46. Zhang H., and Sun G. Feature selection using tabu search method. Pattern Recognition 35 (2002) 701-711