Ensembling evidential k-nearest neighbor classifiers through multi-modal perturbation

Applied Soft Computing Journal

Volumn 7, Issue 3, 2007, Pages 1072-1083

  Altinçay, Hakan ^a  

^a EASTERN MEDITERRANEAN UNIVERSITY   (Turkey)

Author keywords

Classifier ensembles; Evidential k NN classifier; Genetic algorithms; Multi modal perturbation; Random subspace method

Indexed keywords

GENETIC ALGORITHMS; MODAL ANALYSIS; PARAMETER ESTIMATION; PERTURBATION TECHNIQUES; SOFT COMPUTING;

CLASSIFIER ENSEMBLES; EVIDENTIAL K-NN CLASSIFIERS; MULTIMODAL PERTURBATION; RANDOM SUBSPACE METHOD;

FEATURE EXTRACTION;

EID: 34047249450     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2006.10.002     Document Type: Article

References (57)

1. Opitz D., and Maclin R. Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 11 (1999) 169-198
2. Kittler J. On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20 3 (1998) 226-239
3. Alti{dotless}nçay H., and Demirekler M. An information theoretic framework for weight estimation in the combination of probabilistic classifiers for speaker identification. Speech Commun. 30 4 (2000) 255-272
4. C.J. Whitaker, L.I. Kuncheva, Examining the relationship between majority vote accuracy and diversity in bagging and boosting, Technical Report, School of Informatics, University of Wales, Bangor, 2003.
5. Zenobi G., and Cunningham P. Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In: Raedt L.D., and Flach P.A. (Eds). Proceedings of the 12th Conference on Machine Learning, Lecture Notes in Computer Science 2167 (2001) 576-587
6. Melville P., and Mooney R.J. Constructing diverse classifier ensembles using artificial training examples. Proceedings of the IJCAI (2003) 505-510
7. Krogh A., and Vedelsby J. Neural network ensembles, cross validation, and active learning. In: Tesauro G., Touretzky D., and Leen T. (Eds). Advances in Neural Information Processing Systems, vol. 7 (1995), The MIT Press 231-238
8. Zhou Z., and Yu Y. Ensembling local learners through multimodal perturbation. IEEE Trans. Syst. Man Cyber. B: Cyber. 35 4 (2005) 725-735
9. Breiman L. Bagging predictors. Mach. Learn. 24 (1996) 123-140
10. Schapire R.E. The boosting approach to machine learning: an overview. MSRI Workshop on Nonlinear Estimation and Classification. Berkeley, CA (2002)
11. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20 8 (1998) 832-844
12. Whitley D. A genetic algorithm tutorial. Stat. Comput. 4 (1994) 65-85
13. Gen M., and Cheng R. Genetic Algorithms and Engineering Design (1997), John Wiley and Sons, Inc.
14. Leardi R. Application of a genetic algorithm to feature selection under full validation conditions and to outlier detection. J. Chemometr. 8 (1994) 65-79
15. Siedlecki W., and Sklansky J. A note on genetic algorithms for large scale feature selection. Pattern Recogn. Lett. 10 (1989) 335-347
16. Zhang G.P. Neural networks for classification: a survey. IEEE Trans. Syst. Man Cyber. Part C: Appl. Rev. 30 4 (2000) 451-462
17. Benediktsson J.A., Sveinsson J.R., Ingimundarson J.I., Sigurdsson H.S., and Ersoy O.K. Multistage classifiers optimized by neural networks and genetic algorithms. Nonlinear Anal. Theory Methods Appl. 30 3 (1997) 1323-1334
18. Opitz D.W., and Shavlik J.W. Actively searching for an effective neural-network ensemble. Connection Sci. 8 3/4 (1996) 337-353
19. Lazarevic A., Fiez T., and Obradovic Z. Adaptive boosting for spatial functions with unstable driving attributes. Intell. Data Anal. 5 (2001) 1-24
20. Zhou Z., and Yu Y. Adapt bagging to nearest neighbor classifiers. J. Comput. Sci. Technol. 20 1 (2005) 48-54
21. Latinne P., Debeir O., and Decaestecker C. Different ways of weakening decision trees and their impact on classification accuracy of DT combination. In: Kittler J., and Roli F. (Eds). Multiple Classifier Systems. Proceedings of the First International Workshop, MCS 2000. Lecture Notes in Computer Science, Springer-Verlag Ed. (2000)
22. Kuncheva L.I., and Jain L.C. Designing classifier fusion systems by genetic algorithms. IEEE Trans. Evol. Comput. 4 4 (2000) 327-336
23. Gabrys G., and Ruta D. Genetic algorithms in classifier fusion. Appl. Soft Comput. 6 4 (2006) 337-347
24. Alti{dotless}nçay H. Optimal resampling and classifier prototype selection in classifier ensembles using genetic algorithms. Pattern Anal. Appl. 7 (2004) 285-295
25. Pedrajas N.G., Martinez C.H., and Boyer D.O. Cooperative coevolution of artificial neural network enssembles for pattern classification. IEEE Trans. Evol. Comput. 9 3 (2005) 271-302
26. Bao Y., Ishii N., Du X., Yang Z., Everson R., and Yin H. Combining Multiple k-Nearest Neighbor Classifiers Using Different Distance Functions. Lecture Notes in Computer Science, vol. 3177 (2004), Springer, Germany 631-641
27. Bay S.D. Combining nearest neighbor classifiers through multiple feature subsets. Proceedings of the 15th International Conference on Machine Learning (1998), Morgan Kaufmann, San Francisco, CA 37-45
28. Ho T.K., Amin A., Dori D., Pudil P., and Freeman H. Nearest neighbors in random subspaces. Lecture Notes in Computer Science (1998), Springer, Germany 640-648
29. Denœux T. A k-nearest neighbor classification rule based on Dempster-Shafer theory. IEEE Trans. Syst. Man Cyber. 25 5 (1995) 804-813
30. Shafer G. A Mathematical Theory of Evidence (1976), Princeton University Press
31. François J., Grandvalet Y., Denœux T., and Roger J.M. Resample and combine: an approach to improving uncertainty representation in evidential pattern classification. Inform. Fusion 4 (2003) 75-85
32. Shafer G., and Logan R. Implementing Dempster's rule for hierarchical evidence. Artif. Intell. 33 (1987) 271-298
33. Lefevre E., Colot O., and Vannoorenberghe P. Belief function combination and conflict management. Inform. Fusion 3 (2002) 149-162
34. Zadeh L. A simple view of the Dempster-Shafer theory of evidence and its implication for the rule of combination. AI Mag. 7 (1986) 85-90
35. Smets P. The combination of evidence in the transferrable belief model. IEEE Trans. Pattern Anal. Mach. Intell. 12 5 (1990) 447-458
36. Yager R.R. On the Dempster-Shafer framework and new combination rules. Inform. Sci. 41 (1987) 93-138
37. Murphy C.K. Combining belief functions when evidence conflicts. Decision Support Syst. 29 (2000) 1-9
38. Dubois D., and Prade H. Representation and combination of uncertainty with belief functions and possibility measures. Comput. Intell. 4 (1998) 244-264
39. Xu L., Krzyzak A., and Suen C.Y. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Syst. Man Cyber. 22 (1992) 418-435
40. Al-Ani A., and Deriche M. A new technique for combining multiple classifiers using the Dempster-Shafer theory of evidence. J. Artif. Intell. Res. 17 (2002) 333-361
41. Smets P., and Kennes R. The transferrable belief model. Artif. Intell. 66 (1994) 191-234
42. Hegarat-Mascle S.L., Bloch I., and Vidal-Madjar D. Introduction of neighborhood information in evidence theory and application to data fusion of radar and optical images with partial cloud cover. Pattern Recogn. 31 11 (1998) 1811-1823
43. Zouhal L., and Denœux T. An evidence-theoretic k-NN rule with parameter optimization. IEEE Trans. Syst. Man Cyber. 28 (1998) 263-271
44. Goldberg D.E. Genetic Algorithms in Search, Optimization and Machine Learning (1989), Addison-Wesley Publishing Company, Inc.
45. Hao H., Liu C., and Sako H. Comparison of genetic algorithm and sequential search methods for classifier subset selection. Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR'2003) (2003)
46. Ruta D., and Gabrys B. Application of the evolutionary algorithms for classifier selection in multiple clasifier systems with majority voting. In: Kittler J., and Roli F. (Eds). Multiple Classifier Systems. Proceedings of the Second International Workshop, MCS 2001. Lecture Notes in Computer Science, Springer-Verlag Ed. (2001) 399-408
47. Lam L., and Suen C.Y. Optimal combinations of pattern classifiers. Pattern Recogn. Lett. 16 (1995) 945-954
48. Zhou Z., Wu J., and Tang W. Ensembling neural networks: many could be better than all. Artif. Intell. 137 (2002) 239-263
49. Wang X., and Wang H. Classification by evolutionary ensembles. Pattern Recogn. 39 4 (2006) 595-607
50. Nguyen M.H., Abbass H.A., and McKay R.I. Stopping criteria for ensemble of evolutionary artificial neural networks. Appl. Soft Comput. 6 (2006) 100-107
51. Friedrich C.M. Ensembles of evolutionary created artificial neural networks and nearest neighbour classifiers. Proceedings of the Third On-line Conference on Soft-computing in Engineering Design and Manufacturing (WSCs) (1998) 288-298
52. Lin X., Ding X., Chen M., Zhang R., and Wu Y. Adaptive confidence transform based classifier combination for Chinese character recognition. Pattern Recogn. Lett. 19 (1998) 975-988
53. Spears W.M. Adapting crossover in evolutionary algorithms. In: McDonnell J.R., Reynolds R.G., and Fogel D.B. (Eds). Proceedings of the Fourth Annual Conference on Evolutionary Programming (1995), MIT Press, Cambridge, MA 367-384
54. Maslov I.V., and Gertner I. Multi-sensor fusion: an evolutionary algorithm approach. Inform. Fusion 7 3 (2006) 304-330
55. Eshelman L.J., and Schaffer J.D. Real-coded genetic algorithms and interval-schemata. Foundations of Genetic Algorithms 2 (FOGA-2) (1993), Morgan Kaufmann Publishers, San Mateo, CA pp. 187-202
56. Kuncheva L.I. Combining Pattern Classifiers Methods and Algorithms (2004), John Wiley and Sons
57. Okun O., and Priisalu H. Multiple views in ensembles of nearest neighbor classifiers. Proceedings of the Workshop on Learning with Multiple Views, 22nd ICML. Germany (2005)