Boosting k-nearest neighbor classifier by means of input space projection

Expert Systems with Applications

Volumn 36, Issue 7, 2009, Pages 10570-10582

  García Pedrajas, Nicolás ^a   Ortiz Boyer, Domingo ^a  

^a UNIVERSITY OF CÓRDOBA   (Spain)

Author keywords

Boosting; k Nearest neighbors; Subspace methods

Indexed keywords

BOOSTING; BOOSTING METHODS; CLASS LABELS; CLASSIFICATION ALGORITHMS; COMPLEX METHODS; ENSEMBLE METHODS; GENERALIZATION CAPABILITIES; GENERALIZATION ERRORS; INPUT SELECTIONS; INPUT SPACES; K-NEAREST NEIGHBOR CLASSIFIERS; K-NEAREST NEIGHBORS; K-NEAREST NEIGHBORS CLASSIFIERS; K-NN CLASSIFIERS; NOISE TOLERANCES; RANDOM SUBSPACE METHODS; RE SAMPLINGS; STANDARD METHODS; SUBSPACE METHODS; TEST SETS; UCI MACHINE LEARNING REPOSITORIES;

CLASSIFIERS; FUZZY CONTROL; MEMBERSHIP FUNCTIONS; TEXT PROCESSING;

LEARNING SYSTEMS;

EID: 67349220171     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2009.02.065     Document Type: Article

References (42)

1. Altinçay H. Ensembling evidential k-nearest neighbor classifiers through multi-modal perturbation. Applied Soft Computing 7 (2007) 1072-1083
2. Amores J., Sebe N., and Radeva P. Boosting the distance estimation. Application to the k-nearest neighbor classifier. Pattern Recognition Letters 27 (2006) 201-209
3. Athitsos, V., & Sclaroff, S. (2005). Boosting nearest neighbor classiers for multiclass recognition. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (Vol. 3, pp. 45-52).
4. Bao, Y., Ishii, N., & Du, X. (2004). Combining multiple k-nearest neighbor classifiers using different distance functions. In: Proceedings of the fifth international conference on intelligent data engineering and automated learning. Lecture notes in computer science. UK: Springer, Exeter (Vol. 3177, pp. 634-641).
5. Bauer E., and Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36 1/2 (1999) 105-142
6. Bay S.D. Nearest neighbor classification from multiple feature subsets. Intelligent Data Analysis 3 3 (1999) 191-209
7. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
8. Breiman, L. (1996b). Bias, variance, and arcing classifiers. Technical Report 460, Department of Statistics, University of California, Berkeley, CA.
9. Breiman L. Stacked regressions. Machine Learning 24 1 (1996) 49-64
10. Breiman L. Random forests. Machine Learning 45 (2001) 5-32
11. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
12. Dietterich T.G. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10 7 (1998) 1895-1923
13. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40 (2000) 139-157
14. Domeniconi, C., & Yan, B., 2004. Nearest neighbor ensemble. In: Proceedings of the 17th international conference on pattern recognition (ICPR'04) (Vol. 1, pp. 228-231).
15. Domeniconi C., Peng J., and Gunopulos D. Locally adaptive metric nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 9 (2002) 1281-1285
16. Eshelman, L. J. (1990). The CHC adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombination. San Mateo, CA: Morgan Kauffman.
17. François J., Grandvalet Y., Denœux T., and Roger J.M. Resample and combine: An approach to improving uncertainty representation in evidential pattern classification. Information Fusion 4 (2003) 75-85
18. Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. In Proceedings of the thirteenth international conference on machine learning, Bari, Italy (pp. 148-156).
19. Friedman M. A comparison of alternative tests of significance for the problem of M rankings. Annals of Mathematical Statistics 11 (1940) 86-92
20. García-Pedrajas N., García-Osorio C., and Fyfe C. Nonlinear boosting projections for ensemble construction. Journal of Machine Learning Research 8 (2007) 1-33
21. García-Pedrajas, N. (in press). Supervised projection approach for boosting classifiers. Pattern Recognition.
22. García-Pedrajas N., and Ortiz-Boyer D. Boosting random subspace method. Neural Network 21 (2008) 1344-1362
23. Grabowski, S. (2002). Voting over multiple k-NN classifiers. In Proceedings of the international conference on modern problems of radio engineering, telecommunications and computer science (pp. 223-225).
24. Hastie T., and Tibshirani R. Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 18 6 (1996) 607-615
25. Hettich, S., Blake, C., & Merz, C. (1998). UCI repository of machine learning databases, .
26. Ho, T. K. (1998a). Nearest neighbors in random subspaces. In Proceedings of the second international workshop on statistical techniques in pattern recognition, Syndey, Australia (pp. 640-648).
27. Ho T.K. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 (1998) 832-844
28. Jolliffe I.T. Principal components analysis (1986), Springer-Verlag, New York, NY
29. Kittler J., and Alkoot F.M. Moderating k-NN classifiers. Pattern Analysis & Applications 5 3 (2002) 326-332
30. Kohavi, R. & Wolpert, D. H. (1996). Bias plus variance decomposition for zero-one loss functions. In L. Saitta (Ed.), Machine learning: Proceedings of the thirteenth international conference (pp. 275-283). Morgan Kaufmann.
31. Kong, E. B. & Dietterich, T.G. (1995). Error-correcting output coding corrects bias and variance. In: Prieditis, A., Lemmer, J.F. (Eds.), Machine Learning: Proceedings of the Twelfth International Conference. Elsevier Science Publishers, pp. 275-283.
32. Lazarevic, A., Fiez, T., & Obradovic, Z. (2000). Adaptive boosting for spatial functions with unstable driving attributes. In Proceedings of the Pacific-Asia conference on knowledge discovery and data mining (pp. 329-340). Japan: Springer-Verlag.
33. Lerner B., Guterman H., Aladjem M., and Dinstein I. A comparative study of neural networks based feature extraction paradigms. Pattern Recognition Letters 20 1 (1999) 7-14
34. O'Sullivan, J., Langford, J., Caruna, R., & Blum, A. (2000). Featureboost: A metalearning algorithm that improves model robustness. In Proceedings of the seventeenth international conference on machine learning (pp. 703-710).
35. Quinlan, J. R. (1996). Bagging, boosting, and c4.5. In Proceedings of the thirteenth national conference on artificial intelligence (pp. 725-730). AAAI Press and the MIT Press.
36. Todorovski L., and Dzeroski S. Combining classifiers with meta decision trees. Machine Learning 50 (2003) 223-249
37. Tremblay, G., Sabourin, R., & Maupin, P. (2004). Optimizing nearest neighbour in random subspaces using a multi-objective genetic algorithm. In: Proceedings of the 17th international conference on pattern recognition (Vol. 1, pp. 208-211).
38. Viswanath P., Murty M.N., and Bhatnagar S. Fusion of multiple approximate nearest neighbor classifiers for fast and efficient classification. Information Fusion 5 (2004) 239-250
39. Webb G.I. Multiboosting: A technique for combining boosting and wagging. Machine Learning 40 2 (2000) 159-196
40. Woods K., Kegelmeyer W., and Bowyer K. Combination of multiple classiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 (1997) 405-410
41. Zhou Z.H., and Yu Y. Adapt bagging to nearest neighbor classifiers. Journal of Computer Science and Technology 20 1 (2005) 48-54
42. Zhou Z.H., and Yu Y. Ensembling local learners through multimodal perturbation. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 35 4 (2005) 725-735