Constructing ensembles of classifiers by means of weighted instance selection

IEEE Transactions on Neural Networks

Volumn 20, Issue 2, 2009, Pages 258-277

  García Pedrajas, Nicolás ^a  

^a UNIVERSITY OF CÓRDOBA   (Spain)

Author keywords

Boosting; Decision trees; Ensembles of classifiers; Instance selection; k nearest neighbors (k NN); Support vector machines (SVMs)

Indexed keywords

DECISION TREES; FUZZY CONTROL; IMAGE RETRIEVAL; MEMBERSHIP FUNCTIONS; SUPPORT VECTOR MACHINES; TEXT PROCESSING;

BOOSTING; ENSEMBLES OF CLASSIFIERS; INSTANCE SELECTION; K-NEAREST NEIGHBORS (K-NN); SUPPORT VECTOR MACHINES (SVMS);

CLASSIFIERS;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; EVALUATION; METHODOLOGY; SUPPORT VECTOR MACHINE;

ALGORITHMS; PATTERN RECOGNITION, AUTOMATED; SUPPORT VECTOR MACHINES;

EID: 60849105643     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2005496     Document Type: Article

References (62)

1. C. J. Merz, "Using correspondence analysis to combine classifiers," Mach. Learn., vol. 36, no. 1, pp. 33-58, Jul. 1999.
2. T. Windeatt, "Accuracy/diversity and ensemble MLP classifier design," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1194-1211, Sep. 2006.
3. T. G. Dietterich, "Ensemble methods in machine learning," in Lecture Notes in Computer Science, J. Kittler and F. Roli, Eds. Berlin, Germany: Springer-Verlag, 2000, vol. 1857, pp. 1-15.
4. L. I. Kuncheva, "Combining classifiers: Soft computing solutions," in Pattern Recognition: From Classical to Modern Approaches, S. K. Pal and A. Pal, Eds. Singapore: World Scientific, 2001, pp. 427-451.
5. Y. Freund and R. Schapire, "Experiments with a new boosting algorithm," in Proc. 13th Int. Conf. Mach. Learn., Bari, Italy, 1996, pp. 148-156.
6. G. I. Webb, "Multiboosting: A technique for combining boosting and wagging," Mach. Learn., vol. 40, no. 2, pp. 159-196, Aug. 2000.
7. S. Dzeroski and B. Zenko, "Is combining classifiers with stacking better than selecting the best one?," Mach. Learn., vol. 54, pp. 255-273, 2004.
8. A. Fern and R. Givan, "Online ensemble learning: An empirical study," Mach. Learn., vol. 53, pp. 71-109, 2003.
9. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, 1996.
10. L. Breiman, "Bias, variance, and arcing classifiers," Dept. Statist., Univ. California, Berkeley, CA, Tech. Rep. 460, 1996.
11. R. E. Schapire, Y. Freund, P. L. Bartlett, and W. S. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 5, pp. 1651-1686, 1998.
12. E. Bauer and R. Kohavi, "An empirical comparison of voting classification algorithms: Bagging, boosting, and variants," Mach. Learn., vol. 36, no. 1/2, pp. 105-142, Jul./Aug. 1999.
13. H. Liu and H. Motoda, "On issues of instance selection," Data Mining Knowl. Disc., vol. 6, pp. 115-130, 2002.
14. J. R. Cano, F. Herrera, and M. Lozano, "Using evolutionary algorithms as instance selection for data reduction in KDD: An experimental study," IEEE Trans. Evol. Comput., vol. 7, no. 6, pp. 561-575, Dec. 2003.
15. C. Domeniconi, G. D., and J. Peng, "Large margin nearest neighbor classifiers," IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 899-909, Jul. 2005.
16. H. Brighton and C. Mellish, "Advances in instance selection for instance- based learning algorithms," Data Mining Knowl. Disc., vol. 6, pp. 153-172, 2002.
17. D. W. Aha, "A study of instance-based algorithms for supervised learning tasks: Mathematical, empirical, and psychological evaluations," Ph.D. dissertation, Dept. Inf. Comput. Sci., Univ. California at Irvine, Irvine, CA, 1990.
18. L. Wiskott, J.-M. Fellous, N. Kríger, and C. von der Malsburg, "Face recognition by elastic bunch graph matching," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 775-779, Jul. 1997.
19. G. Shakhnarovich, P. Viola, and T. Darrell, "Fast pose estimation with parameter-sensitive hashing," in Proc. 9th Int. Conf. Comput. Vis. Nice, France, 2003, pp. 750-757.
20. S. Belongie, J. Malik, and J. Puzicha, "Shape matching and object recognition using shape contexts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 4, pp. 509-522, Apr. 2002.
21. T. K. Ho and H. S. Baird, "Large-scale simulation studies in image pattern recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 10, pp. 1067-1079, Oct. 1997.
22. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, pp. 263-286, 1995.
23. E. B. Kong and T. G. Dietterich, "Error-correcting output coding corrects bias and variance," in Proc. 12th Nat. Conf. Artif. Intell. 1996, pp. 725-730.
24. S. D. Bay, "Nearest neighbor classification from multiple feature subsets," Intell. Data Anal., vol. 3, no. 3, pp. 191-209, 1999.
25. J. O'Sullivan, J. Langford, R. Caruna, and A. Blum, "Featureboost: A metalearning algorithm that improves model robustness," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 703-710.
26. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998.
27. L. Nanni and A. Lumini, "Evolved feature weighting for random subspace classifier," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 363-366, Feb. 2008.
28. E. Kleinberg, "On the algorithmic implementation of stochastic discrimination," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 5, pp. 473-490, May 2000.
29. M. Skurichina and R. P. W. Duin, "Bagging and the random subspace method for redundant feature spaces," in Proc. 2nd Int. Workshop Multiple Classifier Syst., J. Kittler and R. Poli, Eds., Cambridge, U.K., 2001, pp. 1-10.
30. R. Munro, D. Ler, and J. Patrick, "Meta-learning orthographic and contextual models for language independent named entity recognition," in Proc. 7th Conf. Natural Lang. Learn., 2003, pp. 192-195.
31. L. Hall, K. Bowyer, R. Banfield, D. Bhadoria, W. Kegelmeyer, and S. Eschrich, "Comparing pure parallel ensemble creation techniques against bagging," in Proc. 3rd IEEE Int. Conf. Data Mining, Melbourne, FL, 2003, pp. 533-536.
32. D. R. Wilson and T. R. Martinez, "Reduction techniques for instancebased learning algorithms," Mach. Learn., vol. 38, pp. 257-286, 2000.
33. D. L. Wilson, "Asymptotic properties of nearest neighbor rules using edited data," IEEE Trans. Syst. Man Cybern., vol. SMC-2, no. 3, pp. 408-421, 1972.
34. P. E. Hart, "The condensed nearest neighbor rule," IEEE Trans. Inf. Theory, vol. IT-14, no. 3, pp. 515-516, May 1968.
35. G. W. Gates, "The reduced nearest neighbor rule," IEEE Trans. Inf. Theory, vol. IT-18, no. 3, pp. 431-433, May 1972.
36. L. Kuncheva, "Editing for the k-nearest neighbors rule by a genetic algorithm," Pattern Recognit. Lett., vol. 16, pp. 809-814, 1995.
37. H. Ishibuchi and T. Nakashima, "Pattern and feature selection by genetic algorithms in nearest neighbor classification," J. Adv. Comput. Intell. Intell. Inf., vol. 4, no. 2, pp. 138-145, 2000.
38. C. R. Reeves and D. R. Bush, "Using genetic algorithms for training data selection in RBF networks," in Instances Selection and Construction for Data Mining, H. Liu and H. Motoda, Eds. Norwell, MA: Kluwer, 2001, pp. 339-356.
39. D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Reading, MA: Addison-Wesley, 1989.
40. D. Whitley, "The GENITOR algorithm and selective pressure," in Proc. 3rd Int. Conf. Genetic Algorithms, M. K. Publishers, Ed., Los Altos, CA, 1989, pp. 116-121.
41. L. J. Eshelman, The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination. San Mateo, CA: Morgan Kauffman, 1990.
42. S. Baluja, "Population-based incremental learning," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-94-163, 1994.
43. S. J. Louis and G. Li, "Combining robot control strategies using genetic algorithms with memory," in Lecture Notes in Computer Science, Evolutionary Programming VI. Berlin, Germany: Springer-Verlag, 1997, vol. 1213, pp. 431-442.
44. H. Ishibuchi and T. Nakashima, "Multi-objective pattern and feature selection by a genetic algorithm," in Proc. Genetic Evol. Comput. Conf., Las Vegas, NV, 2000, pp. 1069-1076.
45. H. Ishibuchi, T. Nakashima, and M. Nii, "Learning of neural networks with GA-based instance selection," in Proc. IFSA World Congr. 20th NAFIPS Int. Conf., Jul. 2001, vol. 4, pp. 2102-2107.
46. C. E. Brodley, "Recursive automatic bias selection for classifier construction," Mach. Learn., vol. 20, no. 1/2, pp. 63-94, 1995.
47. L. Wang, N. Zhou, and F. Chu, "A general wrapper approach to selection of class-dependent features," IEEE Trans. Neural Netw., vol. 19, no. 7, pp. 1267-1278, Jul. 2008.
48. K.-J. Kim, "Artificial neural networks with evolutionary instance selection for financial forecasting," Expert Syst. Appl., vol. 30, pp. 519-526, 2006.
49. S. Hettich, C. Blake, and C. Merz, UCI Repository of Machine Learning Databases, 1998 [Online]. Available: http://www.ics.uci.edu/mlearn/ MLRepository.html
50. T. G. Dietterich, "Approximate statistical tests for comparing supervised classification learning algorithms," Neural Comput., vol. 10, no. 7, pp. 1895-1923, 1998.
51. J. Demšar, "Statistical comparisons of classifiers over multiple data sets," J. Mach. Learn. Res., vol. 7, pp. 1-30, 2006.
52. R. L. Iman and J. M. Davenport, "Approximations of the critical regions of the Friedman statistic," Commun. Statist., vol. 6, pp. 571-595, 1980.
53. F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics, vol. 1, pp. 80-83, 1945.
54. GNU General Public License Version 3, Free Software Foundation, Inc., Jun. 2007 [Online]. Available: http://www.gnu.org/licenses/ gpl-3.0-standalone.html
55. R. Polikar, L. Udpa, S. Udpa, and V. Honavar, "Learn++: An incremental learning algorithm for supervised neural networks," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol. 31, no. 4, pp. 497-508, Nov. 2001.
56. P. Melville and R. J. Mooney, "Creating diversity in ensembles using artificial data," Inf. Fusion, vol. 6, pp. 99-111, 2005.
57. G. Zenobi and P. Cunningham, "Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error," in Lecture Notes on Artificial Intelligence L. de Raedt and P. Flach, Eds. Berlin, Germany: Springer-Verlag, 2001, vol. 2167, pp. 576-587.
58. J. H. Chen, H. M. Chen, and S. Y. Ho, "Design of nearest neighbor classifiers: Multi-objective approach," Int. J. Approx. Reason., vol. 40, no. 1-2, pp. 3-22, 2005.
59. S. H. Son and J. Y. Kim, "Data reduction for instance-based learning using entropy-based partitioning," in Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2006, vol. 3982, pp. 590-599.
60. J. R. Quinlan, "Bagging, boosting, and c4.5," in Proc. 13th Nat. Conf. Artif. Intell., 1996, pp. 725-730.
61. T. G. Dietterich, "An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization," Mach. Learn., vol. 40, pp. 139-157, 2000.
62. L. Breiman, "Random forests," Mach. Learn., vol. 45, pp. 5-32, 2001.