Democratic instance selection: A linear complexity instance selection algorithm based on classifier ensemble concepts

Artificial Intelligence

Volumn 174, Issue 5-6, 2010, Pages 410-441

  García Osorio, César ^a   de Haro García, Aida ^b   García Pedrajas, Nicolás ^b  

^a UNIVERSITY OF BURGOS   (Spain)

^b UNIVERSITY OF CÓRDOBA   (Spain)

Author keywords

Ensembles; Huge problems; Instance selection; Instance based learning

Indexed keywords

CLASSIFIER ENSEMBLES; COMPUTATIONAL COSTS; DATA SETS; ENSEMBLES OF CLASSIFIERS; EXECUTION TIME; FAST EXECUTION TIME; INSTANCE BASED LEARNING; INSTANCE SELECTION; LARGE DATASETS; LINEAR COMPLEXITY; SCALING PROBLEM; TESTING ERRORS; UCI MACHINE LEARNING REPOSITORY; VOTING SCHEMES; WEAK LEARNER;

ALGORITHMS; CLASSIFIERS; PHILOSOPHICAL ASPECTS;

COMPUTATIONAL EFFICIENCY;

EID: 76549101909     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2010.01.001     Document Type: Article

References (48)

1. Craven M., DiPasquoa D., Freitagb D., McCalluma A., Mitchella T., Nigama K., and Slatterya S. Learning to construct knowledge bases from the World Wide Web. Artificial Intelligence 118 1-2 (2000) 69-113
2. Provost F.J., and Kolluri V. A survey of methods for scaling up inductive learning algorithms. Data Mining and Knowledge Discovery 2 (1999) 131-169
3. Liu H., Motada H., and Yu L. A selective sampling approach to active feature selection. Artificial Intelligence 159 1-2 (2004) 49-74
4. Cano J.R., Herrera F., and Lozano M. Using evolutionary algorithms as instance selection for data reduction in KDD: An experimental study. IEEE Transactions on Evolutionary Computation 7 6 (2003) 561-575
5. Brighton H., and Mellish C. Advances in instance selection for instance-based learning algorithms. Data Mining and Knowledge Discovery 6 (2002) 153-172
6. Cano J.R., Herrera F., and Lozano M. Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability. Data & Knowledge Engineering 60 1 (2007) 90-108
7. Domingos P., and Hulten G. A general framework for mining massive data streams. Journal of Computational and Graphical Statistics 12 4 (2003) 945-949
8. García-Pedrajas N., García-Osorio C., and Fyfe C. Nonlinear boosting projections for ensemble construction. Journal of Machine Learning Research 8 (2007) 1-33
9. Schapire R.E., Freund Y., Bartlett P.L., and Lee W.S. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics 26 5 (1998) 1651-1686
10. García-Osorio C., and Fyfe C. Regaining sparsity in kernel principal components. Neurocomputing 67 (2005) 398-402
11. Asimov D. The grand tour: A tool for viewing multidimensional data. SIAM Journal on Scientific and Statistical Computing 6 1 (1985) 128-143
12. Buja A., and Asimov D. Grand Tour methods: An outline. In: Allen D. (Ed). Computer Science and Statistics: Proceedings of the Seventeenth Symposium on the Interface (1986), Elsevier Science Publisher B.V., North Holland, Amsterdam 63-67
13. Buja A., Cook D., Asimov D., and Hurley C. Computational Methods for High-Dimensional Rotations in Data Visualization (2005), North-Holland Publishing Co. Ch. 14, pp. 391-414
14. Wegman E.J., and Solka J.L. On some mathematics for visualising high dimensional data. Indian Journal of Statistics (Series A Pt. 2) 64 (2002) 429-452
15. Andrews D.F. Plots of high dimensional data. Biometrics 28 (1972) 125-136
16. Wegman E.J., and Shen J. Three-dimensional Andrews plots and the grand tour. Computing Science and Statistics 25 (1993) 284-288
17. García-Pedrajas N., Romero del Castillo J.A., and Ortiz-Boyer D. A cooperative coevolutionary algorithm for instance selection for instance-based learning. Machine Learning 78 3 (2010) 381-420
18. Cano J.R., Herrera F., and Lozano M. Stratification for scaling up evolutionary prototype selection. Pattern Recognition Letters 26 7 (2005) 953-963
19. Kim S.-W., and Oommen B.J. Enhancing prototype reduction schemes with recursion: A method applicable for "large" data sets. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics 34 3 (2004) 1384-1397
20. de Haro-García A., and Pedrajas N.G. A divide-and-conquer recursive approach for scaling up instance selection algorithms. Data Mining and Knowledge Discovery 18 3 (2009) 392-418
21. Domingos P., and Hulten G. A general method for scaling up machine learning algorithms and its application to clustering. Proceedings of the Eighteenth International Conference on Machine Learning (2001), Morgan Kaufmann 106-113
22. Domingos P., and Hulten G. Learning from infinite data in finite time. Proceedings of Advances in Neural Information Systems vol. 14 (2001), Vancouver, Canada 673-680
23. Howffding W. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58 (1963) 13-30
24. G. Hulten, P. Domingos, Mining complex models from arbitrarily large databases in constant time, in: Proceedings of the International Conference on Knowledge Discovery and Data Mining, Edmonton, Canada, 2002, pp. 525-531
25. Hettich S., Blake C., and Merz C. UCI Repository of machine learning databases (1998). http://www.ics.uci.edu/~mlearn/MLRepository.html
26. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
27. Liu H., and Motoda H. On issues of instance selection. Data Mining and Knowledge Discovery 6 (2002) 115-130
28. Wilson D.R., and Martinez T.R. Reduction techniques for instance-based learning algorithms. Machine Learning 38 (2000) 257-286
29. Wilson D.L. Asymptotic properties of nearest neighbor rules using edited data. IEEE Transactions on Systems, Man, and Cybernetics 2 3 (1972) 408-421
30. Gates G.W. The reduced nearest neighbor rule. IEEE Transactions on Information Theory 18 3 (1972) 431-433
31. Barandela R., Ferri F.J., and Sánchez J.S. Decision boundary preserving prototype selection for nearest neighbor classification. International Journal of Pattern Recognition and Artificial Intelligence 19 6 (2005) 787-806
32. Kuncheva L. Editing for the k-nearest neighbors rule by a genetic algorithm. Pattern Recognition Letters 16 (1995) 809-814
33. Ishibuchi H., and Nakashima T. Pattern and feature selection by genetic algorithms in nearest neighbor classification. Journal of Advanced Computational Intelligence and Intelligent Informatics 4 2 (2000) 138-145
34. Reeves C.R., and Bush D.R. Using genetic algorithms for training data selection in RBF networks. In: Liu H., and Motoda H. (Eds). Instances Selection and Construction for Data Mining (2001), Kluwer, Norwell, Massachusetts, USA 339-356
35. Goldberg D.E. Genetic Algorithms in Search, Optimization and Machine Learning (1989), Addison-Wesley, Reading, MA
36. D. Whitley, The GENITOR algorithm and selective pressure, in: M.K. Publishers (Ed.), Proc. 3rd International Conf. on Genetic Algorithms, Los Altos, CA, 1989, pp. 116-121
37. Eshelman L.J. The CHC Adaptive Search Algorithm: How to Have Safe Search when Engaging in Nontraditional Genetic Recombination (1990), Morgan Kauffman, San Mateo, CA
38. S. Baluja, Population-based incremental learning, Tech. Rep. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, 1994
39. J. Maudes-Raedo, J.J. Rodríguez-Díez, C. García-Osorio, Disturbing neighbors diversity for decision forest, in: G. Valentini, O. Okun (Eds.), Workshop on Supervised and Unsupervised Ensemble Methods and Their Applications (SUEMA 2008), Patras, Greece, 2008, pp. 67-71
40. Sebban M., Nock R., Chauchat J.H., and Rakotomalala R. Impact of learning set quality and size on decision tree performances, International Journal of Computers. Systems and Signals 1 1 (2000) 85-105
41. Brodley C.E., and Friedl M.A. Identifying mislabeled training data. Journal of Artificial Intelligence Research 11 (1999) 131-167
42. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann, San Mateo
43. Chang C.-C., and Lin C.-J. LIBSVM: A library for support vector machines, software available at (2001). http://www.csie.ntu.edu.tw/~cjlin/libsvm
44. Sebban M., and Nock R. Identifying and eliminating irrelevant instances using information theory. In: Hamilton H., and Yang Q. (Eds). 13th Biennial Conference of the Canadian Society for Computational Studies of Intelligence, AI 2000. Montreal. Lecture Notes in Artificial Intelligence vol. 1822 (2000), Springer 90-101
45. Sane S.S., and Ghatol A.A. A novel Supervised Instance Selection algorithm. International Journal on Business Intelligence and Data Mining 2 4 (2007) 471-495
46. Bauer E., and Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36 1/2 (1999) 105-142
47. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning 40 (2000) 139-157
48. García-Pedrajas N. Constructing ensembles of classifiers by means of weighted instance selection. IEEE Transactions on Neural Networks 20 2 (2008) 258-277