Supervised subspace projections for constructing ensembles of classifiers

Information Sciences

Volumn 193, Issue , 2012, Pages 1-21

  García Pedrajas, Nicolás ^a   Maudes Raedo, Jesús ^b   García Osorio, César ^b   Rodríguez Díez, Juan J ^b  

^a UNIVERSITY OF CÓRDOBA   (Spain)

^b UNIVERSITY OF BURGOS   (Spain)

Author keywords

Classification; Ensembles of classifiers; Subspace methods; Supervised projections

Indexed keywords

BASE LEARNERS; BIAS AND VARIANCE; CLASS LABELS; ENSEMBLE METHODS; ENSEMBLES OF CLASSIFIERS; PROJECTION METHOD; RANDOM SUBSPACE METHOD; RANDOM SUBSPACES; SUB-SPACE METHODS; SUBSPACE PROJECTION; SUPERVISED PROJECTIONS; TESTING ERRORS; UCI MACHINE LEARNING REPOSITORY;

CLASSIFICATION (OF INFORMATION); DECISION TREES;

ADAPTIVE BOOSTING;

EID: 84857789335     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2011.06.023     Document Type: Article

References (49)

1. N. García-Pedrajas, C. García-Osorio, and C. Fyfe Nonlinear boosting projections for ensemble construction Journal of Machine Learning Research 8 2007 1 33 (Pubitemid 46155122)
2. I. Koprinska, J. Poon, J. Clark, and J. Chan Learning to classify e-mail Information Sciences 177 2007 2167 2187 (Pubitemid 46356315)
3. A. Ula, M. Semerci, O.T. Yildiz, and E. Alpaydin Incremental construction of classifier and discriminant ensembles Information Sciences 179 2009 1298 1318
4. J. Hansen Combining predictors: comparison of five meta machine learning methods Information Sciences 119 1-2 1999 91 105 (Pubitemid 30544699)
5. T.G. Dietterich Ensemble Methods in Machine Learning J. Kittler, F. Roli, Proceedings of the First International Workshop on Multiple Classifier Systems Lecture Notes in Computer Science vol. 1857 2000 Springer-Verlag 1 15
6. L.I. Kuncheva Combining classifiers: Soft computing solutions S.K. Pal, A. Pal, Pattern Recognition: From Classical to Modern Approaches 2001 World Scientific 427 451
7. Y. Freund, R. Schapire, Experiments with a new boosting algorithm, in: Proceedings of the Thirteenth International Conference on Machine Learning, Bari, Italy, 1996, pp. 148-156.
8. L. Breiman Bagging predictors Machine Learning 24 2 1996 123 140 (Pubitemid 126724382)
9. L. Breiman, Bias, Variance, and Arcing Classifiers, Technical Report, 460, Department of Statistics, University of California, Berkeley, CA, 1996.
10. R.E. Schapire, Y. Freund, P.L. Bartlett, and W.S. Lee Boosting the margin: a new explanation for the effectiveness of voting methods Annals of Statistics 26 5 1998 1651 1686 (Pubitemid 128376902)
11. E. Bauer, and R. Kohavi An empirical comparison of voting classification algorithms: bagging, boosting, and variants Machine Learning 36 1/2 1999 105 142
12. L. Kuncheva, and C.J. Whitaker Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy Machine Learning 51 2 2003 181 207
13. T.G. Dietterich An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization Machine Learning 40 2000 139 157
14. T.K. Ho The random subspace method for constructing decision forests IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 1998 832 844 (Pubitemid 128741345)
15. M. Skurichina, R.P.W. Duin, Bagging and the random subspace method for redundant feature spaces, in: J. Kittler, R. Poli (Eds.), Proceedings of the Second International Workshop on Multiple Classifier Systems MCS 2001, Cambridge, UK, 2001, pp. 1-10. (Pubitemid 33303208)
16. J.J. Rodríguez, L.I. Kuncheva, and C.J. Alonso Rotation forest: a new classifier ensemble method IEEE Transactions on Pattern Analysis and Machine Intelligence 28 10 2006 1619 1630 (Pubitemid 46405081)
17. N. García-Pedrajas, and C. García-Osorio Constructing ensembles of classifiers using supervised projection methods based on misclassified instances Expert Systems with Applications 38 1 2010 343 359
18. D.F. Specht A general regression neural network IEEE Transactions on Neural Networks 2 6 1991 568 576
19. K. Fukunaga, and J. Mantock Nonparametric discriminant analysis IEEE Transaction on Pattern Analysis and Machine Intelligence 5 6 1983 671 678
20. B.-C. Kuo, and D.A. Landgrebe Nonparametric weighted feature extraction for classification IEEE Transactions on Geoscience and Remote Sensing 42 5 2004 1096 1105
21. B.-C. Kuo, L.W. Ko, C.-H. Pai, D.A. Landgrebe, Regularized feature extractions for hyperspectral data classification, in: International Geoscience and Remote Sensing Symposium, vol. 3, 2003, pp. 1767-1769.
22. A. Sierra, and A. Echevarría Evolutionary discriminant analysis IEEE Transactions on Evolutionary Computation 10 1 2006 81 92 (Pubitemid 43174987)
23. X. Geng, D.-C. Zhan, and Z.-H. Zhou Supervised nonlinear dimensionality reduction for visualization and classification IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 35 6 2005 1098 1107 (Pubitemid 41800411)
24. D. de Ridder, R. Duin, Locally Linear Embedding for classification (Technical Report PH-2002-01), Technical Report, Pattern Recognition Group, Department of Imaging Science and Technology, Delft University of Technology, 2002.
25. S.T. Roweis, and L.K. Saul Nonlinear dimensionality reduction by locally linear embedding Science 290 5500 2000 2323 2326 (Pubitemid 32041578)
26. Q. Zhao, D. Zhang, H. Lu, Supervised LLE in ICA space for facial expression recognition, in: Proceedings of the International Conference on Neural Networks and Brain ICNNB'05, vol. 3, 2005, pp. 1970-1975. doi:10.1109/ICNNB.2005.1615010. (Pubitemid 46289316)
27. Y. Bengio, J.-F. Paiement, P. Vincent, O. Delalleau, N.L. Roux, and M. Ouimet Out-of-sample extensions for LLE, Isomap, MDS, Eigenmaps, and spectral clustering S. Thrun, L. Saul, B. Schölkopf, Advances in Neural Information Processing Systems 16 2004 MIT Press Cambridge, MA 177 184
28. O. Kouropteva, O. Okun, and M. Pietikäinen Incremental locally linear embedding Pattern Recognition 38 10 2005 1764 1767 (Pubitemid 41037108)
29. M.H.C. Law, and A.K. Jain Incremental nonlinear dimensionality reduction by manifold learning IEEE Transactions on Pattern Analysis and Machine Intelligence 28 3 2006 377 391 (Pubitemid 43159634)
30. S. Schuon, M. Durkovic, K. Diepold, J. Scheuerle, S. Markward, Truly incremental locally linear embedding, in: 1st International Workshop on Cognition for Technical Systems.
31. S. Xiang, F. Nie, Y. Song, C. Zhang, and C. Zhang Embedding new data points for manifold learning via coordinate propagation Knowledge and Information Systems 19 2 2009 159 184
32. L. Shi, Q. Yang, E. Liu, J. Li, and Y. Dong An incremental manifold learning algorithm based on the small world model Proceedings of the 2010 International Conference on Life system Modeling and Intelligent Computing, and 2010 International Conference on Intelligent Computing for Sustainable Energy and Environment: Part I, LSMS/ICSEE'10 2010 Springer-Verlag Berlin, Heidelberg 324 332
33. J. Yan, B. Zhang, S. Yan, N. Liu, Q. Yang, Q. Cheng, H. Li, Z. Chen, and W.-Y. Ma A scalable supervised algorithm for dimensionality reduction on streaming data Information Sciences 176 2006 2042 2065
34. A. Schclar, and L. Rokach Random projection ensemble classifiers J. Filipe, J. Cordeiro, Proceedings of the 11th International Conference on Enterprise Information Systems 2009 Springer-Verlag 309 316
35. J. Maudes, J.J. Rodríguez, C. García-Osorio, and C. Pardo Random projections for linear svm ensembles Applied Intelligence 34 3 2011 347 359
36. X.-Z. Wang, and C.-R. Dong Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy IEEE Transactions on Fuzzy Systems 2009 556 567
37. L.I. Kuncheva Full-class set classification using the hungarian algorithm International Journal of Machine Learning and Cybernetics 1 2010 53 61
38. B. Biggio, G. Fumera, and F. Roli Multiple classifier systems for robust classifier design in adversarial environments International Journal of Machine Learning and Cybernetics 1 2010 27 41
39. A. Frank, and A. Asuncion UCI machine learning repository 2010 http://www.archive.ics.uci.edu/ml
40. T.G. Dietterich Approximate statistical tests for comparing supervised classification learning algorithms Neural Computation 10 7 1998 1895 1923 (Pubitemid 128463689)
41. J. Demšar Statistical comparisons of classifiers over multiple data sets Journal of Machine Learning Research 7 2006 1 30 (Pubitemid 43022939)
42. G.I. Webb MultiBoosting: a technique for combining boosting and wagging Machine Learning 40 2 2000 159 196
43. J.R. Quinlan Bagging, boosting, and C4.5 Proceedings of the Thirteenth National Conference on Artificial Intelligence 1996 AAAI Press and the MIT Press 725 730
44. D.D. Margineantu, and T.G. Dietterich Pruning adaptive boosting D.H. Fisher, Proceedings of the Fourteenth International Conference on Machine Learning 1997 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA 211 218
45. Y. Zhang, and S. Bhattacharyya Genetic programming in classifying large-scale data: an ensemble method Information Sciences 163 2004 85 101
46. E.B. Kong, and T.G. Dietterich Error-correcting output coding corrects bias and variance A. Prieditis, J.F. Lemmer, Machine Learning: Proceedings of the Twelfth International Conference 1995 Elsevier Science Publishers 275 283
47. R. Kohavi, and D.H. Wolpert Bias plus variance decomposition for zero-one loss functions L. Saitta, Machine Learning: Proceedings of the Thirteenth International Conference 1996 Morgan Kaufman 275 283
48. L. Breiman Stacked regressions Machine Learning 24 1 1996 49 64 (Pubitemid 126724379)
49. J.H. Friedman On bias, variance, 0/1 - loss, and the curse-of- dimensionality Data Mining and Knowledge Discovery 1 1997 55 77