An experimental study of one- and two-level classifier fusion for different sample sizes

Pattern Recognition Letters

Volumn 32, Issue 14, 2011, Pages 1756-1767

  Zhang, Chun Xia ^a,b   Duin, Robert P W ^b  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b Delft University of Technology ^*  (Netherlands)

Author keywords

Classifier fusion rule; Ensemble classifier; Fixed combiner; Trainable combiner; Training sample size

Indexed keywords

CLASSIFIER FUSION; ENSEMBLE CLASSIFIERS; FIXED COMBINER; TRAINABLE COMBINERS; TRAINING SAMPLE SIZE;

SAMPLING;

EID: 80052546565     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2011.07.009     Document Type: Article

References (47)

1. F.M. Alkoot, and J. Kittler Experimental evaluation of expert fusion strategies Pattern Recognition Lett. 20 11-13 1999 1361 1369 (Pubitemid 32261916)
2. L. Breiman Bagging predictors Machine Learn. 24 2 1996 123 140 (Pubitemid 126724382)
3. L. Breiman Randomizing outputs to increase prediction accuracy Machine Learn. 40 3 2000 229 242
4. J.B.D. Cabrera On the impact of fusion strategies on classification errors for large ensembles of classifiers Pattern Recognition 39 11 2006 1963 1978 (Pubitemid 44175453)
5. A.M.P. Canuto, M.C.C. Abreu, L.d. Melo Oliveira, J.C. Xavier Jr., and A.d.M. Santos Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensembles Pattern Recognition Lett. 28 4 2007 472 486 (Pubitemid 44918135)
6. D.C. Chen, and X.Z. Cheng An asymptotic analysis of some expert fusion methods Pattern Recognition Lett. 22 8 2001 901 904
7. J. Czyz, J. Kittler, and L. Vandendorpe Multiple classifier combination for face-based identity verification Pattern Recognition 37 7 2004 1459 1469 (Pubitemid 38653374)
8. M. Diaz, and J.S. Rao Non-parametric bootstrap ensembles for detection of tumor lesions Pattern Recognition Lett. 28 16 2007 2273 2283 (Pubitemid 47538782)
9. Duin, R.P.W., 2002. The combining classifiers: to train or not to train? In: Proc. 16th Internat. Conf. Pattern Recognition, Quebec City, Canada, vol. 2, pp. 765-770.
10. Duin, R.P.W., Juszczak, P., Paclík, P., Pȩkalska, E., Ridder, D., Tax, D.M.J., Verzakov, S., 2007. PRTools4: a matlab toolbox for pattern recognition, Delft University of Technology, Delft.
11. R.P.W. Duin, and D.M.J. Tax Classifier conditional posterior probabilities A. Amin, D. Dori, P. Pudil, H. Freeman, Adv. Pattern Recognition Lecture Notes in Computer Science vol. 1451 1998 Springer Berlin 611 619 (Pubitemid 128117163)
12. R.P.W. Duin, and D.M.J. Tax Experiments with classifier combination rules J. Kittler, F. Roli, MCS Lecture Notes in Computer Science vol. 1857 2000 Springer Berlin 16 29
13. S. Džeroski, and B. Ženko Is combining classifiers with stacking better than selecting the best ones? Machine Learn. 54 3 2004 255 273
14. Y. Freund, and R.E. Schapire Experiments with a new boosting algorithm Proc. 13th Internat. Conf. Machine Learning 1996 Morgan Kaufmann Press Bari, Italy 148 156
15. G. Fumera, and F. Roli A theoretical and experimental analysis of linear combiners for multiple classifier systems IEEE Trans. Pattern Anal. Machine Intell. 27 6 2005 942 956 (Pubitemid 40889844)
16. Giacinto G., Roli F., 1999. Methods for dynamic classifier selection. In: Proc. 10th Internat. Conf. Image Anal. Process., Venice, Italy, pp. 659-664.
17. T.K. Ho The random subspace method for constructing decision forests IEEE Trans. Pattern Anal. Machine Intell. 20 8 1998 832 844 (Pubitemid 128741345)
18. J. Kittler, and F.M. Alkoot Sum versus vote fusion in multiple classifier systems IEEE Trans. Pattern Anal. Machine Intell. 25 1 2003 110 115
19. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas On combining classifiers IEEE Trans. Pattern Anal. Machine Intell. 20 3 1998 226 239 (Pubitemid 128741312)
20. L.I. Kuncheva Switching between selection and fusion in combining classifiers: an experiment IEEE Trans. Systems Man Cybernet. B 32 2 2002 146 156 (Pubitemid 34249373)
21. L.I. Kuncheva A theoretical study on six classifier fusion strategies IEEE Trans. Pattern Anal. Machine Intell. 24 2 2002 281 286 (Pubitemid 34198215)
22. L.I. Kuncheva Combining Pattern Classifiers: Methods and Algorithms 2004 John Wiley & Sons New Jersey
23. L.I. Kuncheva, J.C. Bezdek, and R.P.W. Duin Decision templates for multiple classifier fusion: an experimental comparison Pattern Recognition 34 2 2001 299 314 (Pubitemid 32871876)
24. C.J. Lawson, and R.J. Hanson Solving Least Squares Problems 1995 SIAM Publications Philadelphia
25. J.F. Lichtenauer, E.A. Hendriks, and M.J.T. Reinders Sign language recognition by combining statistical DTW and independent classification IEEE Pattern Anal. Machine Intell. 30 11 2008 56 (Supplemental material, )
26. J.E. Malek, A.M. Alimi, and R. Tourki Problems in pattern classification in high dimensional spaces: behavior of a class of combined neuro-fuzzy classifiers Fuzzy Sets Systems 128 1 2002 15 33 (Pubitemid 34440522)
27. G. Martínez-Muñoz, and A. Suárez Out-of-bag estimation of the optimal sample size in bagging Pattern Recognition 43 1 2010 143 152
28. E. Menahem, L. Rokach, and Y. Elovici Troika-an improved stacking schema for classification tasks Inform. Sci. 179 24 2009 4097 4122
29. P. Paclík, T.C.W. Landgrebe, D.M.J. Tax, and R.P.W. Duin On deriving the second-stage training set for trainable combiners N.C. Oza, R. Polikar, J. Kittler, F. Roli, MCS Lecture Notes in Computer Science vol. 3541 2005 Springer Berlin 136 146 (Pubitemid 41422612)
30. R. Polikar Ensemble based systems in decision making IEEE Circ. Systems 6 3 2006 21 45
31. A.F.R. Rahman, and M.C. Fairhurst Multiple classifier decision combination strategies for character recognition: a review Internt. J. Doc. Anal. Recognition 5 4 2003 166 194
32. S. Raudys Experts' boasting in trainable fusion rules IEEE Trans. Pattern Anal. Machine Intell. 25 9 2003 1178 1182
33. S. Raudys Trainable fusion rules. I. Large sample size case Neural Networks 19 10 2006 1506 1516 (Pubitemid 44793161)
34. S. Raudys Trainable fusion rules. II. Small sample-size effects Neural Networks 19 10 2006 1517 1527 (Pubitemid 44793162)
35. S. Raudys, and R.P.W. Duin Expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix Pattern Recognition Lett. 19 5-6 1998 385 392 (Pubitemid 128369345)
36. L. Rokach Taxonomy for characterizing ensemble methods in classification tasks: a review and annotated bibliography Comput. Stat. Data Anal. 53 4 2009 4046 4072
37. F. Roli, and G. Fumera Analysis of linear and order statistics combiners for fusion of imbalanced classifiers F. Roli, J. Kittler, MCS Lect. Notes Comput. Sci. vol. 64 2002 Springer Berlin 252 261
38. F. Roli, S. Raudys, and G.L. Marcialis An experimental comparison of fixed and trained fusion rules for crisp classifier outputs F. Roli, J. Kittler, MCS Lecture Notes in Computer Science vol. 2364 2002 Springer Berlin 232 241
39. M. Skurichina, and R.P.W. Duin Bagging for linear classifiers Pattern Recognition 31 7 1998 909 930 (Pubitemid 128428102)
40. M. Skurichina, and R.P.W. Duin Regularisation of linear classifiers by adding redundant features Pattern Anal. Appl. 2 1 1999 44 52
41. D.M.J. Tax, M. Breukelen, R.P.W. Duin, and J. Kittler Combining multiple classifiers by averaging or by multiplying? Pattern Recognition 33 9 2000 1475 1485
42. O.R. Terrades, E. Valveny, and S. Tabbone Optimal classifier fusion in a non-bayesian probabilistic framework IEEE Trans. Pattern Anal. Machine Intell. 31 9 2009 1630 1644
43. K.M. Ting, and I.H. Witten Issues in stacked generalization J. Artif. Intell. Res. 10 1999 271 289 (Pubitemid 30573908)
44. A.R. Webb Statistical Pattern Recognition second ed. 2002 John Wiley & Sons Chichester
45. K. Woods, W.P. Kegelmeyer Jr., and K. Bowyer Combination of multiple classifiers using local accuracy estimates IEEE Trans. Pattern Anal. Machine Intell. 19 4 1997 405 410 (Pubitemid 127762896)
46. L. Xu, A. Krzyak, and C.Y. Suen Methods of combining multiple classifiers and their applications to handwriting recognition IEEE Trans. Systems Man Cybernet. 22 3 1992 418 435
47. C.X. Zhang, and R.P.W. Duin An empirical study of a linear regression combiner on multi-class data sets J.A. Benediktsson, J. Kittler, F. Roli, MCS Lecture Notes in Computer Science vol. 5519 2009 Springer Berlin 478 487