Incremental construction of classifier and discriminant ensembles

Information Sciences

Volumn 179, Issue 9, 2009, Pages 1298-1318

  Ulaş, Aydin ^a   Semerci, Murat ^b   Yildiz, Olcay Taner ^c   Alpaydin, Ethem ^a  

^a BOGAZICI UNIVERSITY   (Turkey)

^b Rensselaer Polytechnic Institute   (United States)

^c ISIK UNIVERSITY   (Turkey)

Author keywords

Classification; Classifier ensembles; Classifier fusion; Discriminant ensembles; Diversity; Machine learning; Stacking; Voting

Indexed keywords

CLASSIFIERS; POLYNOMIAL APPROXIMATION; ROBOT LEARNING;

CLASSIFICATION; CLASSIFIER ENSEMBLES; CLASSIFIER FUSION; DISCRIMINANT ENSEMBLES; DIVERSITY; MACHINE LEARNING; STACKING; VOTING;

LEARNING SYSTEMS;

EID: 60349123276     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2008.12.024     Document Type: Article

References (65)

1. F. Alimoǧlu, E. Alpaydi{dotless}n, Combining multiple representations and classifiers for pen-based handwritten digit recognition, in: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR'97, 1997, pp. 637-641.
2. Alkoot F.M., and Kittler J. Experimental evaluation of expert fusion strategies. Pattern Recognition Letters 20 11-13 (1999) 1361-1369
3. Alpaydi{dotless}n E. Combined 5 × 2 cv F test for comparing supervised classification learning algorithms. Neural Computation 11 8 (1999) 1885-1892
4. A. Asuncion, D.J. Newman, UCI machine learning repository, 2007. .
5. Bakker B., and Heskes T. Clustering ensembles of neural network models. Neural Networks 16 2 (2003) 261-269
6. Banfield R.E., Hall L.O., Bowyer K.W., and Kegelmeyer W.P. Ensemble diversity measures and their application to thinning. Information Fusion 6 1 (2005) 49-62
7. Bertoni A., Folgieri R., and Valentini G. Bio-molecular cancer prediction with random subspace ensembles of support vector machines. Neurocomputing 63 (2005) 535-539
8. Breiman L. Bagging predictors. Machine Learning 24 2 (1996) 123-140
9. Cabrera J.B.D. On the impact of fusion strategies on classification errors for large ensembles of classifiers. Pattern Recognition 39 (2006) 1963-1978
10. R. Caruana, A. Niculescu-Mizil, G. Crew, A. Ksikes, Ensemble selection from libraries of models, in: Proceedings of the International Conference on Machine Learning, ICML'04, 2004, pp. 137-144.
11. C.-C. Chang, C.-J. Lin, LIBSVM: a library for support vector machines, 2001. .
12. Demir C., and Alpaydi{dotless}n E. Cost-conscious classifier ensembles. Pattern Recognition Letters 26 14 (2005) 2206-2214
13. Demsar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7 (2006) 1-30
14. Dietterich T.G. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10 7 (1998) 1895-1923
15. R.P. Duin, The combining classifier: to train or not to train?, in: Proceedings of the International Conference on Pattern Recognition, ICPR'02, 2002, pp. 765-770.
16. Y. Freund, R.E. Schapire, Experiments with a new boosting algorithm, in: Proceedings of the International Conference on Machine Learning, ICML'96, 1996, pp. 148-156.
17. Fumera G., Roli F., and Serrau A. A theoretical analysis of bagging as a linear combination of classifiers. IEEE Transactions on Pattern Analysis Machine Intelligence 30 7 (2008) 1293-1299
18. K.F. Goebel, W. Yan, Choosing classifiers for decision fusion, in: P. Svensson, J. Schubert (Eds.), Proceedings of the International Conference on Information Fusion, Fusion'04, vol. I, 2004, pp. 563-568.
19. Gökberk B., Dutaǧaci{dotless} H., Ulaş A., Akarun L., and Sankur B. Representation plurality and fusion for 3D face recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 38 1 (2008) 155-173
20. Hansen J.V. Combining predictors: comparison of five meta machine learning methods. Information Sciences 119 1-2 (1999) 91-105
21. Hastie T., Tibshirani R., and Friedman J. The Elements of Statistical Learning (2001), Springer-Verlag
22. Ho T.K. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 (1998) 832-844
23. Islam M., Yao X., and Murase K. A constructive algorithm for training cooperative neural network ensembles. IEEE Transactions on Neural Networks 14 (2003) 820-834
24. Jacobs R.A., Jordan M.I., Nowlan S.J., and Hinton G.E. Adaptive mixtures of local experts. Neural Computation 3 (1991) 79-87
25. Jain A., Nandakumar K., and Ross A. Score normalization in multimodal biometric systems. Pattern Recognition 38 (2005) 2270-2285
26. Y. Kim, W.N. Street, F. Menczer, Meta-evolutionary ensembles, in: Proceedings of the International Joint Conference on Neural Networks, IJCNN'02, vol. 3, 2002, pp. 2791-2796.
27. Kittler J., Hatef M., Duin R.P., and Matas J. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 3 (1998) 226-239
28. A.H.-R. Ko, R. Sabourin, A. de Souza Britto Jr., Combining diversity and classification accuracy for ensemble selection in random subspaces, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN'06), 2006, pp. 2144-2151.
29. A.H.-R. Ko, R. Sabourin, A. de Souza Britto Jr., A new objective function for ensemble selection in random subspaces, in: Proceedings of the International Conference on Pattern Recognition (ICPR'06), 2006, pp. 185-188.
30. Kuncheva L.I. A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 2 (2002) 281-286
31. Kuncheva L.I. Combining Pattern Classifiers: Methods and Algorithms (2004), Wiley-Interscience
32. Kuncheva L.I. Special issue on diversity in multiple classifier systems. Information Fusion 6 1 (2005) 1-115
33. Kuncheva L.I., and Rodriguez J.J. Classifier ensembles with a random linear oracle. IEEE Transactions on Knowledge and Data Engineering 19 4 (2007) 500-508
34. Kuncheva L.I., Skurichina M., and Duin R.P.W. An experimental study on diversity for bagging and boosting with linear classifiers. Information Fusion 3 4 (2002) 245-258
35. Kuncheva L.I., and Whitaker C.J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning 51 2 (2003) 181-207
36. Liu C.-L. Classifier combination based on confidence transformation. Pattern Recognition 38 (2005) 11-28
37. D.D. Margineantu, T.G. Dietterich, Pruning adaptive boosting, in: Proceedings of the International Conference on Machine Learning, ICML'97, 1997, pp. 211-218.
38. G. Martínez-Munoz, D. Hernández-Lobato, A. Suárez, An analysis of ensemble pruning techniques based on ordered aggregation, IEEE Transactions on Pattern Analysis Machine Intelligence 31(2) (2009) 245-259.
39. Martínez-Munoz G., and Suárez A. Using boosting to prune bagging ensembles. Pattern Recognition Letters 28 1 (2007) 156-165
40. Merz C.J. Using correspondence analysis to combine classifiers. Machine Learning 36 1-2 (1999) 33-58
41. Partridge D., and Yates W.B. Engineering multiversion neural-net systems. Neural Computation 8 4 (1996) 869-893
42. Pasi G., and Yager R.R. Modeling the concept of majority opinion in group decision making. Information Sciences 176 4 (2006) 390-414
43. Prodromidis A.L., and Stolfo S.J. Cost complexity-based pruning of ensemble classifiers. Knowledge and Information Systems 3 4 (2001) 449-469
44. C.E. Rasmussen, R.M. Neal, G. Hinton, D. van Camp, M. Revow, Z. Ghahramani, R. Kustra, R. Tibshirani, Delve data for evaluating learning in valid experiments, 1995-1996. .
45. Raudys S. Trainable fusion rules. I: Large sample size case. Neural Networks 19 (2006) 1506-1516
46. Raudys S. Trainable fusion rules. II: Small sample-size effects. Neural Networks 19 (2006) 1517-1527
47. Rokach L., Maimon O., and Arbel R. Selective voting: getting more for less in sensor fusion. International Journal of Pattern Recognition and Artificial Intelligence 20 3 (2006) 329-350
48. F. Roli, G. Giacinto, G. Vernazza, Methods for designing multiple classifier systems, in: Proceedings of the International Workshop on Multiple Classifier Systems, MCS'01, 2001, pp. 78-87.
49. Ruta D., and Gabrys B. Classifier selection for majority voting. Information Fusion 6 1 (2005) 63-81
50. Santos E.M.D., Sabourin R., and Maupin P. A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition 41 (2008) 2993-3009
51. A.J.C. Sharkey, N.E. Sharkey, U. Gerecke, G.O. Chandroth, The "test and select" approach to ensemble combination, in: Proceedings of the International Workshop on Multiple Classifier Systems, MCS'00, vol. 1857, 2000, pp. 30-44.
52. Sohn S.Y., and Shin H.W. Experimental study for the comparison of classifier combination methods. Pattern Recognition 40 (2007) 33-40
53. C. Tamon, J. Xiang, On the boosting pruning problem, in: Proceedings of the European Conference on Machine Learning, ECML'00, 2000, pp. 404-412.
54. Tax D.M.J., van Breukelen M., Duin R.P., and Kittler J. Combining multiple classifiers by averaging or multiplying?. Pattern Recognition 33 9 (2000) 1475-1485
55. Ting K.M., and Witten I.H. Issues in stacked generalization. Journal of Artificial Intelligence Research 10 (1999) 271-289
56. Ueda N. Optimal linear combination of neural networks for improving classification performance. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 2 (2000) 207-215
57. A. Ulaş, Incremental construction of cost-conscious ensembles using multiple learners and representations in machine learning, Ph.D. Thesis, Boǧaziçi University, 2008.
58. Wolpert D.H. Stacked generalization. Neural Networks 5 (1992) 241-259
59. Yang Y., Webb G.I., Cerquides J., Korb K.B., Boughton J., and Ting K.M. To select or to weigh: a comparative study of linear combination schemes for superparent-one-dependence estimators. IEEE Transactions on Knowledge and Data Engineering 19 12 (2007) 1652-1665
60. O.T. Yi{dotless}ldi{dotless}z, E. Alpaydi{dotless}n, Linear discriminant trees, in: Proceedings of the International Conference on Machine Learning, ICML'00, 2000, pp. 1175-1182.
61. O.T. Yi{dotless}ldi{dotless}z, A. Ulaş, M. Semerci, E. Alpaydi{dotless}n, AYSU: machine learning data sets for model combination, 2007. .
62. Zhang C.-X., and Zhang J.-S. RotBoost: a technique for combining rotation forest and AdaBoost. Pattern Recognition Letters 29 10 (2008) 1524-1536
63. Zhang Y., and Bhattacharyya S. Genetic programming in classifying large-scale data: an ensemble method. Information Sciences 163 1-3 (2004) 85-101
64. Zhang Y., Burer S., and Street W.N. Ensemble pruning via semi-definite programming. Journal of Machine Learning Research 7 (2006) 1315-1338
65. Zhou Z.-H., Wu J., and Tang W. Ensembling neural networks: many could be better than all. Artificial Intelligence 137 (2002) 239-263