An experimental study on diversity for bagging and boosting with linear classifiers

Information Fusion

Volumn 3, Issue 4, 2002, Pages 245-258

  Kuncheva, L I ^a   Skurichina, M ^b   Duin, R P W ^b  

^a BANGOR UNIVERSITY   (United Kingdom)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Bagging; Boosting; Combining classifier; Diversity

Indexed keywords

CLASSIFIERS; FUNCTIONS; NEURAL NETWORKS; PROBABILITY; VECTORS;

LINEAR DISCRIMINANT CLASSIFIERS;

LEARNING SYSTEMS;

EID: 0036896235     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1566-2535(02)00093-3     Document Type: Article

References (37)

1. Bauer E., Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning. 36:1999;105-142.
2. Breiman L. Bagging predictors. Machine Learning. 26(2):1996;123-140.
3. Breiman L. Arcing classifiers. The Annals of Statistics. 26(3):1998;801-849.
4. Breiman L. Combining predictors. Sharkey A.J.C. Combining Artificial Neural Nets. 1999;31-50 Springer-Verlag, London.
5. P. Cunningham, J. Carney. Diversity versus quality in classification ensembles based on feature selection, Technical Report TCD-CS-2000-02, Department of Computer Science, Trinity College Dublin, 2000.
6. Dietterich T.G. Ensemble methods in machine learning. Kittler J., Roli F. Multiple Classifier Systems. Lecture Notes in Computer Science. vol. 1857:2000;1-15 Springer.
7. Efron B., Tibshirani R. An Introduction to the Bootstrap. 1993;Chapman & Hall, NY.
8. Freund Y., Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences. 55(1):1997;119-139.
9. Freund Y., Schapire R.E. Discussion of the paper Arcing Classifiers by Leo Breiman. The Annals of Statistics. 26(3):1998;824-832.
10. Giacinto G., Roli F. Design of effective neural network ensembles for image classification processes. Image Vision and Computing Journal. 19(9-10):2001;699-707.
11. Golomb S.W., Baumert L.D. The search for Hadamard matrices. American Mathematics Monthly. 70:1963;12-17.
12. Hansen L.K., Salamon P. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence. 12(10):1990;993-1001.
13. S. Hashem, B. Schmeiser, Y. Yih, Optimal linear combinations of neural networks: an overview, in: IEEE International Conference on Neural Networks, Orlando, Florida, 1994, pp. 1507-1512.
14. Ho T.K. The random space method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence. 20(8):1998;832-844.
15. Kohavi R., Wolpert D.H. Bias plus variance decomposition for zero-one loss functions. Saitta L. Machine Learning: Proc. 13th International Conference. 1996;275-283 Morgan Kaufmann.
16. Krogh A., Vedelsby J. Neural network ensembles, cross validation and active learning. Tesauro G., Touretzky D.S., Leen T.K. Advances in Neural Information Processing Systems. vol. 7:1995;231-238 MIT Press, Cambridge, MA.
17. Kuncheva L.I. Using measures of similarity and inclusion for multiple classifier fusion by decision templates. Fuzzy Sets and Systems. 122(3):2001;401-407.
18. Kuncheva L.I. A theoretical study on expert fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence. 24(2):2002;281-286.
19. Kuncheva L.I., Bezdek J.C., Duin R.P.W. Decision templates for multiple classifier fusion: an experimental comparison. Pattern Recognition. 34(2):2001;299-314.
20. Kuncheva L.I., Whitaker C.J. Ten measures of diversity in classifier ensembles: limits for two classifiers. Proc. IEE Workshop on Intelligent Sensor Processing, Birmingham. 2001;10/1-10/6 IEE.
21. L.I. Kuncheva, C.J. Whitaker. Measures of diversity in classifier ensembles. Machine Learning, in press.
22. L.I. Kuncheva, C.J. Whitaker, C.A. Shipp, R.P.W. Duin, Is independence good for combining classifiers?, in: Proc. 15th International Conference on Pattern Recognition, Barcelona, Spain, vol. 2, 2000, pp. 169-171.
23. Lam L. Classifier combinations: implementations and theoretical issues. Kittler J., Roli F. Multiple Classifier Systems. Lecture Notes in Computer Science. vol. 1857:2000;78-86 Springer.
24. Y. Liu, X. Yao, Negatively correlated neural networks for classification, in: Proc. 3rd International Symposium on Artificial Life and Robotics (AROBIII'98), Japan, 1998, pp. 736-739.
25. Y. Liu, X. Yao, Simultaneous learning of negatively correlated neural network, in: Proc 9th Australian Conference on Neural Networks (ACNN'98), Brisbane, Australia, 1998, pp. 183-187.
26. Liu Y., Yao X. Ensemble learning via negative correlation. Neural Networks. 12:1999;1399-1404.
27. D. Partridge, W. Krzanowski, Distinct failure diversity in multiversion software, personal communication.
28. Partridge D., Krzanowski W.J. Software diversity: practical statistics for its measurement and exploitation. Information and Software Technology. 39:1997;707-717.
29. Rosen B.E. Ensemble learning using decorrelated neural networks. Connection Science. 8(3/4):1996;373-383.
30. Ruta D., Gabrys B. Application of the evolutionary algorithms for classifier selection in multiple classifier systems with majority voting. Kittler J., Roli F. Proc. Second International Workshop on Multiple Classifier Systems. Lecture Notes in Computer Science. vol. 2096:2001;Springer-Verlag.
31. D.B. Skalak, The sources of increased accuracy for two proposed boosting algorithms, in: Proc. American Association for Artificial Intelligence, AAAI-96, Integrating Multiple Learned Models Workshop, 1996.
32. M. Skurichina, Stabilizing Weak Classifiers, Ph.D. thesis, Delft University of Technology, Delft, The Netherlands, 2001.
33. Sneath P.H.A., Sokal R.R. Numerical Taxonomy. 1973;W.H. Freeman & Co.
34. Tumer K., Ghosh J. Error correlation and error reduction in ensemble classifiers. Connection Science. 8(3/4):1996;385-404.
35. Tumer K., Ghosh J. Linear and order statistics combiners for pattern classification. Sharkey A.J.C. Combining Artificial Neural Nets. 1999;127-161 Springer-Verlag, London.
36. Xu L., Krzyzak A., Suen C.Y. Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics. 22:1992;418-435.
37. Yule G.U. On the association of attributes in statistics. Philos. Trans., A. 194:1900;257-319.