An analysis of diversity measures

Machine Learning

Volumn 65, Issue 1, 2006, Pages 247-271

  Tang, E K ^a   Suganthan, P N ^a   Yao, X ^b  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b UNIVERSITY OF BIRMINGHAM   (United Kingdom)

Author keywords

Classifier ensemble; Coincident failure diversity; Disagreement measure; Diversity measures; Double fault measure; Entropy measure; Generalized diversity; Interrater agreement; KW variance; Majority vote; Margin distribution; Measure of difficulty

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTATIONAL METHODS; ENTROPY; INFORMATION ANALYSIS; LEARNING SYSTEMS;

CLASSIFIER ENSEMBLE; DISAGREEMENT MEASURES; DIVERSITY MEASURES; DOUBLE FAULT MEASURES; ENTROPY MEASURES; GENERALIZED DIVERSITY; INTERRATER AGREEMENTS; KW VARIANCES; MAJORITY VOTES; MARGIN DISTRIBUTION;

LEARNING ALGORITHMS;

EID: 33749018252     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-006-9449-2     Document Type: Article

References (30)

1. Atukorale, A. S., Downs, T., & Suganthan, P. N. (2003). Boosting HONG Networks. Neurocomputing, 51, 75-86.
2. Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 36, 105-142.
3. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123-140.
4. Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
5. Brown, G., Wyatt, J. L., Harris, R. & Yao, X. (2004). Diversity Creation Methods: A Survey and Categorization. Information Fusion Journal (Special issue on Diversity in Multiple Classifier Systems), 6(1), 5-20.
6. Burges, C. J. C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121-167.
7. Dietterich, T. (2000). An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Machine Learning, 40(2), 1-22.
8. Fleiss, J. (1981). Statistical methods for rates and proportions. John Wiley & Sons.
9. Freund, Y. (1995). Boosting a weak learning algorithm by majority. Information and Computation, 121(2), 256-285.
10. Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In: Proc. 13th Int. Conference on Machine Learning (pp. 148-156). Morgan Kaufmann.
11. Giacinto, G., & Roli, F. (2001). Design of effective neural network ensembles for image classification processes. Image Vision and Computing, 19(9/10), 699-707.
12. Hansen, L. & Salamon, P. (1990). Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10), 993-1001.
13. Ho, T. (1998). The random space method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8), 832-844.
14. Kohavi, R., & Wolpert, D. (1996). Bias plus variance decomposition for zero-one loss functions. In: L. Saitta (Ed.), Proc. 13th Int. Conference on Machine Learning (pp. 275-283). Morgan Kaufmann.
15. Krogh, A., & Vedelsby, J. (1995). Neural network ensembles, cross validation, and active Learning. In: G. Tesauro, D. S. Touretzky and T. K. Leen (Eds.), Advances in Neural Information Processing Systems 7 (pp. 231-238). Cambridge, MA: MIT Press.
16. Kuncheva, L., & Whitaker, C. (2003a). Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51, 181-207.
17. Kuncheva, L.I. (2003b). That elusive diversity in classifier ensembles. In: Proc IbPRIA 2003, Mallorca, Spain, 2003, Lecture Notes in Computer Science, Springer-Verlag, LNCS 2652, 1126-1138.
18. Liu, Y., & Yao, X. (1997). Negatively correlated neural networks can produce best ensembles. Australian Journal of Intelligent Information Processing Systems, 4(3/4), 176-185.
19. Liu, Y., Yao, X., & Higuchi, T. (2000). Evolutionary ensembles with negative correlation learning. IEEE Transactions on Evolutionary Computation, 4, 380-387.
20. Margineantu, D., & Dietterich, T. (1997). Pruning Adaptive Boosting. In: Proceedings ICML'97: International Conference on Machine Learning (pp. 211-218). Los Altos, CA: Morgan Kaufmann.
21. Mason, L., Bartlett, P. L., & Baxter, J. (2000). Improved generalization through explicit optimization of margins. Machine Learning, 38(3), 243-255.
22. Patridge, D., & Krzanowski, W. J. (1997). Software diversity: Practical statistics for its measurement and exploitation. Information & Software Technology, 39, 707-717.
23. Rätsch, G., Onoda, T., & Müller, K.-R. (2001). Soft margins for AdaBoost. Machine Learning, 42(3), 287-320.
24. Schapire, R. E., Freund, Y., Bartlett, P. L., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5), 1651-1686.
25. Schapire, R. (1999). Theoretical views of boosting. In: Proc. 4th European Conference on Computational Learning Theory (pp. 1-10).
26. Skalak, D. (1996). The sources of increased accuracy for two proposed boosting algorithms. In: Proc. American Association for Artificial Intelligence, AAAI-96, Integrating Multiple Learned Models Workshop.
27. Suganthan, P. N. (1999). Hierarchical Overlapped SOM's for Pattern Classification. IEEE Transactions on Neural Networks, 10(1), 193-196.
28. Suykens, J. A. K., Van Gestel, T., De Brabanter, J., De Moor, B., & Vandewalle, J. (2002). Least Squares Support Vector Machines Singapore. World Scientific.
29. Tamon, C., & Xiang, J. (2000). On the Boosting Pruning problem. In: R. L. Mantaras and E. Plaza (Eds.), Machine Learning: Proc. 11th European Conference, Vol. 1810 Lecture Notes in Computer Science (pp. 404-412). Springer.
30. Vapnik, V. (1995). The Nature of Statistical Learning Theory. Berlin: Springer.