Exploiting unlabeled data to enhance ensemble diversity

Data Mining and Knowledge Discovery

Volumn 26, Issue 1, 2013, Pages 98-129

  Zhang, Min Ling ^a,b   Zhou, Zhi Hua ^b  

^a SOUTHEAST UNIVERSITY   (China)

^b NANJING UNIVERSITY   (China)

Author keywords

Diversity; Ensemble learning; Machine learning; Unlabeled data

Indexed keywords

BASE LEARNERS; DATA SETS; DIVERSITY; ENSEMBLE LEARNING; ENSEMBLE METHODS; ERROR PRONES; GENERALIZATION ABILITY; LABELED DATA; SEMI-SUPERVISED; TRAINING DATA; UNLABELED DATA;

COMPUTER APPLICATIONS; DATA MINING;

LEARNING SYSTEMS;

EID: 84872409305     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10618-011-0243-9     Document Type: Article

References (48)

1. Bennett K, Demiriz A, Maclin R (2002) Exploiting unlabeled data in ensemble methods. In: Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining, Edmonton, Canada, pp 289-296
2. Blum A, Mitchell T (1998) Combining labeled and unlabeled data with co-training. In: Proceedings of the 11th annual conference on computational learning theory, Madison, WI, pp 92-100
3. Breiman L (1996) Bagging predictors. Mach Learn 24(2): 123-140
4. Brown G, Wyatt JL (2003) The use of the ambiguity decomposition in neural network ensemble learning methods. In: Proceedings of the 20th international conference on machine learning, Washington, DC, pp 67-74
5. Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge, MA
6. Chen HH, Yao X (2009) Regularized negative correlation learning for neural network ensemble. IEEE Trans Neural Netw 20(12): 1962-1979
7. Chen K, Wang S (2008) Regularized boost for semi-supervised learning. In: Platt JC, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems, vol 20. MIT Press, Cambridge, MA, pp 281-288
8. Chen K, Wang S (2011) Semi-supervised learning via regularized boosting working on multiple semi-supervised assumptions. IEEE Trans Pattern Anal Mach Intell 33(1): 129-143
9. Cunningham P, Carney J (2000) Diversity versus quality in classification ensembles based on feature selection. Technical Report TCD-CS-2000-02, Department of Computer Science, Trinity College Dublin, Dublin, Ireland
10. d'Alché Buc F, Grandvalet Y, Ambroise C (2002) Semi-supervised marginboost. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Cambridge, MA, pp 553-560
11. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1-30
12. Dietterich TG (2000) Ensemble methods in machine learning. In: Proceedings of the 1st international workshop on multiple classifier systems, Cagliari, Italy, pp 1-15
13. Efron B, Tibshirani R (1993) An introduction to the boostrap. Chapman & Hall, New York
14. Frank A, Asuncion A (2010) UCI machine learning repository. Technical report, School of Information and Computer Science, University of California, Irvine, CA. http://archive.ics.uci.edu/ml
15. Freund Y, Schapire RE (1995) A decision-theoretic generalization of on-line learning and an application to boosting. In: Vitányi PMB (ed) Lecture notes in computer science, vol 904. Springer, Berlin, pp 23-37
16. Giacinto G, Roli F (2001) Design of effective neural network ensembles for image classification processes. Image Vis Comput 19(9/10): 699-707
17. Hettich S, Bay SD (1998) The UCI KDD archive. Technical report, Department of Information and Computer Science, University of California, Irvine, CA. http://kdd.ics.uci.edu
18. Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, and active learning. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems, vol 7. MIT Press, Cambridge, MA, pp 231-238
19. Kuncheva LI, Whitaker CJ (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach Learn 51(2): 181-207
20. Li M, Zhou ZH (2005) SETRED: self-training with editing. In: Proceedings of the 9th Pacific-Asia conference on knowledge discovery and data mining, Hanoi, Vietnam, pp 611-621
21. Li M, Zhou ZH (2007) Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples. IEEE Trans Syst Man Cybern A Syst Hum 37(6): 1088-1098
22. Liu Y, Yao X (1999a) Ensemble learning via negative correlation. Neural Netw 12(10): 1399-1404
23. Liu Y, Yao X (1999b) Simultaneous training of negatively correlated neural networks in an ensemble. IEEE Trans Syst Man Cybern B Cybern 29(6): 716-725
24. Lu X, Jain AK (2004) Ethnicity identification from face images. In: Proceedings of SPIE international symposium on defense and security, Kissimmee, FL, pp 114-123
25. Mallapragada PK, Jin R, Jain AK, Liu Y (2009) Semiboost: boosting for semi-supervised learning. IEEE Trans Pattern Anal Mach Intell 31(11): 2000-2014
26. Mason L, Bartlett P, Baxter J, Frean M (2000) Functional gradient techniques for combining hypotheses. In: Smola A, Bartlett P, Schölkopf B, Schuurmans D (eds) Advances in large margin classifiers. MIT Press, Cambridge, pp 221-246
27. McKay R, Abbass H (2001) Analyzing anticorrelation in ensemble learning. In: Proceedings of 2001 conference on artificial neural networks and expert systems, Otago, New Zealand, pp 22-27
28. Melville P (2005) Creating diverse ensemble classifiers to reduce supervision. PhD thesis, Department of Computer Sciences, University of Texas at Austin, Austin, TX
29. Melville P, Mooney RJ (2003) Constructing diverse classifier ensembles using artificial training examples. In: Proceedings of the 18th international joint conference on artificial intelligence, Acapulco, Mexico, pp 505-510
30. Opitz DW (1999) Feature selection for ensembles. In: Proceedings of the 16th national conference on artificial intelligence, Orlando, FL, pp 379-384
31. Opitz DW, Shavlik JW (1996) Actively searching for an effective neural network ensemble. Connection Science 8(3-4): 337-353
32. Partridge D, Krzanowski WJ (1997) Software diversity: practical statistics for its measurement and exploitation. Inf Softw Technol 39(10): 707-717
33. Rosen BE (1996) Ensemble learning using decorrelated neural networks. Connect Sci 8(3): 373-383
34. Saffari A, Grabner H, Bischof H (2008) SERboost: Semi-supervised boosting with expectation regularization. In: Proceedings of the 10th European conference on computer vision, Marseille, France, pp 588-601
35. Saffari A, Leistner C, Bischof H (2009) Regularized multi-class semi-supervised boosting. In: Proceedings of the 2009 IEEE computer society conference on computer vision and pattern recognition, Miami, FL, pp 967-974
36. Skalak D (1996) The sources of increased accuracy for two proposed boosting algorithms. In: Working notes of AAAI-96 workshop on integrating multiple learned models for improving & scaling machine learning algorithms, Portland, OR
37. Valizadegan H, Jin R, Jain AK (2008) Semi-supervised boosting for multi-class classification. In: Proceedings of the 19th European conference on machine learning, Antwerp, Belgium, pp 522-537
38. Wilcoxon F (1945) Individual comparisons by ranking methods. Biometrics 1: 80-83
39. Wolpert DH (1992) Stacked generalization. Neural Netw 5(2): 241-259
40. Zhang ML, Zhou ZH (2010) Exploiting unlabeled data to enhance ensemble diversity. In: Proceedings of the 10th IEEE international conference on data mining, Sydney, Australia, pp 619-628
41. Zhang ML, Zhou ZH (in press) CoTrade: confident co-training with data editing. IEEE Trans Syst Man Cybern B Cybern
42. Zhou ZH (2009a) Ensemble learning. In: Li SZ (ed) Encyclopedia of biometrics. Springer, Berlin
43. Zhou ZH (2009b) When semi-supervised learning meets ensemble learning. In: Proceedings of the 8th international workshop on multiple classifier systems, Reykjavik, Iceland, pp 529-538
44. Zhou ZH, Jiang Y (2004) NeC4.5: neural ensemble based C4.5. IEEE Trans Knowl Data Eng 16(6): 770-773
45. Zhou ZH, Li M (2005) Tri-training: exploiting unlabeled data using three classifiers. IEEE Trans Knowl Data Eng 17(11): 1529-1541
46. Zhou ZH, Li M (2010) Semi-supervised learning by disagreement. Knowl Inf Syst 24(3): 415-439
47. Zhou ZH, Zhan DC, Yang Q (2007) Semi-supervised learning with very few labeled training examples. In: Proceedings of the 22nd AAAI conference on artificial intelligence, Vancouver, Canada, pp 675-680
48. Zhu X (2006) Semi-supervised learning literature survey. Technical report 1530, Department of Computer Science, University of Wisconsin at Madison, Madison, WI