A robust ensemble approach to learn from positive and unlabeled data using SVM base models

Neurocomputing

Volumn 160, Issue , 2015, Pages 73-84

  Claesen, Marc ^a   De Smet, Frank ^a   Suykens, Johan A K ^a   De Moor, Bart ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Classification; Ensemble learning; Semi supervised learning; Support vector machine

Indexed keywords

CLASSIFICATION (OF INFORMATION); STATISTICAL METHODS; SUPPORT VECTOR MACHINES;

BENCHMARK DATA; BINARY CLASSIFIERS; ENSEMBLE APPROACHES; ENSEMBLE LEARNING; FALSE POSITIVE; SEMI-SUPERVISED; STATE-OF-THE-ART APPROACH; UNLABELED DATA;

SEMI-SUPERVISED LEARNING;

ARTICLE; BOOTSTRAPPING; CLASS WEIGHTED SUPPORT VECTOR MACHINE; CLASSIFICATION ALGORITHM; INFORMATION PROCESSING; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; PRIORITY JOURNAL; ROBUST ENSEMBLE OF SUPPORT VECTOR MACHINE; SIMULATION; SUPPORT VECTOR MACHINE; WILCOXON SIGNED RANKS TEST;

EID: 84927956192     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.10.081     Document Type: Article

References (46)

1. Aerts Stein, Lambrechts Diether, Maity Sunit, Van Loo Peter, Coessens Bert, DeSmet Frederik, Tranchevent Leon-Charles, De Moor Bart, Marynen Peter, Hassan Bassem, Carmeliet Peter, Moreau Yves Gene prioritization through genomic data fusion. Nat. Biotechnol. 2006, 24(May (5)):537-544. ISSN: 1087-0156.
2. Carlos Alzate, Johan A.K. Suykens, A semi-supervised formulation to binary kernel spectral clustering, in: 2012 IEEE World Congress on Computational Intelligence (IEEE WCCI/IJCNN 2012) Brisbane, Australia, June 2012.
3. Banfield Robert E., Hall Lawrence O., Bowyer Kevin W., Kegelmeyer W.Philip A comparison of decision tree ensemble creation techniques. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29(1):173-180.
4. Bauer Eric, Kohavi Ron An empirical comparison of voting classification algorithms. bagging, boosting, and variants. Mach. Learn. 1999, 36(1-2):105-139.
5. Bergstra James, Bengio Yoshua Random search for hyper-parameter optimization. J. Mach. Learn. Res. 2012, 13(1):281-305.
6. Blackard Jock A., Dean Denis J. Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Comput. Electron. Agric. 1999, 24(December (3)):131-151.
7. Blanchard Gilles, Lee Gyemin, Scott Clayton Semi-supervised novelty detection. J. Mach. Learn. Res. 2010, 11:2973-3009.
8. Léon Bottou, Chih-Jen Lin, Support Vector Machine Solvers: Large Scale Kernel Machines, 2007, pp. 301-320.
9. Breiman Leo. Bagging predictors. Mach. Learn. 1996, 24(August (2)):123-140. ISSN 0885-6125.
10. Breiman Leo. Randomizing outputs to increase prediction accuracy. Mach. Learn. 2000, 40(3):229-242.
11. Breiman Leo. Random forests. Mach. Learn. 2001, 45(1):5-32.
12. Brown Gavin, Wyatt Jeremy, Harris Rachel, Yao Xin Diversity creation methods. a survey and categorisation. Inf. Fusion 2005, 6(1):5-20.
13. Gavin C. Cawley, Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs, in: International Joint Conference on Neural Networks, IJCNN[U+05F3]06, 2006, IEEE, 2006, pp. 1661-1668.
14. Chang Chih-Chung, Lin Chih-Jen LIBSVM. a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2011, 2:27:1-27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
15. Claesen Marc, De Smet Frank, Suykens Johan A.K., De Moor Bart EnsembleSVM. a library for ensemble learning using support vector machines. J. Mach. Learn. Res. 2014, 15:141-145. URL http://jmlr.org/papers/v15/claesen14a.html.
16. Daemen Anneleen, Gevaert Olivier, Ojeda Fabian, Debucquoy Annelies, Suykens Johan A.K., Sempoux Christine, Machiels Jean-Pascal, Haustermans Karin, De Moor Bart A kernel-based integration of genome-wide data for clinical decision support. Genome Med. 2009, 1(4):39.
17. Jesse Davis, Mark Goadrich, The relationship between precision-recall and ROC curves, in: Proceedings of the 23rd International Conference on Machine Learning ICML [U+05F3]06, New York, NY, USA, 2006, ACM, pp. 233-240. ISBN 1-59593-383-2.
18. Demšar Janez Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.
19. Dietterich Thomas G. An experimental comparison of three methods for constructing ensembles of decision trees. bagging, boosting, and randomization. Mach. Learn. 2000, 40(2):139-157.
20. Duarte Marco F., Hu Yu Hen. Vehicle classification in distributed sensor networks. J. Parallel Distrib. Comput. 2004, 64(7):826-838.
21. Charles Elkan, Keith Noto, Learning classifiers from only positive and unlabeled data, in: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD [U+05F3]08, New York, NY, USA, 2008, ACM, pp. 213-220.
22. Fan Rong-En, Chang Kai-Wei, Hsieh Cho-Jui, Wang Xiang-Rui, Lin Chih-Jen LIBLINEAR. a library for large linear classification. J. Mach. Learn. Res. 2008, 9:1871-1874.
23. Frenay B., Verleysen M. Classification in the presence of label noise. a survey. IEEE Trans. Neural Netw. Learn. Syst. 2014, 25(May (5)):845-869. ISSN 2162-237X. 10.1109/TNNLS.2013.2292894.
24. Grandvalet Yves Bagging equalizes influence. Mach. Learn. 2004, 55(3):251-270.
25. Keijzer Maarten, Babovic Vladan Genetic programming. Genetic Programming, Lecture Notes in Computer Science 2000, vol. 1802:76-90. Springer, Berlin, Heidelberg, ISBN 978-3-540-67339-2, URL http://dx.doi.org/10.1007/978-3-540-46239-2_6. 10.1007/978-3-540-46239-2_6.
26. Ar Lazarevic, Aysel Ozgur, Levent Ertoz, Jaideep Srivastava, Vipin Kumar, A comparative study of anomaly detection schemes in network intrusion detection, in: Proceedings of the Third SIAM International Conference on Data Mining, 2003.
27. Yann LeCun Léon Bottou, Yoshua Bengio, Patrick Haffner, Gradient-based learning applied to document recognition, in: Proceedings of the IEEE, 1998, pp. 2278-2324.
28. Wee Sun Lee, Bing Liu, Learning with positive and unlabeled examples using weighted logistic regression, in: Proceedings of the Twentieth International Conference on Machine Learning (ICML), 2003, pp. 448-455.
29. Xiaoli Li, Bing Liu, Learning to classify texts using positive and unlabeled data, in: IJCAI[U+05F3]03: Proceedings of the 18th International Joint Conference on Artificial Intelligence, San Francisco, CA, USA, Morgan Kaufmann Publishers Inc, 2003, pp. 587-592.
30. Bing Liu, Wee Sun Lee, Philip S. Yu, Xiaoli Li, Partially supervised classification of text documents, in: ICML [U+05F3]02: Proceedings of the Nineteenth International Conference on Machine Learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2002, pp. 387-394, ISBN 1-55860-873-7.
31. Bing Liu, Yang Dai, Xiaoli Li, Wee Sun Lee, Philip S. Yu, Building text classifiers using positive and unlabeled examples, in: Proceedings of the Third IEEE International Conference on Data Mining ICDM [U+05F3]03, IEEE Computer Society, Washington, DC, USA, 2003, pp. 179-186. ISBN 0-7695-1978-4.
32. Zhigang Liu, Wenzhong Shi, Deren Li, Qianqing Qin, Partially supervised classification-based on weighted unlabeled samples support vector machine, in: Proceedings of the First International Conference on Advanced Data Mining and Applications, ADMA[U+05F3]05, Springer-Verlag, Berlin, Heidelberg, 2005, pp. 118-129. ISBN 3-540-27894-X, 978-3-540-27894-8.
33. Martínez-Muñoz Gonzalo., Suárez Alberto. Out-of-bag estimation of the optimal sample size in bagging. Pattern Recognit. 2010, 43(1):143-152.
34. Fantine Mordelet Jean-Philippe Vert, A bagging SVM to learn from positive and unlabeled examples. Pattern Recognition Letters 2014, 37:201-209.
35. Mordelet Fantine, Vert Jean-Philippe ProDiGe. prioritization of disease genes with multitask machine learning from positive and unlabeled examples. BMC Bioinform. 2011, 12(1):389.
36. Peter Nemenyi, Distribution-free multiple comparisons, in: Biometrics, vol. 18, page 263. International Biometric Soc 1441 I ST, NW, Suite 700, Washington, DC 20005-2210, 1962.
37. Edgar Osuna, Robert Freund, Federico Girosi, Support Vector Machines: Training and Applications, Technical Report AIM-1602, 1997.
38. Mykola Pechenizkiy, Alexey Tsymbal, Seppo Puuronen, Oleksandr Pechenizkiy, Class noise and supervised learning in medical domains: the effect of feature extraction, in: 19th IEEE International Symposium on IEEE Computer-Based Medical Systems, 2006, CBMS 2006, 2006, pp. 708-713.
39. Danil Prokhorov, IJCNN 2001 Neural Network Competition, Slide presentation in IJCNN, 2001.
40. J. Sunil Rao, Robert Tibshirani, The Out-of-Bootstrap Method for Model Averaging and Selection, University of Toronto, 1997.
41. Shoichet Brian.K. Virtual screening of chemical libraries. Nature 2004, 432(December (7019)):862-865. ISSN 0028-0836.
42. Sifrim Alejandro, Popovic Dusan, Tranchevent Léon-Charles, Arderschirdavani Amin, Sakai Ryo, Konings Peter, Vermeesch Joris, Aerts Jan, De Moor Bart, Moreau Yves eXtasy. variant prioritization by genomic data fusion. Nat. Methods 2013, 10:1083-1084. http://dx.doi.org/10.1038/nmeth.2656.
43. Yu Hwanjo Single-class classification with mapping convergence. Mach. Learn. 2005, 61(November (1-3)):49-69. ISSN 0885-6125.
44. Hwanjo Yu, Jiawei Han, Kevin C. Chang, PEBL: positive example based learning for web page classification using SVM, in: KDD [U+05F3]02: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM Press, New York, NY, USA, 2002, pp. 239-248. ISBN 158113567X.
45. Zhu Xingquan, Wu Xindong Class noise vs. attribute noise. a quantitative study. Artif. Intell. Rev. 2004, 22(3):177-210.
46. Claesen Marc, Simm Jaak, Popovic Dusan, Moreau Yves, De Moor Bart Easy Hyperparameter Search Using Optunity. CoRR 2014, volume (abs/1412.1114) http://arxiv.org/abs/1412.1114.