Recovering true classifier performance in positive-unlabeled learning

31st AAAI Conference on Artificial Intelligence, AAAI 2017

Volumn , Issue , 2017, Pages 2066-2072

  Jain, Shantanu ^a   White, Martha ^a   Radivojac, Predrag ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION MODELS; CLASSIFIER PERFORMANCE; CORRECTION APPROACHES; LABELED AND UNLABELED DATA; OPTIMAL CLASSIFIERS; PERFORMANCE MEASURE; RECEIVER OPERATING CHARACTERISTIC CURVES; STATE-OF-THE-ART ALGORITHMS;

ARTIFICIAL INTELLIGENCE;

EID: 85024505461     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Blanchard, G.; Lee, G.; and Scott, C. 2010. Semi-supervised novelty detection. J Mach Learn Res 11: 2973-3009.
2. Blum, A., and Mitchell, T. 1998. Combining labeled and unlabeled data with co-training. In Proceedings of the 11th Annual Conference on Computational Learning Theory, COLT 1998, 92-100.
3. Boyd, K.; Costa, V. S.; Davis, J.; and Page, C. D. 2012. Unachievable region in precision-recall space and its effect on empirical evaluation. In Proceedings of the 29th International Conference on Machine Learning, ICML 2012, 639-646.
4. Breiman, L. 1996. Bagging predictors. Mach Learn 24: 123-140.
5. Claesen, M.; Davis, J.; De Smet, F.; and De Moor, B. 2015. Assessing binary classifiers using only positive and unlabeled data. arXiv preprint arXiv: 1504. 06837.
6. Clark, W. T., and Radivojac, P. 2013. Information-theoretic evaluation of predicted ontological annotations. Bioinformatics 29 (13): i53-i61.
7. Cortes, C.; Mohri, M.; Riley, M.; and Rostamizadeh, A. 2008. Sample selection bias correction theory. In Proceedings of the 19th International Conference on Algorithmic Learning Theory, ALT 2008, 38-53.
8. Davis, J., and Goadrich, M. 2006. The relationship between precision-recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, 233-240.
9. Denis, F.; Gilleron, R.; and Letouzey, F. 2005. Learning from positive and unlabeled examples. Theor Comput Sci 348 (16): 70-83.
10. du Plessis, M. C., and Sugiyama, M. 2014. Class prior estimation from positive and unlabeled data. IEICE Trans Inf & Syst E97-D (5): 1358-1362.
11. du Plessis, M. C.; Niu, G.; and Sugiyama, M. 2015. Class-prior estimation for learning from positive and unlabeled data. In Proceedings of the 7th Asian Conference on Machine Learning, volume 45 of ACML 2015, 221-236.
12. Elkan, C., and Noto, K. 2008. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2008, 213-220.
13. Elkan, C. 2001. The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence, IJCAI 2001, 973-978.
14. Fawcett, T. 2006. An introduction to ROC analysis. Pattern Recogn Lett 27: 861-874.
15. Flach, P. A., and Kull, M. 2015. Precision-recall-gain curves: PR analysis done right. In Advances in Neural Information Processing Systems, NIPS 2015, 838-846.
16. Flach, P. A. 2003. The geometry of ROC space: understanding machine learning metrics through ROC isometrics. In Proceedings of the 20th International Conference on Machine Learning, ICML 2003, 194-201.
17. Hanley, J., and McNeil, B. J. 1982. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143 (1): 29-36.
18. Hastie, T.; Tibshirani, R.; and Friedman, J. H. 2001. The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer Verlag.
19. Hernández-Orallo, J.; Flach, P.; and Ferri, C. 2012. A unified view of performance metrics: translating threshold choice into expected classification loss. J Mach Learn Res 13 (1): 2813-2869.
20. Iakoucheva, L. M.; Radivojac, P.; Brown, C. J.; O'Connor, T. R.; Sikes, J. G.; Obradovic, Z.; and Dunker, A. K. 2004. The importance of intrinsic disorder for protein phosphorylation. Nucleic Acids Res 32 (3): 1037-1049.
21. Jain, S.; White, M.; Trosset, M. W.; and Radivojac, P. 2016. Nonparametric semi-supervised learning of class proportions. arXiv preprint arXiv: 1601. 01944.
22. Jain, S.; White, M.; and Radivojac, P. 2016. Estimating the class prior and posterior from noisy positives and unlabeled data. In Advances in Neural Information Processing Systems, NIPS 2016, 2685-2693.
23. Jiang, Y.; Clark, W. T.; Friedberg, I.; and Radivojac, P. 2014. The impact of incomplete knowledge on the evaluation of protein function prediction: a structured-output learning perspective. Bioinformatics 30 (17): i609-i616.
24. Lichman, M. 2013. UCI Machine Learning Repository.
25. Menon, A. K.; van Rooyen, B.; Ong, C. S.; and Williamson, R. C. 2015. Learning from corrupted binary labels via class-probability estimation. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, 125-134.
26. Peng, K.; Vucetic, S.; Han, B.; Xie, H.; and Obradovic, Z. 2003. Exploiting unlabeled data for improving accuracy of predictive data mining. In Proceedings of the 3rd IEEE International Conference on Data Mining, ICDM 2003, 267-274.
27. Ramaswamy, H. G.; Scott, C.; and Tewari, A. 2016. Mixture proportion estimation via kernel embedding of distributions. arXiv preprint arXiv: 1603. 02501.
28. Riedmiller, M., and Braun, H. 1993. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In Proceedings of the IEEE International Conference on Neural Networks, ICNN 1993, 586-591.
29. Saerens, M.; Latinne, P.; and Decaestecker, C. 2002. Adjusting the outputs of a classifier to new a priori probabilities: a simple procedure. Neural Comput 14: 21-41.
30. Sanderson, T., and Scott, C. 2014. Class proportion estimation with application to multiclass anomaly rejection. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics, AISTATS 2014, 850-858.
31. Scott, C.; Blanchard, G.; and Handy, G. 2013. Classification with asymmetric label noise: consistency and maximal denoising. J Mach Learn Res W&CP 30: 489-511.
32. Ward, G.; Hastie, T.; Barry, S.; Elith, J.; and Leathwick, J. 2009. Presence-only data and the EM algorithm. Biometrics 65 (2): 554-563.
33. Whalen, A. D. 1971. Detection of signals in noise. New York, NY: Academic press.