On the application of ROC analysis to predict classification performance under varying class distributions

Machine Learning

Volumn 58, Issue 1, 2005, Pages 25-32

  Webb, Geoffrey I ^a   Ting, Kai Ming ^b  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITY OF BALLARAT   (Australia)

Author keywords

Model evaluation; ROC analysis

Indexed keywords

CLASSIFIERS; ERROR ANALYSIS; FUNCTIONS; LEARNING ALGORITHMS; MATHEMATICAL MODELS; PROBABILITY DISTRIBUTIONS; ROBUSTNESS (CONTROL SYSTEMS); SET THEORY;

CLASS DISTRIBUTION; COST FUNCTIONS; MODEL EVALUATION; ROC ANALYSIS;

LEARNING SYSTEMS;

EID: 14844366200     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10994-005-4257-7     Document Type: Article

References (12)

1. Adams, N. M., & Hand, D. J. (1999). Comparing classifiers when the misallocation costs are uncertain. Pattern Recognition, 32, 1139-1147.
2. Bartlett, P., Ben-David, S., & Kulkarni, S. (2000). Learning changing concepts by exploiting the structure of change. Machine Learning, 41, 153-174.
3. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30:7, 1145-1159.
4. Duda, O. R., Hart, P. E., & Stork, D. G. (2001). Pattern classification. John Wiley.
5. Flach, P. (2003). The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Proceedings of The Twentieth International Conference on Machine Learning.
6. Holte, R. C. (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11:1, 63-90.
7. Provost, F., & Fawcett, T. (1997). Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (pp. 43-48).
8. Provost, F., & Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42, 203-231.
9. Provost, F., Fawcett, T., & Kohavi, R. (1998). The case against accuracy estimation for comparing induction algorithms. In Proceedings of The Fifteenth International Conference on Machine Learning (pp. 43-48). San Francisco: Morgan Kaufmann.
10. Quinlan, J. R. (1987). Learning decision trees. Machine Learning, 1:1, 1-25.
11. Swets, J. A. (1988). Measuring the accuracy of diagnostic systems. Science, 240, 1285-1293.
12. Weinstein, M. C., & Fineberg, H. V. (1980). Clinical decision analysis. Saunders.