Assessing the Performance of Classification Methods

International Statistical Review

Volumn 80, Issue 3, 2012, Pages 400-414

  Hand, David J ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

AUC; Diagnosis; Error rate; H measure; Performance evaluation; ROC curve; Sensitivity; Signal detection; Specificity; Supervised classification

Indexed keywords

EID: 84869883076     PISSN: 03067734     EISSN: 17515823     Source Type: Journal    
DOI: 10.1111/j.1751-5823.2012.00183.x     Document Type: Article

References (34)

1. Baker, S.G., Cook, N.R., Vickers, A. & Kramer, B.S. (2009). Using relative utility curves to evaluate risk prediction. J. R. Statist. Soc., 172, 729-748.
2. Briggs, W.M. & Zaretski, R. (2008). The skill plot: A graphical technique for evaluating continuous diagnostic tests, Biometrics, 64, 250-261.
3. Cortes, C. & Mohri, M. (2003). AUC optimization vs error rate minimization. In Advances in Neural Information Processing Systems, 16. Eds. S. Thrun, L. Saul & B. Schölkopf Cambridge, Massachusetts : MIT Press. Available at: http://books.nips.cc/nips16.html.
4. rd International Conference on Machine Learning, 148, pp. 233-240. New York : ACM.
5. Drummond, C. & Holte, R.C. (2006). Cost curves: An improved method for visualizing classifier performance. Mach. Learn., 65, 95-130.
6. Egan, J.P. (1975). Signal Detection Theory and ROC Analysis New York : Academic Press.
7. Friedman, J.H. (1997). On bias, variance, 0/1-loss, and the curse of dimensionality. Data Mining Knowledge Discov., 1, 55-77.
8. Gönen, M. (2007). Analyzing Receiver Operating Characteristic Curves with SAS Cary, NC : SAS Institute Inc.
9. Gneiting, T. & Raftery, A.E. (2007). Strictly proper scoring rules, prediction, and estimation. J. Amer. Statist. Assoc., 102, 359-378.
10. Hand, D.J. (1981). Discrimination and Classification Chichester : Wiley.
11. Hand, D.J. (1994). Deconstructing statistical questions (with discussion). J. R. Statist. Soc., 157, 317-356.
12. Hand, D.J. (1997). Construction and Assessment of Classification Rules Chichester : Wiley.
13. Hand, D.J. (2002). Good practice in retail credit scorecard assessment. J. Oper. Res. Soc., 56, 1109-1117.
14. Hand, D.J. (2006). Classifier technology and the illusion of progress (with discussion). Statist. Sci., 21, 1-34.
15. Hand, D.J. (2009). Measuring classifier performance: A coherent alternative to the area under the ROC curve. Mach. Learn., 77, 103-123
16. Hand, D.J. (2010). Evaluating diagnostic tests: The area under the ROC curve and the balance of errors. Statist. Med., 29, 1502-1510.
17. Hand, D.J. & Anagnostopoulos, C. (2012). When is the area under the ROC curve and appropriate measure of classifier performance. Technical Report, Department of Mathematics, Imperial College, London.
18. Hand, D.J. & Anagnostopoulos, C. (2012b). A better beta for the H measure. Available at: http://arxiv.org/abs/1202.2564.
19. Hastie, T., Tibshirani, R. & Friedman J. (2009). The Elements of Statistical Learning, 2nd ed. New York : Springer.
20. Inman, H.F. & Bradley, E.L. (1989). The overlapping coefficient as a measure of agreement between probability distributions and point estimation of the overlap of two normal densities. Commun. Statist., Part A-Theory Meth. 18, 3851-3874.
21. Jamain, A. (2004). Meta analysis of Classification Methods PhD thesis, Department of Mathematics, Imperial College, London
22. Jamain, A. & Hand, D.J. (2008). Mining supervised classification performance studies: A meta-analytic investigation. J. Classificat., 25, 87-112.
23. Krzanowski, W.J. & Hand, D.J. (2009). ROC Analysis for Continuous Data London : Chapman and Hall.
24. McLish, D.K. (1989). Analyzing a portion of the ROC curve. Med. Decision Making, 9, 190-195.
25. Pepe, M.S. (2000). The Statistical Evaluation of Medical Tests for Classification and Prediction Oxford : Oxford University Press.
26. Rom, D.R. & Hwang, E. (1996). Testing for individual and population equivalence based on the proportion of similar responses. Statist. Med., 15, 1489-1505.
27. Stine, R.A. & Heyse, J.F. (2001). Non-parametric estimates of overlap. Statist. Med., 20, 215-236.
28. Walter, S.D. (2005). The partial area under the summary ROC curve. Statist. Med., 24, 2025-2040.
29. Webb, A. (2002). Statistical Pattern Recognition, 2nd ed. Chichester : Wiley.
30. Wieand, S., Gail, M.H., James, B.R. & James, K.L. (1989). A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data. Biometrika, 76, 585-592.
31. Williams, P., Hand, D.J. & Tarnopolsky, A. (1982). The problem of screening for uncommon disorders-A comment on the Eating Attitudes Test. Psychol. Med., 12, 431-434.
32. Winkler, R.L. (1969). Scoring rules and the evaluation of probability assessors. J. Amer. Statist. Assoc., 64, 1073-1078.
33. Youden, W.J. (1950). Index for rating diagnostic tests. Cancer, 3, 32-35.
34. Zhou, X.-H., Obuchowski, N.A. & McClish, D.K. (2002). Statistical Methods in Diagnostic Medicine New York : Wiley.