A comparison of MCC and CEN error measures in multi-class prediction

PLoS ONE

Volumn 7, Issue 8, 2012, Pages

  Jurman, Giuseppe ^a   Riccadonna, Samantha ^a   Furlanello, Cesare ^a  

^a FONDAZIONE BRUNO KESSLER   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPARATIVE STUDY; CONFUSION ENTROPY; CORRELATION COEFFICIENT; ENTROPY; INFORMATION PROCESSING; MATTHEWS CORRELATION COEFFICIENT; PREDICTION; STATISTICAL ANALYSIS;

ALGORITHMS; AREA UNDER CURVE; ARTIFICIAL INTELLIGENCE; COMPUTATIONAL BIOLOGY; COMPUTERS; ENTROPY; GENOTYPE; MODELS, STATISTICAL; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS; PROBABILITY; REPRODUCIBILITY OF RESULTS; ROC CURVE; SOFTWARE; STATISTICS AS TOPIC;

EID: 84864668842     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0041882     Document Type: Article

References (35)

1. Demšar J, (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research 7: 1-30.
2. García S, Herrera F, (2008) An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons. Journal of Machine Learning Research 9: 2677-2694.
3. Hand D, (2009) Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning 77: 103-123.
4. Sokolova M, Lapalme G, (2009) A systematic analysis of performance measures for classification tasks. Information Processing and Management 45: 427-437.
5. Ferri C, Hernández-Orallo J, Modroiu R, (2009) An experimental comparison of performance measures for classification. Pattern Recognition Letters 30: 27-38.
6. Felkin M (2007) Comparing Classification Results between N-ary and Binary Problems. In: Studies in Computational Intelligence, Springer-Verlag, volume 43. pp. 277-301.
7. Diri B, Albayrak S, (2008) Visualization and analysis of classifiers performance in multi-class medical data. Expert Systems with Applications 34: 628-634.
8. Hanley J, McNeil B, (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143: 29-36.
9. Bradley A, (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 30: 1145-1159.
10. Everson R, Fieldsend J, (2006) Multi-class ROC analysis from a multi-objective optimisation perspective. Pattern Recognition Letters 27: 918-927.
11. Landgrebe T, Duin R (2005) On Neyman-Pearson optimisation for multiclass classifiers. In: Proceedings 16th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 165-170.
12. Landgrebe T, Duin R (2006) A simplified extension of the Area under the ROC to the multiclass domain. In: Proceedings 17th Annual Symposium of the Pattern Recognition Association of South Africa. PRASA, pp. 241-245.
13. Landgrebe T, Duin R, (2008) Efficient multiclass ROC approximation by decomposition via confusion matrix perturbation analysis. IEEE Transactions Pattern Analysis Machine Intelligence 30: 810-822.
14. Ferri C, Hernández-Orallo J, Salido M (2003) Volume under the ROC surface for multi-class problems. In: Proceedings of 14th European Conference on Machine Learning. Springer-Verlag, pp. 108-120.
15. Van Calster B, Van Belle V, Condous G, Bourne T, Timmerman D, et al. (2008) Multi-class AUC metrics and weighted alternatives. In: Proceedings 2008 International Joint Conference on Neural Networks, IJCNN08. IEEE, pp. 1390-1396.
16. Li Y (2009) A generalization of AUC to an ordered multi-class diagnosis and application to lon-gitudinal data analysis on intellectual outcome in pediatric brain-tumor patients. Ph.D. thesis, College of Arts and Sciences, Georgia State University.
17. Hand D, Till R, (2001) A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45: 171-186.
18. Freitas C, De Carvalho J, Oliveira J Jr, Aires S, Sabourin R (2007) Confusion matrix disagreement for multiple classifiers. In: Rueda L, Mery D, Kittler J, editors, Proceedings of 12th Iberoamerican Congress on Pattern Recognition, CIARP 2007, LNCS 4756. Springer-Verlag, pp. 387-396.
19. Freitas C, De Carvalho J, Oliveira J Jr, Aires S, Sabourin R (2007) Distance-based Disagreement Classifiers Combination. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 2007. IEEE, pp. 2729-2733.
20. Matthews B, (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta- Protein Structure 405: 442-451.
21. Baldi P, Brunak S, Chauvin Y, Andersen C, Nielsen H, (2000) Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16: 412-424.
22. The MicroArray Quality Control (MAQC) Consortium (2010) The MAQC-II Project: A comprehensive study of common practices for the development and validation of microarray-based predictive models. Nature Biotechnology 28: 827-838.
23. Gorodkin J, (2004) Comparing two K-category assignments by a K-category correlation coefficient. Computational Biology and Chemistry 28: 367-374.
24. Supper J, Spieth C, Zell A (2007) Reconstructing Linear Gene Regulatory Networks. In: Marchiori E, Moore J, Rajapakse J, editors, Proceedings of the 5th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, EvoBIO2007, LNCS 4447. Springer-Verlag, pp. 270-279.
25. Stokic D, Hanel R, Thurner S, (2009) A fast and efficient gene-network reconstruction method from multiple over-expression experiments. BMC Bioinformatics 10: 253.
26. Shannon C, (1948) A Mathematical Theory of Communication. The Bell System Technical Journal 27: 379-656, 379-423, 623-656.
27. van Son R (1994) A method to quantify the error distribution in confusion matrices. Technical Report IFA Proceedings 18, Institute of Phonetic Sciences, University of Amsterdam.
28. Abramson N (1963) Information theory and coding. McGraw-Hill, 201 pp.
29. Sindhwani V, Bhattacharge P, Rakshit S (2001) Information theoretic feature crediting in multiclass Support Vector Machines. In: Grossman R, Kumar V, editors, Proceedings First SIAM International Conference on Data Mining, ICDM01. SIAM, pp. 1-18.
30. Wei JM, Yuan XJ, Hu QH, Wang SQ, (2010) A novel measure for evaluating classifiers. Expert Systems with Applications 37: 3799-3809.
31. Wei JM, Yuan XJ, Yang T, Wang SQ, (2010) Evaluating Classifiers by Confusion Entropy. Information Processing & Management Submitted.
32. Huang J, Ling C, (2005) Using AUC and Accuracy in Evaluating Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering 17: 299-310.
33. Sørlie T, Tibshirani R, Parker J, Hastie T, Marron JS, et al. (2003) Repeated observation of breast tumor subtypes in independent gene expression data sets. Proceedings of the National Academy of Sciences 100: 8418-8423.
34. Hastie T, Tibshirani R, Friedman JH (2003) The Elements of Statistical Learning. Springer.
35. Hand D, (2010) Evaluating diagnostic tests: The area under the ROC curve and the balance of errors. Statistics in Medicine 29: 1502-1510.