Constructing new and better evaluation measures for machine learning

IJCAI International Joint Conference on Artificial Intelligence

Volumn , Issue , 2007, Pages 859-864

  Huang, Jin ^a   Ling, Charles X ^b  

^a UNIVERSITY OF OTTAWA   (Canada)

^b UNIVERSITY OF WESTERN ONTARIO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL DATASETS; EVALUATION MEASURES; GENERAL APPROACH; GREEDY SEARCH; LEARNING MODELS; PREDICTION PERFORMANCE; PREDICTIVE PERFORMANCE; ROOT MEAN SQUARE ERRORS;

LEARNING SYSTEMS; MEAN SQUARE ERROR; NEURAL NETWORKS;

LEARNING ALGORITHMS;

EID: 65649085468     PISSN: 10450823     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. C.L. Blake and C.J. Merz. UCI Repository of machine learning databases [http://www.ics.uci.edu/?mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science, 1998.
2. A. P. Bradley. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30:1145-1159, 1997.
3. Rich Caruana and Alexandru Niculescu-Mizil. Data mining in metric space: An empirical analysis of supervised learning performance criteria. In Proceedings of the 10th ACM SIGKDD conference, 2004.
4. Allen Louis Edwards. An introduction to linear regression and correlation. W. H. Freeman, 1976.
5. C. Ferri, P. A. Flach, and J. Hernandez-Orallo. Learning decision trees using the area under the ROC curve. In Proceedings of the Nineteenth International Conference on Machine Learning (ICML 2002), pages 139-146, 2002.
6. D.M. Green and J.A. Swets. Signal Detection Theory and Psychophysics. Wiley, New York, 1966.
7. D. J. Hand and R. J. Till. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning, 45:171-186, 2001.
8. Alan Herschtal and Bhavani Raskutti. Optimising area under the ROC curve using gradient descent. In Proceedings of the 21st International Conference on Machine Learning, 2004.
9. J. F. Kenney and E. S Keeping. Mathematics of Statistics. Princeton, NJ, 1962.
10. I. Kononenko. Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition. In B. Wielinga, editor, Current Trends in Knowledge Acquisition. IOS Press, 1990.
11. C. X. Ling, J. Huang, and H. Zhang. AUC: a statistically consistent and more discriminating measure than accuracy. In Proceedings of 18th International Conference on Artificial Intelligence (IJCAI-2003), pages 519-526, 2003.
12. F. Provost and P. Domingos. Tree induction for probability-based ranking. Machine Learning, 52:3:199-215, 2003.
13. F. Provost and T. Fawcett. Analysis and visualization of classifier performance: comparison under imprecise class and cost distribution. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pages 43-48. AAAI Press, 1997.
14. F. Provost, T. Fawcett, and R. Kohavi. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 445-453. Morgan Kaufmann, 1998.
15. J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann: San Mateo, CA, 1993.
16. Alain Rakotomamonjy. Optimizing AUC with SVMs. In Proceedings of European Conference on Artificial Intelligence Workshop on ROC Curve and AI, 2004.
17. Saharon Rosset. Model selection via the AUC. In Proceedings of the 21st International Conference on Machine Learning, 2004.