Why question machine learning evaluation methods? (An illustrative review of the shortcomings of current methods)

AAAI Workshop - Technical Report

Volumn WS-06-06, Issue , 2006, Pages 6-11

  Japkowicz, Nathalie ^a  

^a UNIVERSITY OF OTTAWA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONFIDENCE ESTIMATION METHODS; EVALUATION METHODS; T-TEST;

CLASSIFICATION (OF INFORMATION); COMPUTER SELECTION AND EVALUATION; DATA MINING; LEARNING ALGORITHMS; STATISTICAL TESTS;

LEARNING SYSTEMS;

EID: 33845990606     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. [Dietterich, T.G., 1996], Proper Statistical Tests for Comparing Supervised Classification Learning Algorithms (Technical Report). Department of Computer Science, Oregon State University, Corvallis, OR.
2. [Drummond, C. and Holte, R.], 2000, "Explicitly Representing Expected Cost: An Alternative to ROC Representation". Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 198-207.
3. [Drummond, C. and Holte, R., 2005], "Learning to Live with False Alarms", Data Mining Methods for Anomaly Detection workshop at the 11th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2005). pp. 21-24.
4. [Elazmeh, 2006], Personnal Communication.
5. [Elazmeh, W., Japkowicz, N., and Matwin, S., 2006], "A Framework for Measuring Classification Difference with Imbalance", AAAI'2006 Workshop on Machine Learning Evaluation
6. [Fawcett, T., 2003], ROC Graphs: Notes and Practical Considerations for Researchers, HP Labs Tech Report HPL-2003-4.
7. [Mitchell, T., 1997], Machine Learning, McGraw-Hill.
8. [Provost, F. & Fawcett, T.], "Robust Classification for Imprecise Environments", Machine Learning Journal, vol 42, no.3, 2001
9. [Salzberg, S.], On Comparing Classifiers: "Pitfalls to Avoid and a Recommended Approach", Data Mining and Knowledge Discovery 1:3 (1997), 317-327.
10. [Sokolova, M., Japkowicz, N. and Szpakowicz, S.], "On Evaluation of Learning from new Types of Data", Submitted to the AAAI '2006 Workshop on Machine Learning Evaluation