Machine learning as an experimental science

AAAI Workshop - Technical Report

Volumn WS-06-06, Issue , 2006, Pages 1-5

  Drummond, Chris ^a  

^a NATIONAL RESEARCH COUNCIL CANADA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER SCIENCE; DECISION TABLES; STATISTICAL TESTS;

EXPERIMENTAL SCIENCE; HYPOTHESIS TESTING; PERFORMANCE TESTING; PUBLICATIONS;

LEARNING SYSTEMS;

EID: 33845993969     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (39)

1. Bay, S. D.; Kibler, D.; Pazzani, M. J.; and Smyth, P. 2000. The UCI KDD archive of large data sets for data mining research and experimentation. SIGKDD Explorations. 2(2): 14.
2. Blake, C. L., and Merz, C. J. 1998. UCI repository of machine learning databases, University of California, Irvine, CA. www.ics.uci.edu/~mlearn/ MLRepository.html.
3. Chow, S. L. 1998. Precis of statistical significance: Rationale, validity, and utility. Behavioral And Brain Sciences 21:169-239.
4. Cohen, J. 1994. The earth is round (p ¿ .05). American Psychologist 49:997-1003.
5. Drummond, C., and Holte, R. C. 2005. Learning to live with false alarms. In Proceedings of the KDD Data Mining Methods for Anomaly Detection workshop, 21-24.
6. Evgeniou, T.; Pontil, M.; and Elisseeff, A. 2004. Leave-one-out error, stability, and generalization of voting combination of classifiers. Machine Learning 55(1):71-97.
7. Ferri, C.; Flach, P.; Hernndez-Orallo, J.; and Lachiche, N., eds. 2004. ECAI 2004 Workshop on ROC Analysis in AI.
8. Fisher, R. A. 1955. Statistical methods and scientific induction. Journal of the Royal Statistical Society Ser. B 17:69-78.
9. Forman, G., and Cohen, I. 2005. Beware the null hypothesis: Critical value tables for evaluating classifiers. In Proceedings of the 16th European Conference on Machine Learning, 133-145.
10. Gardner, M., and Altman, D. G. 1986. Confidence intervals rather than p values: estimation rather than hypothesis testing. British Medical Journal 292:746-750.
11. Gill, J., and Meier, K. 1999. The insignificance of null hypothesis significance testing. Political Research Quarterly 647-674.
12. Greiner, R.; Silver, B.; Becker, S.; and Gruninger, M. 1988. A review of machine learning at AAAI-87. Machine Learning 3(1):79-92.
13. Hagood, M. J. 1941. The notion of the hypothetical universe. In The Significance Test Controversy: A Reader. Chicago: Aldine. chapter 4, 65-78.
14. Holte, R. C. 1993. Very simple classification rules perform well on most commonly used datasets. Machine Learning 11(1):63-91.
15. Jensen, D. D., and Cohen, P. R. 2000. Multiple comparisons in induction algorithms. Machine Learning 38(3):309-338.
16. Johnson, D. H. 1999. The insignificance of statistical significance testing. Journal of Wildlife Management 63(3):763-772.
17. Keogh, E. 2005. Recent advances in mining time series data: Why most of them are not really "advances"... Invited Talk at the European Conference on Machine Learning.
18. Kibler, D., and Langley, P. 1988. Machine learning as an experimental science. In Proceedings of the Third European Working Session on Learning, 81-92.
19. Kodratoff, Y. 1994. Guest editor's introduction: The comprehensibility manifesto. AI Communications 7(2).
20. Langley, P. 1988. Machine learning as an experimental science. Machine Learning 3:5-8.
21. Langley, P. 2000. Crafting papers on machine learning. In Proceedings of the Seventeenth International Conference on Machine Learning, 1207-1211.
22. Margineantu, D.; Schumann, J.; and Drumheller, M., eds. 2004. Proceedings of the NIPS Workshop on Verification, Validation, and Testing of Learning Systems.
23. Meehl, P. E. 1997. The problem is epistemology, not statistics: Replace significance tests by confidence intervals and quantify accuracy of risky numerical predictions. In What If There Were No Significance Tests? Mahwah, NJ: Erlbaum, chapter 14, 393-25.
24. Michalski, R. S. 1983. A theory and methodology of inductive learning. Artificial Intelligence 20:111-161.
25. Moran, J. L., and Solomon, P. J. 2004. A farewell to p-values? Critical care and Resuscitation Journal.
26. Nalbone, D. P., ed. 2006. The Journal of Articles in Support of the Null Hypothesis. Fresno, CA: Reysen Group.
27. Nix, T. W., and Barnette, J. J. 1998. The data analysis dilemma: Ban or abandon, a review of null hypothesis significance testing. Research In The Schools 5(2):3-14.
28. Oblinger, D.; Lau, T.; Gil, Y.; and Bauer, M., eds. 2005. AAAI Workshop on Human Comprehensible Machine Learning.
29. Pazzani, M.; Merz, C.; Murphy, P.; Ali, K.; Hume, T.; and Brunk, C. 1994. Reducing misclassification costs. In Proceedings of the Eleventh International Conference on Machine Learning, 217-225.
30. Provost, F., and Fawcett, T. 2001. Robust classification for imprecise environments. Machine Learning 42:203-231.
31. Provost, F.; Fawcett, T.; and Kohavi, R. 1998. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the Fifteenth International Conference on Machine Learning, 43-48.
32. Riedmiller, M., and Littman, M. L., eds. 2005. Proceedings of the NIPS Reinforcement Learning Benchmarks and Bake-offs II.
33. Saitta, L., and Neri, F. 1998. Learning in the "real world". Machine Learning 30(2-3):133-163.
34. Salzberg, S. L. 1997. On comparing classifiers: Pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery 1:317-327.
35. Schmidt, F. L. 1996. Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods 1(2):115-129.
36. Sutton, R. S., and Littman, M. L., eds. 2004. Proceedings of the NIPS Reinforcement Learning Benchmarks and Bake-offs.
37. Thagard, P. 1988. Computational Philosophy of Science. MIT Press.
38. Thearling, K., and Stein, R., eds. 1998. KDD'98 workshop Keys to the Commercial Success of Data Mining.
39. Tukey, J. W. 1977. Exploratory Data Analysis. Reading, MA: Addison-Wesley.