Accuracy of machine learning models versus "hand crafted" expert systems - A credit scoring case study

Expert Systems with Applications

Volumn 36, Issue 3 PART 1, 2009, Pages 5264-5271

  Ben David, Arie ^a   Frank, Eibe ^b  

^a HOLON INSTITUTE OF TECHNOLOGY   (Israel)

^b UNIVERSITY OF WAIKATO   (New Zealand)

Author keywords

Accuracy; Classification; Cohen's kappa; Credit scoring; Expert systems; Hit ratio; Machine learning models; Regression

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); LEARNING SYSTEMS;

ACCURACY; COHEN'S KAPPAS; CREDIT SCORING; HIT RATIO; MACHINE LEARNING MODELS; REGRESSION;

EXPERT SYSTEMS;

EID: 58349110098     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2008.06.071     Document Type: Article

References (37)

1. Atkenson C.G., Moore A.W., and Schaal S. Locally weighted learning. Artificial Intelligence Review 11 (1997) 11-73
2. Bishop C.M. Neural networks for pattern recognition (1995), University Press, Oxford
3. Breiman L. Bagging Predictors Machine Learning 24 (1996) 123-140
4. Breiman L. Random forests. Machine Learning 45 (2001) 5-32
5. Cicchetti D.V., and Feinstein A.R. High agreement but low kappa: Resolving the paradoxes. Journal of Clinical Epidemiology 43 6 (1990) 543-558 [In two parts]
6. Cleary, J. G., & Trigg, L. E. (1995). K*: An instance-based learner using an entropic distance measure, In Proceedings of the 12th International Conference on Machine Learning (pp. 108-114). Morgan Kaufmann.
7. Cook R.J. Kappa and its dependence on marginal rates. In: Armitage P., and Colton T. (Eds). Encyclopedia of BioStatistics (1998), Wiely, NY 2166-2168
8. Cohen J.A. Coefficient of agreement for nominal scales. Educational and Psychological Measurement (1960) 37-46
9. Cohen, W. W. (1995). Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning, (pp. 115-123). Morgan Kaufmann.
10. Creecy R.H., Masand B.M., Smith S.J., and Waltz D.L. Trading MIPS and memory for knowledge engineering. Communications of the ACM 35 8 (1992) 48-83
11. Daren, O., Warner, G., White, A., & Bessant, C. (2005). Comparison of machine learning techniques and expert systems for the determination of microarray quality. Poster presentation no. G-A6. Computational Biology, Madrid. .
12. Domingos P., and Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29 (1997) 103-130
13. th International Joint Conference on AI (pp. 1022-1027). Morgan Kaufmann.
14. Frank, E., & Witten, I. H. (1998). Generating accurate rule sets without global optimization. In Proceedings of the 15th International Conference on Machine Learning (pp. 144-151). Morgan Kaufmann.
15. Frank, E., & Hall, M. (2001). A simple approach to ordinal classification, In Proceedings of the 12th European Conference on Machine Learning (pp. 145-156). Springer-Verlag.
16. Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Proceedings of the 13th International Conference on Machine Learning (pp. 148-156). Morgan Kaufmann.
17. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: A statistical view of boosting. Annals of Statistic 28 (2000) 337-407
18. Hall, M. (1998). Correlation-based feature subset selection for machine learning. Ph.D. thesis, Department of Computer Science, University of Waikato, Hamilton, New Zealand.
19. Holmes, G., Hall, M., & Frank, E. (1999). Generating rule sets from model trees. In Proceedings of the 12th Australian Joint Conference on Artificial Intelligence (pp. 1-12). Springer-Verlag.
20. Holte R. Very simple classification rules perform well on most commonly used datasets. Machine Learning 11 (1993) 63-91
21. Kohavi, R. (2005). The Power of Decision Tables. In Proceedings of the Eighth European Conference on Machine Learning (pp. 174-189). Springer-Verlag.
22. Kohavi, R. (1996). Scaling up the accuracy of Naive-Bayes classifiers: A decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (pp. 202-207). ACM Press.
23. Landwehr N., Hall M., and Frank E. Logistic model trees. Machine Learning 59 (2005) 161-205
24. Lim T.S., Loh W.Y., and Shih Y.S. A comparison of prediction accuracy, complexity and training time of thirty-three old and new classification algorithms. Machine Learning 40 (2000) 203-229
25. Melville, P., & Mooney, R. J. (2003). Constructing diverse classifier ensembles using artificial training examples. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (pp. 505-510). Morgan Kaufmann.
26. Michalski, R. S., Chilausky, R. L. (1980). Knowledge acquisition by encoding expert rules versus computer induction from examples: A case study of involving soybean pathology. In International Journal of Man-Machine Studies (vol. 12) (pp. 63-87). [Republished in International Journal of Human-Computer Studies (same journal), 51(2), August 1999, pp. 239-263]
27. Musen M. Automated support for building and extending expert models. Machine Learning 4 (1989) 347-375
28. Platt J. Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B., Burges C., and Smola A. (Eds). Advances in Kernel Methods Support Vector Learning (1998), MIT Press (pp. 185-208)
29. Quinlan J.R. Simplifying decision trees. In: Gaines B., and Boose J. (Eds). Knowledge Acquisition for Knowledge-Based Systems (1987), Academic Press (pp. 239-252)
30. Quinlan, J. R. (1992). Learning with continuous classes. In Proceedings of the Fifth Australian Joint Conference on Artificial Intelligence (pp. 343-348). World Scientific.
31. Quinlan J.R. Programs for machine learning (1993), Morgan Kaufmann
32. Rifkin R., and Klautau A. In defense of one-vs-all classification. Journal of Machine Learning Research 5 (2004) 101-141
33. In: Rumelhart D.E., and McLelland J.L. (Eds). Parallel distributed processing learning: internal representation by error propagation vol. 1 (1986), MIT Press, Cambridge, MA
34. Salzberg S. A nearest hyperrectangle learning method. Machine Learning 6 (1991) 251-276
35. Smola, A. J., Schölkopf, B. (1998). A tutorial on support vector regression. In NeuroCOLT2 Technical Report Series, NC2-TR-1998-030.
36. Webb G.I. MultiBoosting: A technique for combining boosting and wagging. Machine Learning 40 (2000) 159-196
37. Witten I.H., and Frank E. Data mining: Practical machine learning tools and techniques (2005), Morgan Kaufmann