Performance of classification models from a user perspective

Decision Support Systems

Volumn 51, Issue 4, 2011, Pages 782-793

  Martens, David ^a   Vanthienen, Jan ^b   Verbeke, Wouter ^b   Baesens, Bart ^b,c,d  

^a UNIVERSITY OF ANTWERP   (Belgium)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^d Vlerick   (Belgium)

Author keywords

Classification; Comprehensibility; Data mining; Justifiability; Metrics

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECISION SUPPORT SYSTEMS; INFORMATION SYSTEMS;

CLASSIFICATION MODELS; COMPREHENSIBILITY; CUSTOMER CHURN PREDICTION; DOMAIN CONSTRAINT; DOMAIN KNOWLEDGE; JUSTIFIABILITY; METRICS; USER PERSPECTIVES;

DATA MINING;

EID: 80052936929     PISSN: 01679236     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dss.2011.01.013     Document Type: Article

References (72)

1. D.W. Aha, D.F. Kibler, and M.K. Albert Instance-based learning algorithms Machine Learning 6 1991 37 66
2. E. Altendorf, E. Restificar, and T. Dietterich Learning from sparse data by exploiting monotonicity constraints Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence 2005 Edinburgh, Scotland
3. I. Askira-Gelman Knowledge discovery: comprehensibility of the results HICSS'98: Proceedings of the Thirty-First Annual Hawaii International Conference on System Sciences Volume 5 1998 IEEE Computer Society Washington, DC, USA 247
4. B. Baesens, R. Setiono, C. Mues, and J. Vanthienen Using neural network rule extraction and decision tables for credit-risk evaluation Management Science 49 2003 312 329
5. B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, J. Suykens, and J. Vanthienen Benchmarking state-of-the-art classification algorithms for credit scoring The Journal of the Operational Research Society 54 2003 627 635
6. A. Ben-David Monotonicity maintenance in information-theoretic machine learning algorithms Machine Learning 19 1995 29 43
7. A. Ben-David, L. Sterling, and T. Tran Adding monotonicity to learning algorithms may impair their accuracy Expert Systems with Applications 36 2009 6627 6634
8. D. Billman, and D. Davila Consistency is the hobgoblin of human minds: people care but concept learning models do not Proceedings of the 17th Annual Conference of the Cognitive Science Society 1995 188 193
9. C. Bishop Neural Networks for Pattern Recognition 1996 Oxford University Press Oxford, UK
10. L. Breiman, J. Friedman, R. Olshen, and C. Stone Classification and Regression Trees 1984 Chapman & Hall New York
11. D. Carvalho, A. Freitas, and N. Ebecken Evaluating the correlation between objective rule interestingness measures and real human interest A. Jorge, L. Torgo, P. Brazdil, R. Camacho, J. Gama, PKDD Lecture Notes in Computer Science volume 3721 2005 Springer 453 461
12. P. Clark, and T. Niblett The CN2 induction algorithm Machine Learning 3 1989 261 283
13. W.W. Cohen Fast effective rule induction A. Prieditis, S. Russell, Proc. of the 12th International Conference on Machine Learning 1995 Morgan Kaufmann Tahoe City, CA 115 123
14. N. Cristianini, and J. Shawe-Taylor An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods 2000 Cambridge University Press New York, NY, USA
15. B. Cumps, D. Martens, M. De Backer, S. Viaene, G. Dedene, R. Haesen, M. Snoeck, and B. Baesens Inferring rules for business/ict alignment using ants Information Management 46 2009 116 124
16. H. Daniels, and M. Velikova Derivation of monotone decision models from noisy data IEEE Transactions on Systems, Man, and Cybernetics. Part C: Applications and Reviews 36 2006 705 710 (Pubitemid 46405293)
17. P. Domingos The role of occam's razor in knowledge discovery Data Mining and Knowledge Discovery 3 1999 409 425
18. R. Duda, P. Hart, and D. Stork Pattern Classification second edition 2001 John Wiley and Sons
19. J. Egan Signal Detection Theory and ROC Analysis. Series in Cognition and Perception 1975 Academic Press New York
20. T. Fawcett Using rule sets to maximize roc performance IEEE International Conference on Data Mining 0 2001 131
21. T. Fawcett An introduction to roc analysis Pattern Recognition Letters 27 2006 861 874
22. T. Fawcett Prie: a system for generating rulelists to maximize roc performance Data Mining and Knowledge Discovery 17 2008 207 224
23. T. Fawcett, and F. Provost Adaptive fraud detection Data Mining and Knowledge Discovery 1-3 1997 291 316
24. Federal Trade Commission for the Consumer Facts for consumers: equal credit opportunity Technical Report, FTC March 1998
25. A. Feelders Prior knowledge in economic applications of data mining Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Data Bases Lecture Notes in Computer Science volume 1910 2000 Springer 395 400
26. A. Feelders, and M. Pardoel Pruning for monotone classification trees Advances in Intelligent Data Analysis V volume 2810 2003 Springer 1 12
27. D. Hand Pattern Detection and Discovery D. Hand, N. Adams, R. Bolton, Pattern detection and discovery Lecture Notes in Computer Science volume 2447 2002 Springer 1 12
28. D. Hand Protection or privacy? Data mining and personal data, Pacific-Asia Conference, PAKDD Lecture Notes in Computer Science volume 3918 2006 Springer 1 10 (Pubitemid 44019396)
29. D. Hand Measuring classifier performance: a coherent alternative to the area under the roc curve Machine Learning 77 2009 103 123
30. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning, Data Mining, Inference, and Prediction 2001 Springer
31. J. Huysmans, C. Mues, B. Baesens, and J. Vanthienen An Empirical Evaluation of the Comprehensibility of Decision Table, Tree and Rule Based Predictive Models 2007
32. J. Huysmans, B. Baesens, D. Martens, K. Denys, and J. Vanthienen New trends in data mining Tijdschrift voor economie en Management, volume L 2005 697 711
33. D. Kleinbaum, L. Kupper, K.E. Muller, and A. Nizam Applied Regression Analysis and Multivariable Methods 1997 Duxbury Press
34. Y. Kodratoff The comprehensibility manifesto KDD Nuggets (94:9) 1994
35. O.O. Maimon, and L. Rokach Decomposition Methodology For Knowledge Discovery And Data Mining: Theory And Applications (Machine Perception and Artificial Intelligence) 2005 World Scientific Publishing Company
36. D. Martens, M. De Backer, R. Haesen, B. Baesens, C. Mues, and J. Vanthienen Ant-based approach to the knowledge fusion problem Proceedings of the Fifth International Workshop on Ant Colony Optimization and Swarm Intelligence Lecture Notes in Computer Science 2006 Springer 85 96
37. D. Martens, B. Baesens, T. Van Gestel, and J. Vanthienen Comprehensible credit scoring models using rule extraction from support vector machines European Journal of Operational Research 183 2007 1466 1476 (Pubitemid 47039190)
38. D. Martens, M. De Backer, R. Haesen, M. Snoeck, J. Vanthienen, and B. Baesens Classification with ant colony optimization IEEE Transaction on Evolutionary Computation 11 2007 651 665 (Pubitemid 47527996)
39. D. Martens, L. Bruynseels, B. Baesens, M. Willekens, and J. Vanthienen Predicting going concern opinion with data mining Decision Support Systems 45 2008 765 777
40. D. Martens, T. Van Gestel, and B. Baesens Decompositional rule extraction from support vector machines by active learning IEEE Transactions on Knowledge and Data Engineering 21 2009 178 191
41. R. Michalski A theory and methodology of inductive learning Artificial Intelligence 20 1983 111 161
42. M. Ohsaki, S. Kitaguchi, K. Okamoto, H. Yokoi, and T. Yamaguchi Evaluation of rule interestingness measures with a clinical dataset on hepatitis PKDD'04: Proceedings of the 8th European Conference on Principles and Practice of Knowledge Discovery in Databases 2004 Springer-Verlag New York, Inc. New York, NY, USA 362 373 volume (Pubitemid 39740417)
43. L. Passmore, J. Goodside, L. Hamel, L. Gonzales, T. Silberstein, and J. Trimarchi Assessing decision tree models for clinical in-vitro fertilization data Technical Report TR03-296 2003 Dept. of Computer Science and Statistics, University of Rhode Island
44. M. Pazzani Influence of prior knowledge on concept acquisition: experimental and computational results Journal of Experimental Psychology. Learning, Memory, and Cognition 17 1991 416 432
45. M. Pazzani Learning with globally predictive tests Discovery Science 1998 220 231
46. M. Pazzani, and S. Bay The independent sign bias: gaining insight from multiple linear regression Proceedings of the Twenty First Annual Conference of the Cognitive Science Society 1999 525 530
47. M. Pazzani, S. Mani, and W. Shankle Acceptance by medical experts of rules generated by machine learning Methods of Information in Medicine 40 2001 380 385 (Pubitemid 33144702)
48. F. Provost, T. Fawcett, and R. Kohavi The case against accuracy estimation for comparing classifiers J. Shavlik, Proceedings of the Fifteenth International Conference on Machine Learning (ICML) 1998 Morgan Kaufmann San Francisco, CA, U.S.A 445 453
49. J.R. Quinlan C4.5 Programs for Machine Learning 1993 Morgan Kaufmann Publishers Inc San Francisco, CA, USA
50. J. Seifert Data mining and homeland security: an overview CRS Report for Congress 2006
51. R. Setiono, B. Baesens, and C. Mues Risk management and regulatory compliance: a data mining framework based on neural network rule extraction Proceedings of the International Conference on Information Systems 2006 ICIS
52. R. Setiono, B. Baesens, and C. Mues Recursive neural network rule extraction for data with mixed attributes IEEE Transactions on Neural Networks 19 2008 299 307 (Pubitemid 351359294)
53. A. Silberschatz, and A. Tuzhilin On subjective measures of interestingness in knowledge discovery KDD 1995 275 281
54. J. Sill Monotonic networks Advances in Neural Information Processing Systems volume 10 1998 The MIT Press
55. E. Sommer, An approach to quantifying the quality of induced theories, in: C. Nedellec (Ed.), Proceedings of the IJCAI Workshop on Machine Learning and Comprehensibility, 1995.
56. J.A.K. Suykens, T.V. Gestel, J.D. Brabanter, B. De Moor, and J. Vandewalle Least Squares Support Vector Machines 2002 World Scientific Singapore
57. J. Swets, and R. Pickett Evaluation of Diagnostic Systems: Methods from Signal Detection Theory 1982 Academic Press New York
58. P.N. Tan, M. Steinbach, and V. Kumar Introduction to Data Mining 2006 Pearson Education Boston, MA
59. L. Thomas, D. Edelman, and J. Crook Credit Scoring and its Applications 2002 SIAM Philadelphia, PA
60. T. Van Gestel, and B. Baesens Credit Risk Management 2009 Oxford University Press Basic Concepts
61. T. Van Gestel, J. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De Moor, and J. Vandewalle Benchmarking least squares support vector machine classifiers Machine Learning 54 2004 5 32
62. T. Van Gestel, B. Baesens, P. Van Dijcke, J. Suykens, J. Garcia, and T. Alderweireld Linear and nonlinear credit scoring by combining logistic regression and support vector machines Journal of Credit Risk 1 2005
63. T. Van Gestel, B. Baesens, P. Van Dijcke, J. Garcia, J. Suykens, and J. Vanthienen A process model to develop an internal rating system: sovereign credit ratings Decision Support Systems 42 2006 1131 1151 (Pubitemid 44559705)
64. T. Van Gestel, D. Martens, B. Baesens, D. Feremans, J. Huysmans, and J. Vanthienen Forecasting and analyzing insurance companies' ratings International Journal of Forecasting 23 2007 513 529
65. O. Vandecruys, D. Martens, B. Baesens, C. Mues, M. De Backer, and R. Haesen Mining software repositories for comprehensible software fault prediction models The Journal of Systems and Software 81 2008 823 839 (Pubitemid 351389560)
66. J. Vanthienen, C. Mues, and A. Aerts An illustration of verification and validation in the modelling phase of KBS development Data & Knowledge Engineering 27 1998 337 352 (Pubitemid 128433765)
67. J. Vanthienen, C. Mues, G. Wets, and K. Delaere A tool-supported approach to inter-tabular verification Expert Systems with Applications 15 1998 277 285 (Pubitemid 128422946)
68. V.N. Vapnik The Nature of Statistical Learning Theory 1995 Springer-Verlag New York, Inc. New York, NY, USA
69. M. Velikova, H. Daniels, and A. Feelders Solving partially monotone problems with neural networks Proceedings of the International Conference on Neural Networks 2006 Vienna, Austria
70. W. Verbeke, D. Martens, C. Mues, and B. Baesens Building comprehensible customer churn prediction models with advanced rule induction techniques Expert Systems with Applications 38 2011 2354 2364
71. M. Wellman Fundamental concepts of qualitative probabilistic networks Artificial Intelligence 44 1990 257 303 (Pubitemid 20731995)
72. I.H. Witten, and E. Frank Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations 2000 Morgan Kaufmann Publishers Inc. San Francisco, CA, USA