A new hybrid ensemble credit scoring model based on classifiers consensus system approach

Expert Systems with Applications

Volumn 64, Issue , 2016, Pages 36-55

  Ala'raj, Maher ^a   Abbod, Maysam F ^a  

^a BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

Classifier ensembles; Consensus approach; Credit scoring; Data filtering; Feature selection; Hybrid models

Indexed keywords

DECISION TREES; FEATURE EXTRACTION; SUPPORT VECTOR MACHINES;

CLASSIFIER ENSEMBLES; CONSENSUS APPROACH; CREDIT SCORING; DATA FILTERING; HYBRID MODEL;

DATA HANDLING;

EID: 84979529983     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2016.07.017     Document Type: Article

References (94)

1. Abellán, J., Mantas, C.J., Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications 41 (2014), 3825–3830.
2. Acuna, E., Rodriguez, C., The treatment of missing values and its effect on classifier Accuracy. Classification, clustering, and data mining applications, 2004, Springer, 639–647.
3. Akkoç, S., An empirical comparison of conventional techniques, neural network and the three stage hybrid adaptive neuro fuzzy inference system (ANFIS) model for credit-scoring analysis: The case of Turkish credit card data. European Journal of Operational Research 222 (2012), 168–178.
4. Ala'raj, M., Abbod, M.F., Classifiers consensus system approach for credit scoring. Knowledge-Based Systems 104 (2016), 89–105.
5. Angelini, E., di Tollo, G., Roli, A., A neural network approach for credit risk evaluation. The Quarterly Review of Economics and Finance 48 (2008), 733–755.
6. Antonakis, A., Sfakianakis, M., Assessing naive Bayes as a method for screening credit applicants. Journal of Applied Statistics 36 (2009), 537–545.
7. Asuncion, A., & Newman, D.J. (2007). UCI Machine Learning Repository. Irvine, CA: School of Information and Computer Science, University of California. http://www.ics.uci.edu/~mlearn/MLRepository.html.
8. Atiya, A.F., Parlos, A.G., New results on recurrent network training: Unifying the algorithms and accelerating convergence. Neural Networks, IEEE Transactions on 11 (2000), 697–709.
9. Baesens, B., Van Gestel, T., Viaene, S., Stepanova, M., Suykens, J., Vanthienen, J., Benchmarking state-of-the-art classification algorithms for credit scoring. Journal of the Operational Research Society 54 (2003), 627–635.
10. Basir, O.A., Shen, H.C., New approach for aggregating multi‐sensory data. Journal of Robotic Systems 10 (1993), 1075–1093.
11. Bellotti, T., Crook, J., Support vector machines for credit scoring and discovery of significant features. Expert Systems with Applications 36 (2009), 3302–3308.
12. Benediktsson, J.A., Swain, P.H., Consensus theoretic classification methods. Systems, Man and Cybernetics, IEEE Transactions on 22 (1992), 688–704.
13. Berger, R.L., A necessary and sufficient condition for reaching a consensus using DeGroot's method. Journal of the American Statistical Association 76 (1981), 415–418.
14. Bhattacharyya, S., Maulik, U., Soft computing for image and multimedia data processing. 2013, Springer.
15. Bishop, C.M., Pattern recognition and machine learning. 2006, Springer.
16. Breiman, L., RFs. Machine-Learning 45:1 (2001), 5–32.
17. Brier, G.W., Verification of forecasts expressed in terms of probability. Monthly Whether Review 78:1 (1950), 1–3.
18. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J., Classification and regression trees. 1984, Wadsworth, Belmont, CA.
19. Chen, F., Li, F., Combination of feature selection approaches with SVM in credit scoring. Expert Systems with Applications 37 (2010), 4902–4909.
20. Chen, W., Ma, C., Ma, L., Mining the customer credit using hybrid support vector machine technique. Expert Systems with Applications 36 (2009), 7611–7616.
21. Chitroub, S., Classifier combination and score level fusion: Concepts and practical aspects. International Journal of Image and Data Fusion 1 (2010), 113–135.
22. Cortes, C., Vapnik, V., Support-vector networks. Machine Learning 20 (1995), 273–297.
23. DeGroot, M.H., Reaching a consensus. Journal of the American Statistical Association 69 (1974), 118–121.
24. Demšar, J., Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research 7 (2006), 1–30.
25. Desai, V.S., Crook, J.N., Overstreet, G.A., A comparison of neural networks and linear scoring models in the credit union environment. European Journal of Operational Research 95 (1996), 24–37.
26. Dunn, O.J., Multiple comparisons among means. Journal of the American Statistical Association 56 (1961), 52–64.
27. Finlay, S., Multiple classifier architectures and their application to credit risk assessment. European Journal of Operational Research 210 (2011), 368–378.
28. Friedman, M., A comparison of alternative tests of significance for the problem of m rankings. The Annals of Mathematical Statistics 11 (1940), 86–92.
29. Friedman, J.H., Multivariate adaptive regression splines. The Annals of Statistics 19 (1991), 1–67.
30. Gabriel, K.R., Sokal, R.R., A new statistical approach to geographic variation analysis. Systematic Biology 18 (1969), 259–278.
31. García, V., Marqués, A., Sánchez, J.S., On the use of data filtering techniques for credit risk prediction with instance-based models. Expert Systems with Applications 39 (2012), 13267–13276.
32. García, V., Marqués, A.I., Sánchez, J.S., An insight into the experimental design for credit risk and corporate bankruptcy prediction systems. Journal of Intelligent Information Systems 44 (2015), 159–189.
33. Gorzałczany, M.B., Rudziński, F., A multi-objective genetic optimization for fast, fuzzy rule-based credit classification with balanced accuracy and interpretability. Applied Soft Computing 40 (2016), 206–220.
34. Hand, D.J., Measuring classifier performance: A coherent alternative to the area under the ROC curve. Machine Learning 77 (2009), 103–123.
35. Harris, T., Credit scoring using the clustered support vector machine. Expert Systems with Applications 42 (2015), 741–750.
36. Haykin, S., Adaptive filters. IEEE Signal Processing Magazine 16 (1999), 20–22.
37. Hsieh, N., Hung, L., A data driven ensemble classifier for credit scoring analysis. Expert Systems with Applications 37 (2010), 534–545.
38. Huang, C., Chen, M., Wang, C., Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications 33 (2007), 847–856.
39. Jekabsons, G. (2009). Adaptive Regression Splines Toolbox for Matlab. Ver, 1, 3–17.
40. Kennedy, K., Mac Namee, B., Delany, S.J., Using semi-supervised classifiers for credit scoring. Journal of the Operational Research Society 64 (2012), 513–529.
41. Kittler, J., Hatef, M., Duin, R.P., Matas, J., On combining classifiers. Pattern Analysis and Machine Intelligence, IEEE Transactions on 20 (1998), 226–239.
42. Kumar, P.R., Ravi, V., Bankruptcy prediction in banks and firms via statistical and intelligent techniques–A review. European Journal of Operational Research 180 (2007), 1–28.
43. Lahsasna, A., Ainon, R.N., Teh, Y.W., Credit-scoring models using soft computing methods: A survey. Int. Arab J. Inf. Technol. 7 (2010), 115–123.
44. Lai, K.K., Yu, L., Zhou, L., Wang, S., Credit risk evaluation with least square support vector machine. Anonymous rough sets and knowledge technology, 2006, Springer, 490–495.
45. Lee, T., Chiu, C., Lu, C., Chen, I., Credit-scoring using the hybrid neural discriminant technique. Expert Systems with Applications 23:3 (2002), 245–254.
46. Lee, T., Chen, I., A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines. Expert Systems with Applications 28 (2005), 743–752.
47. Lee, T., Chiu, C., Chou, Y., Lu, C., Mining the customer credit using classification and regression tree and multivariate adaptive regression splines. Computational Statistics & Data Analysis 50 (2006), 1113–1130.
48. Lessmann, S., Baesens, B., Seow, H., Thomas, L.C., Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research 247 (2015), 124–136.
49. Li, X., Zhong, Y., An overview of personal credit scoring: Techniques and future work. International Journal of Intelligence Science 2 (2012), 181–189.
50. Li, S., Shiue, W., Huang, M., The evaluation of consumer loans using support vector machines. Expert Systems with Applications 30 (2006), 772–782.
51. Li, H., Sun, J., Majority voting combination of multiple case-based reasoning for financial distress prediction. Expert Systems with Applications 36:3 (2009), 4363–4373.
52. Lin, W., Hu, Y., Tsai, C., Machine learning in financial crisis prediction: A survey. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 42 (2012), 421–436.
53. Liu, Y., Schumann, M., Data mining feature selection for credit-scoring models. Journal of the Operational Research Society 56:9 (2005), 1099–1108.
54. Louzada, F., Ara, A., & Fernandes, G.B. (2016). Classification methods applied to credit scoring: A systematic review and overall comparison. arXiv: 1602.02137.
55. Malhotra, R., Malhotra, D., Evaluating consumer loans using neural networks. Omega 31 (2003), 83–96.
56. Marqués, A., García, V., Sánchez, J.S., Exploring the behaviour of base classifiers in credit scoring ensembles. Expert Systems with Applications 39 (2012), 10244–10250.
57. Marqués, A., García, V., Sánchez, J.S., Two-level classifier ensembles for credit risk assessment. Expert Systems with Applications 39 (2012), 10916–10922.
58. Nanni, L., Lumini, A., An experimental comparison of ensemble of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications 36 (2009), 3028–3033.
59. Partalas, I., Tsoumakas, G., Vlahavas, I., An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning 81:3 (2010), 257–282.
60. Pietruszkiewicz, W., Dynamical systems and nonlinear Kalman filtering applied in classification. Proceedings of 2008 7th IEEE International Conference Cybernetic International Systems, 2008, 263–268.
61. Piramuthu, S., On preprocessing data for financial credit risk evaluation. Expert Systems with Applications 30 (2006), 489–497.
62. Rhemtulla, M., Brosseau-Liard, P.É., Savalei, V., When can categorical variables be treated as continuous. A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 2012, 354.
63. Rokach, L., Ensemble-based classifiers. Artificial Intelligence Review 33 (2010), 1–39.
64. Sabzevari, H., Soleymani, M., Noorbakhsh, A., A comparison between statistical and data mining methods for credit-scoring in case of limited available data. Proceedings of the 3rd CRC Credit-scoring Conference, Edinburgh, UK, 2007.
65. Shaban, K., Basir, O., Kamel, M., Hassanein, K., Intelligent information fusion approach in cooperative multiagent systems. Automation Congress, 2002 Proceedings of the 5th Biannual World, 13, 2002, 429–434.
66. Šušteršič, M., Mramor, D., Zupan, J., Consumer credit-scoring models with limited data. Expert Systems with Applications 36:3 (2009), 4736–4744.
67. Tan, M., Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning, 1993, 330–337.
68. Thomas, L.C., Consumer Credit Models: Pricing, Profit and Portfolios: Pricing, Profit and Portfolios. 2009, Oxford University Press, OUP Oxford.
69. Tomczak, J.M., Zięba, M., Classification restricted Boltzmann machine for comprehensible credit scoring model. Expert Systems with Applications 42 (2015), 1789–1796.
70. Tsai, C., Combining cluster analysis with classifier ensembles to predict financial distress. Information Fusion 16 (2014), 46–58.
71. Tsai, C., Feature selection in bankruptcy prediction. Knowledge-Based Systems 22 (2009), 120–127.
72. Tsai, C., Chen, M., Credit rating by hybrid machine learning techniques. Applied Soft Computing 10 (2010), 374–380.
73. Tsai, C., Cheng, K., Simple instance selection for bankruptcy prediction. Knowledge-Based Systems 27 (2012), 333–342.
74. Tsai, C., Chou, J., Data pre-processing by genetic algorithms for bankruptcy prediction. Proceedings of IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), 2011, 1780–1783.
75. Tsai, C., Wu, J., Using neural network ensembles for bankruptcy prediction and credit scoring. Expert Systems with Applications 34 (2008), 2639–2649.
76. Verikas, A., Kalsyte, Z., Bacauskiene, M., Gelzinis, A., Hybrid and ensemble-based soft computing techniques in bankruptcy prediction: A survey. Soft Computing 14 (2010), 995–1010.
77. Wang, G., Hao, J., Ma, J., Jiang, H., A comparative assessment of ensemble learning for credit scoring. Expert Systems with Applications 38 (2011), 223–230.
78. Wang, C., Huang, Y., Evolutionary-based feature selection approaches with new criteria for data mining: A case study of credit approval data. Expert Systems with Applications 36:3 (2009), 5900–5908.
79. Wang, G., Ma, J., Huang, L., Xu, K., Two credit scoring models based on dual strategy ensemble trees. Knowledge-Based Systems 26 (2012), 61–68.
80. Weiss, S.M., Indurkhya, N., Predictive Data Mining: a practical guide. Morgan Kaufmann, 6, 1998.
81. West, D., Dellana, S., Qian, J., Neural network ensemble strategies for financial decision applications. Computers & Operations Research 32 (2005), 2543–2559.
82. Wilson, D.R., Martinez, T.R., Reduction techniques for instance-based learning algorithms. Machine Learning 38 (2000), 257–286.
83. Woods, K., Bowye, K., Kegelmeyer, W.P. Jr., Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 1997, 391–396.
84. Xiao, H., Xiao, Z., Wang, Y., Ensemble classification based on supervised clustering for credit scoring. Applied Soft Computing 43 (2016), 73–86.
85. Xiao, J., Xie, L., He, C., Jiang, X., Dynamic classifier ensemble model for customer classification with imbalanced class distribution. Expert Systems with Applications 39 (2012), 3668–3675.
86. Xu, L., Krzyżak, A., Suen, C.Y., Methods of combining multiple classifiers and their applications to handwriting recognition. Systems, man and cybernetics, IEEE transactions on 22 (1992), 418–435.
87. Yao, P., Feature Selection Based on SVM for Credit Scoring, Computational Intelligence and Natural Computing. International Conference on, Los Alamitos, CA, USA: IEEE Computer Society, 2, 2009, 44–47.
88. Yu, L., Liu, H., Feature selection for high-dimensional data: A fast correlation-based filter solution. ICML, vol. 3, 2003, 856–863.
89. Yu, L., Wang, S., Lai, K.K., An intelligent-agent-based fuzzy group decision making model for financial multicriteria decision support: The case of credit scoring. European Journal of Operational Research 195 (2009), 942–959.
90. Yu, L., Wang, S., Lai, K.K., Credit risk assessment with a multistage neural network ensemble learning approach. Expert Systems with Applications 34 (2008), 1434–1444.
91. Yu, L., Yue, W., Wang, S., Lai, K.K., Support vector machine based multiagent ensemble learning for credit risk evaluation. Expert Systems with Applications 37 (2010), 1351–1360.
92. Zhang, D., Zhou, X., Leung, S.C., Zheng, J., Vertical bagging decision trees model for credit scoring. Expert Systems with Applications 37 (2010), 7838–7843.
93. Zhou, L., Lai, K.K., Yu, L., Least squares support vector machines ensemble models for credit scoring. Expert Systems with Applications 37 (2010), 127–133.
94. Zhou, L., Tam, K.P., Fujita, H., Predicting the listing status of Chinese listed companies with multi-class classification models. Information Sciences 328 (2016), 222–236.