Explaining machine learning models in sales predictions

Expert Systems with Applications

Volumn 71, Issue , 2017, Pages 416-428

  Bohanec, Marko ^a,b   Kljajić Borštnar, Mirjana ^b   Robnik Šikonja, Marko ^c  

^a Salvirt doo   (Slovenia)

^b UNIVERSITY OF MARIBOR   (Slovenia)

^c UNIVERSITY OF LJUBLJANA   (Slovenia)

Author keywords

B2B Sales forecasting; Black box models; Intelligent system; Machine learning; Prediction explanation

Indexed keywords

DECISION MAKING; DECISION SUPPORT SYSTEMS; FORECASTING; INTELLIGENT SYSTEMS; LEARNING SYSTEMS; MACHINE LEARNING; SALES;

BLACK-BOX MODEL; BUSINESS ENVIRONMENTS; BUSINESS TO BUSINESS; DATA DRIVEN DECISION; DECISION SUPPORT PROBLEM; GRAPH REPRESENTATION; MACHINE LEARNING MODELS; SALES FORECASTING;

PREDICTIVE ANALYTICS;

EID: 85006817279     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2016.11.010     Document Type: Article

References (54)

1. Aggarwal, C.C., Chen, C., Han, J., The inverse classification problem. Journal of Computer Science and Technology 25:3 (2010), 458–468.
2. Andrews, R., Diederich, J., Tickle, A.B., Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8:6 (1995), 373–384.
3. Armstrong, J.S., Green, K.C., Graefe, A., Golden rule of forecasting: Be conservative. Journal of Business Research 68:8 (2015), 1717–1731.
4. Arnold, V., Clark, N., Collier, P.A., Leech, S.A., Sutton, S.G., The differential use and effect of knowledge-based system explanations in novice and expert judgment decisions. Mis Quarterly, 2006, 79–97.
5. Baehrens, D., Schroeter, T., Harmeling, S., Kawanabe, M., Hansen, K., Müller, K.-R., How to explain individual classification decisions. Journal of Machine Learning Research 11:June (2010), 1803–1831.
6. Barbella, D., Benzaid, S., Christensen, J.M., Jackson, B., Qin, X.V., Musicant, D.R., Understanding support vector machine classifications via a recommender system-like approach. Stahlbock, R., Crone, S.F., Lessmann, S., (eds.) Proceedings of international conference on data mining, 2009, 305–311.
7. Barbosa, A., Paulovich, F., Paiva, A., Goldenstein, S., Petronetto, F., Nonato, L., Visualizing and interacting with kernelized data. IEEE Transactions on Visualization and Computer Graphics 22:3 (2016), 1314–1325.
8. Becker, B., Kohavi, R., Sommereld, D., Visualizing the simple bayesian classier. KDD workshop on issues in the integration of data mining and data visualization, 1997.
9. Bohanec, M. (2016). Anonymized B2B sales forecasting data set http://www.salvirt.com/research/b2bdataset/.
10. Bohanec, M., Kljajić Borštnar, M., Robnik-Šikonja, M., Integration of machine learning insights into organizational learning: A case of B2B sales forecasting. D'Ascenzo, F., Magni, M., Lazazzara, A., Za, S., (eds.) Blurring the boundaries through digital innovation: Individual, organizational, and societal challenges, 19, 2016, Springer.
11. Bosnić, Z., Demšar, J., Kešpret, G., Rodrigues, P.P., Gama, J., Kononenko, I., Enhancing data stream predictions with reliability estimators and explanation. Engineering Applications of Artificial Intelligence 34 (2014), 178–192.
12. Breiman, L., Random forests. Machine Learning Journal 45 (2001), 5–32.
13. Brynjolfsson, E., Hitt, L. M., & Kim, H. H. (2011). Strength in numbers: How does data-driven decision-making affect firm performance? Available at SSRN 1819486.
14. Caragea, D., Cook, Dianne Honavar, V., Towards simple, easy-to-understand, yet accurate classifiers. Third IEEE international conference on data mining, ICDM 2003., 2003, 497–500.
15. Caruana, R., Niculescu-Mizil, A., An empirical comparison of supervised learning algorithms. Proceedings of the 23rd international conference on machine learning ICML ’06, 2006, ACM, New York, NY, USA, 161–168.
16. Costea, A., Bleotu, V., A new fuzzy clustering algorithm for evaluating the performance of non-banking financial institutions in Romania. Economic Computation and Economic Cybernetics Studies and Research 46:4 (2012), 179–199.
17. D'Haen, J., Van der Poel, D., Model-supported business-to-business prospect prediction based on an iterative customer acquisition framework. Industrial Marketing Management 42 (2013), 544–551.
18. Florez-Lopez, R., Ramon-Jeronimo, J.M., Enhancing accuracy and interpretability of ensemble strategies in credit risk assessment. a correlated-adjusted decision forest proposal. Expert Systems with Applications 42:13 (2015), 5737–5753.
19. Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E., Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics 24:1 (2015), 44–65.
20. Gönül, M.S., Önkal, D., Lawrence, M., The effect of structural characteristics of explanations on use of a DSS. Decision Support Systems, 2006, 1481–1493.
21. Good, I.J., Probability and the weighing of evidence. 1950, C. Griffin, London.
22. Hamel, L., Visualization of support vector machines with unsupervised learning. Proceedings of 2006 IEEE symposium on computational intelligence in bioinformatics and computational biology, 2006.
23. Hansen, K., Baehrens, D., Schroeter, T., Rupp, M., Müller, K.-R., Visual interpretation of kernel-based prediction models. Molecular Informatics 30:9 (2011), 817–826.
24. Holzinger, A., Interactive machine learning for health informatics: when do we need the human-in-the-loop?. Brain Informatics 3:2 (2016), 119–131.
25. Jacobsson, H., Rule extraction from recurrent neural networks: A taxonomy and review. Neural Computation 17:6 (2005), 1223–1263.
26. Jakulin, A., Možina, M., Demšar, J., Bratko, I., Zupan, B., Nomograms for visualizing support vector machines. Grossman, R., Bayardo, R., Bennett, K.P., (eds.) Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery and data mining, 2005, ACM, 108–117.
27. Kononenko, I., Inductive and bayesian learning in medical diagnosis. Applied Artificial Intelligence 7:4 (1993), 317–337.
28. Lemaire, V., Clérot, F., An input variable importance definition based on empirical data probability and its use in variable selection. Proceedings of IEEE international joint conference on neural networks, 2004, 1375–1380 vol.2, 10.1109/IJCNN.2004.1380149.
29. Lemaire, V., Féraud, R., Voisine, N., Contact personalization using a score understanding method. Proceedings of international joint conference on neural networks, 2008.
30. Lilien, G.L., The B2B knowledge gap. International Journal of Research in Marketing 33:3 (2016), 543–556.
31. Lubsen, J., Pool, J., van der Does, E., A practical device for the application of a diagnostic or prognostic function. Methods of Information in Medicine 17 (1978), 127–129.
32. Madigan, D., Mosurski, K., Almond, R.G., Graphical explanation in belief networks. Journal of Computational and Graphical Statistics 6:2 (1997), 160–181.
33. Martens, D., Provost, F., Explaining documents classifications. Technical Report, 2011, Center for Digital Economy Research, New York University, Stern School of Business Working paper CeDER-11-01.
34. McAfee, A., Brynjolfsson, E., Big data: The management revolution. Harvard Business Review 90:10 (2012), 61–67.
35. Merkert, J., Mueller, M., Hubl, M., A survey of the application of machine learning in decision support systems. European conference on information systems (ECIS), completed research papers. Munster, Germany, 2015.
36. Meyer, G., Adomavicius, G., Johnson, P.E., Elidrisi, M., Rush, W.A., Sperl-Hillen, J.M., O'Connor, P.J., A machine learning approach to improving dynamic decision making. Information Systems Research 25:2 (2014), 239–263.
37. Možina, M., Demšar, J., Kattan, M.W., Zupan, B., Nomograms for visualization of naive bayesian classifier. Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D., (eds.) Knowledge discovery in databases: PKDD 2004, 2004, Springer, 337–348.
38. Nonaka, I., Takeuchi, H., The knowledge-creating company: How Japanese companies create the dynamics of innovation. 1995, Oxford University Press.
39. Poulet, F., SVM and graphical algorithms: A cooperative approach. Fourth IEEE international conference on data mining (ICDM’04), 2004, 499–502.
40. Poulin, B., Eisner, R., Szafron, D., Lu, P., Greiner, R., Wishart, D.S. Anvik, J., Visual explanation of evidence with additive classifiers. Proceedings of AAAI’06, 2006, AAAI Press.
41. Pregeljc, M., Štrumbelj, E., Mihelcic, M., Kononenko, I., Learning and explaining the impact of enterprises organizational quality on their economic results. Magdalena-Benedito, R., Martnez-Sober, M., Martnez-Martnez, J.M., Escandell-Moreno, P., Vila-Francs, J., (eds.) Intelligent data analysis for real-life applications: Theory and practice, 2012, Information Science Reference, IGI Global, 228–248.
42. Provost, F., Fawcett, T., Data science and its relationship to big data and data-driven decision making. Big Data 1:1 (2013), 51–59.
43. Provost, F.J., Fawcett, T., Kohavi, R., The case against accuracy estimation for comparing induction algorithms. Proceedings of international conference on machine learning, ICML’98, 98, 1998, 445–453.
44. Robnik-Šikonja, M. (2015). Explainprediction: Explanation of predictions for classification and regression. R package version 1.1.2.
45. Robnik-Šikonja, M., Kononenko, I., Discretization of continuous attributes using ReliefF. Solina, F., Zajc, B., (eds.) Proceedings of electrotehnical and computer science conference (ERK’95), 1995, Slovene section of IEEE, Ljubljana, B149–152.
46. Robnik-Šikonja, M., Kononenko, I., Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering 20:5 (2008), 589–600.
47. Saltelli, A., Chan, K., Scott, E.M., Sensitivity analysis. 2000, Wiley, New York.
48. Schulz, A., Gisbrecht, A., Hammer, B., Using discriminative dimensionality reduction to visualize classifiers. Neural Processing Letters 42:1 (2015), 27–54.
49. Simon, H. A. (1960). The new science of management decision.
50. Štrumbelj, E., Bosnić, Z., Kononenko, I., Zakotnik, B., Kuhar, C.G., Explanation and reliability of prediction models: The case of breast cancer recurrence. Knowledge and Information Systems 24:2 (2010), 305–324.
51. Verikas, A., Gelzinis, A., Bacauskiene, M., Mining data with random forests: A survey and results of new tests. Pattern Recognition 44:2 (2011), 330–349.
52. Štrumbelj, E., Kononenko, I., An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research 11 (2010), 1–18.
53. Štrumbelj, E., Kononenko, I., Robnik-Šikonja, M., Explaining instance classifications with interactions of subsets of feature values. Data & Knowledge Engineering 68:10 (2009), 886–904.
54. Yan, J., Zhang, C., Zha, H., Gong, M., Sun, C., Huang, J., Yang, X., On machine learning towards predictive sales pipeline analytics. Twenty-ninth AAAI conference on artificial intelligence, 2015, 1945–1951.