Gaining insight into student satisfaction using comprehensible data mining techniques

European Journal of Operational Research

Volumn 218, Issue 2, 2012, Pages 548-562

  Dejaeger, Karel ^a   Goethals, Frank ^b   Giangreco, Antonio ^b   Mola, Lapo ^b,c   Baesens, Bart ^a,d,e  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b IESEG SCHOOL OF MANAGEMENT   (France)

^c UNIVERSITY OF VERONA   (Italy)

^d UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^e VLERICK BUSINESS SCHOOL   (Belgium)

Author keywords

Comprehensibility; Data mining; Education evaluation; Multi class classification

Indexed keywords

COMPREHENSIBILITY; DATA MINING TECHNIQUES; EDUCATION EVALUATION; EDUCATION INSTITUTIONS; EDUCATION MARKET; EDUCATIONAL INSTITUTIONS; GAINING INSIGHTS; LARGE SCALE SURVEYS; MODEL PERFORMANCE; MULTI-CLASS CLASSIFICATION; STRATEGIC DECISION MAKING; STUDENT POPULATIONS; STUDENT SATISFACTION;

DECISION MAKING; DRIVER TRAINING; EDUCATION COMPUTING; POPULATION STATISTICS; STUDENTS;

DATA MINING;

EID: 84855700287     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2011.11.022     Document Type: Article

References (92)

1. P. Allison Logistic Regression using the SAS System: Theory and Applications 2000 SAS press Cary, NC
2. E. Allwein, R. Schapire, and Y. Singer Reducing multiclass to binary: a unifying approach for margin classifiers Journal of Machine Learning 1 2000 113 141 (Pubitemid 33738778)
3. E. Altman, and H. Rijken How rating agencies achieve rating stability Journal of Banking & Finance 28 11 2004 2679 2714 (Pubitemid 39305396)
4. P. Amietta, I Luoghi dell'Apprendimento: Methodi, Strumenti e Casi di Eccellenza, FrancoAngeli, Milan, 2000.
5. R. Andrews, J. Diederich, and A. Tickle Survey and critique of techniques for extracting rules from trained artificial neural networks Knowledge-Based Systems 8 6 1995 373 389
6. W. Arthur Jr., W. Bennet Jr., P.A. Edens, and S.T. Bell Effectiveness of training in organizations: a meta-analysis of design and evaluation features Journal of Applied Psychology 88 2 2003 627 635
7. I. Askira-Gelman, Knowledge discovery: comprehensibility of the results, in: Proceedings of the 31th Anual Hawaii International Conference on System Sciences, vol. 5, USA, 1998, pp. 247-256.
8. B. Baesens, C. Mues, D. Martens, and J. Vanthienen 50 years of data mining and OR: upcoming trends and challenges Journal of the Operational Research Society 60 2009 16 23
9. B. Baesens, R. Setiono, C. Mues, and J. Vanthienen Using neural network rule extraction and decision tables for credit-risk evaluation Management Science 49 3 2003 312 329
10. B. Baesens, T. Van Gestel, S. Viaene, M. Stepanova, and J. Vanthienen Benchmarking state-of-the-art classification algorithms for credit scoring Journal of the Operational Research Society 54 6 2003 627 635
11. N. Barakat, and A. Bradley Rule extraction from support vector machines: a review Neurocomputing 74 2010 178 190
12. N. Barakat, J. Diederich, Learning-based rule-extraction from support vector machines, in: Proceedings of the 14th International Conference on Computer Theory and Applications, Alexandia, Egypt, 2004.
13. I. Benbasat, and R.N. Taylor Behavioral aspects of information processing for the design of management information systems IEEE Transactions on Systems, Man, and Cybernetics 12 4 1982 439 450
14. C.M. Bishop Neural Networks for Pattern Recognition 1995 Oxford University Press Oxford
15. A. Bradley The use of the area under the ROC curve in the evaluation of machine learning algorithms Pattern Recognition 30 7 1997 1145 1159 (Pubitemid 127406521)
16. L. Breiman, J.H. Friedman, R.A. Olsen, and C.J. Stone Classification and Regression Trees 1984 Wadsworth & Books/Cole Advanced Books & Software Monterey, CA
17. J. Bridle Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition Neurocomputing: Algorithms, Architectures and Applications 1989 Springer France
18. J. Cannon-Browers, E. Salas, S. Tannenbaum, and J. Mathieu Toward theoretically based principles of training effectiveness: a model and initial empirical investigation Military Psychology 7 1995 141 164
19. W. Cascio Costing Human Resources: The Financial Impact of Behavior in Organizations fourth ed. 1999 Kent Publishing Boston
20. J. Catlett, Tailoring rulesets to misclassification costs, in: Proceedings of the 1995 Conference on AI and Statistics, 1995, pp. 88-94.
21. J. Chih, C. Liu, and H. Lee Relationship between trainee attitudes and dimensions of training satisfaction: an empirical study with training institute employees International Journal of Management 25 4 2008 756 763
22. N. Chimote Training programs: evaluation of trainees' expectations and experience The IUP Journal of Organizational Behaviour 9 3 2010 28 47
23. F. Davis Perceived usefulness, perceived ease of use, and user acceptance of information MIS Quarterly 3 13 1989 319 339
24. M. De Almeida, S. Matwin, Machine learning method for software quality model building, In: Proceedings of 11th International Symposium on Methodologies for Intelligent Systems, (1999) 565-573.
25. K. Dejaeger, W. Verbeke, D. Martens, B. Baesens, Data mining techniques for software effort estimation: a comparative study. IEEE Transactions on Software Engineering, 2011. Available from: < http://doi.ieeecomputersociety. org/10.1109/TSE.2011.55 >.
26. T. Dietterich, and G. Bakiri Solving multiclass learning via error-correcting output codes Journal of Artificial Intelligence Research 2 1995 263 286
27. P. Domingos The role of Occam's razor in knowledge discovery Data Mining and Knowledge Discovery 3 4 1999 409 425
28. J. Egan Signal Detection Theory and ROC Analysis 1975 Academic Press New York
29. T. Fawcett An introduction to ROC analysis Pattern Recognition Letters 27 8 2006 861 874
30. T. Fawcett, and F. Provost Adaptive fraud detection Data Mining and Knowledge Discovery 1 3 1997 291 316
31. C. Ferri, J. Hernández-Orallo, M. Salido, Volume under the ROC surface for multi-class problems, in: Proceedings of the 14th European Conference on Machine Learning, 2003, pp. 108-120. (Pubitemid 37230970)
32. R. Fletcher Practical Methods of Optimization 1981 John Wiley & sons Chichester
33. Y. Freund, and R. Schapire A decision-theoretic generalisation of on-line learning and an application to boosting Journal of Computer and System Sciences 55 1 1997 119 139 (Pubitemid 127433398)
34. M. Friedl, and C. Brodley Decision tree classification of land cover from remotely sensed data Remote Sensing of Environment 61 1997 399 409 (Pubitemid 28052029)
35. G. Fung, S. Sandilya, R. Rao, Rule extraction from linear support vector machines, in: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, 2005, pp. 32-40. (Pubitemid 43218265)
36. M. Gatta, and P. Ross Rethinking occupational integration Sociological Forum 20 3 2005 369 402 (Pubitemid 41436890)
37. A. Giangreco, A. Carugati, and A. Sebastiano Are we doing the right thing? Food for thoughts on training evaluation and its context Personnel Review 39 2 2010 162 177
38. A. Giangreco, A. Carugati, A. Sebastiano, and D. Della Bella Trainees' reactions to training: shaping groups and courses for happier trainees in an italian context The International Journal of Human Resources Management 21 13 2010 2468 2487
39. A. Giangreco, F. Goethals, Students' overall satisfaction with training in a business education institution going through international accreditation processes, Working paper, IESEG School of Management, 2011.
40. A. Giangreco, A. Sebastiano, and R. Peccei Trainees' reactions to training: an analysis of the factors affecting overall satisfaction with training The International Journal of Human Resources Management 20 1 2009 96 111
41. A. Hamblin Evaluation and Control of Training 1974 McGraw-Hill New York
42. D. Hand, and R. Till A simple generalisation of the area under the ROC curve for multiple class classification problems Machine Learning 45 2001 171 186 (Pubitemid 33635984)
43. J. Hanley, and B. McNeil The meaning and use of the area under a receiver operating characteristic (ROC) curve Radiology 143 1982 29 36 (Pubitemid 12142173)
44. T. Hastie, and R. Tibshirani Classification by pairwise coupling The Annals of Statistics 26 2 1998 451 471
45. T. Hastie, R. Tibshirani, and J. Friedman The Elements of Statistical Learning: Data Mining, Inference and Prediction second ed. 2009 Springer Science New York
46. J. Huysmans, K. Dejaeger, C. Mues, J. Vanthienen, and B. Baesens An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive systems Decision Support Systems 51 1 2011 141 154
47. L. Hyafil, and R. Rivest Constructing optimal decision trees is NP-complete Information Processing Letters 5 1 1976 15 17
48. R. Kaufman, J. Keller, and R. Watkins What works and what doesn't: evaluation beyond Kirkpatrick Performance & Instructions 35 2 1995 8 12
49. P. Kidder, and J. Rouiller Evaluating the success of a large-scale training effort National Productivity Review 16 2 1997 79 89
50. D. Kirkpatrick Evaluation of Training 1967 McGraw-Hill New York
51. D. Kirkpatrick Great ideas revisited Training and Development 50 1 1996 54 59
52. C. Kirsopp, M. Shepperd, Making inferences with small numbers of training sets, in: IEE Proceedings Software, vol. 149, 2002, pp. 123-130. (Pubitemid 35313946)
53. S. Kotsiantis, S. Zaharakis, and P. Pintelas Supervised machine learning: a review of classification techniques Informatica 31 2007 249 268 (Pubitemid 350208045)
54. S. Lee, and J. Pershing Dimensions and design criteria for developing training reactions evaluations Human Resources Development International 5 2 2002 175 197
55. S. Lessmann, B. Baesens, C. Mues, and S. Pietsch Benchmarking classification models for software defect prediction: a proposed framework and novel findings IEEE Transactions on Software Engineering 34 4 2008 485 496
56. W. Loh, and Y. Shih Split selection methods for classification trees Statistica Sinica 7 1997 815 840
57. L. Long, C. Dubois, and R. Faley Online training: the value of capturing trainee reactions Journal of Workplace Learning 20 1 2008 21 37
58. D. Martens, B. Baesens, T. Van Gestel, and J. Vanthienen Comprehensible credit scoring models using rule extraction from support vector machines European Journal of Operational Research 183 3 2007 1466 1476 (Pubitemid 47039190)
59. P. McCullagh, and J. Nelder Generalized Linear Models 1990 Chapman & Hall New York
60. T. Menzies, J. Greenwald, and A. Frank Data mining static code attributes to learn defect predictors IEEE Transactions on Software Engineering 32 11 2007 2 13 (Pubitemid 46002165)
61. R. Michalski A theory and methodology of inductive learning Artificial intelligence 20 2 1983 111 161
62. A. Monsifrot, F. Bodin, R. Quiniou, A machine learning approach to automatic production of compiler heuristics, in: Proceedings of the 10th International Conference on Artificial Intelligence: Methodology, Systems, Applications, 2002, pp. 41-50.
63. D. Mossman Three-way ROCs Medical Decision Making 19 1999 78 89 (Pubitemid 29031797)
64. S. Murthy, S. Kasif, and S. Salzberg A system for induction of oblique decision trees Journal of Artificial Intelligence Research 2 1994 1 32
65. OECD, Education at a Glance 2010: OECD indicators. Technical report, 2010. Available from: www.OECD.org.
66. H. Peng, F. Long, and C. Ding Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence 27 8 2005 1226 1238 (Pubitemid 41245053)
67. J. Phillips Return on Investment in Training and Performance Improvement Programs 2003 Butterworth-Heineman Boston
68. P. Pineda Evaluation of training in organisations: a proposal for an integrated model Journal of European Industrial Training 34 7 2010 673 693
69. F. Provost, and P. Domingos Tree induction for probability-based ranking Machine Learning 52 3 2003 199 215
70. F. Provost, T. Fawcett, R. Kohavi, The case against accuracy estimation for comparing induction algorithms, in: Proceedings of the 15th International Conference on Machine Learning, 1998.
71. G. Rabey Diagnose - then ACT. Some thoughts on Training Today Industrial and Commercial Training 39 3 2007 164 169
72. D. Russ-Eft, Expanding scope of evaluation in today's organizations, in: Proceedings of the 9th International Conference on HRD Research and Practice Across Europe, IESEG School of Management, Lille, France, 2008.
73. L. Sargent, B. Allen, J. Frahm, and G. Morris Enhancing the experience of student teams in large classes Journal of Management Education 33 5 2009 526 552
74. N. Sebe, M. Lew, Y. Sun, I. Cohen, T. Gevers, and T. Huang Authentic facial expression analysis Image and Vision Computing 25 12 2007 1856 1863 (Pubitemid 47333322)
75. C. Shannon, and W. Weaver The Mathematical Theory of Communication 1949 University of Illinois Press Urbana
76. A. Sinha, and H. Zhao Incorporating domain knowledge into data mining classifiers: an application in indirect lending Decision Support Systems 46 1 2008 287 299
77. T. Sitzmann, K. Brown, W. Casper, K. Ely, and R. Zimmerman A review and meta-analysis of the nomological network of trainee reactions Journal of Applied Psychology 93 2008 280 295
78. J.A.K. Suykens, T. Van Gestel, and J. De Brabanter Least Squares Support Vector Machines 2002 World Scientific Singapore
79. J.A.K. Suykens, and J. Vandewalle Least squares support vector machine classifiers Neural Processing Letters 9 1999 293 300
80. P.-N. Tan, M. Steinbach, and V. Kumar Introduction to Data Mining 2006 Pearson Addison Wesley Boston
81. A. Tickle, R. Andrews, M. Golea, and J. Diederich The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks IEEE Transactions on Neural Networks 9 6 1998 1057 1068 (Pubitemid 128742391)
82. A. Tziner, M. Fisher, T. Senior, and J. Weisber Effects of trainee characteristics on training effectiveness International Journal of Selection and Assessment 15 2 2007 167 174
83. T. Van Gestel, J.A.K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De Moor, and J. Vandewalle Benchmarking least squares support vector machine classifiers Machine Learning 54 2004 5 32
84. V.N. Vapnik Statistical Learning Theory 1998 John Wiley New York, USA
85. R. Velada, and A. Caetano Training transfer: the mediating role of percpetion of learning Journal of European Industrial Training 31 4 2007 283 296
86. V. Venkatesh, M. Morris, G. Davis, and F. David User acceptance of information technology: toward a unified view MIS Quarterly 3 27 2003 425 478
87. W. Verbeke, K. Dejaeger, D. Martens, and B. Baesens New insights into churn prediction in the telecommunication sector: a profit driven data mining approach European Journal of Operational Research 218 1 2012 211 229
88. P. Warr, M. Bird, N. Rackham, Evaluation of Management Training: A Practical Framework, with Cases, for Evaluating Training Needs and Results, Gower, London, 1970.
89. C. Wei, and I. Chiu Turning telecommunications call details to churn prediction: a data mining approach Expert Systems with Applications 23 2 2002 103 112
90. D. West Neural network credit scoring models Computers & Operations Research 27 11-12 2000 1131 1152 (Pubitemid 30398142)
91. I. Witten, and E. Frank Data Mining: Practical Machine Learning Tools and Techniques 2005 Morgan Kaufmann Publishers San Francisco
92. J.M. Zurada Introduction to Artificial Neural Systems 1995 PWS Publishing Company Boston