A machine learning-based usability evaluation method for eLearning systems

Decision Support Systems

Volumn 56, Issue 1, 2013, Pages 63-73

  Oztekin, Asil ^a   Delen, Dursun ^b   Turkyilmaz, Ali ^c   Zaim, Selim ^d  

^a UNIVERSITY OF MASSACHUSETTS LOWELL   (United States)

^b OKLAHOMA STATE UNIVERSITY   (United States)

^c FATIH UNIVERSITY   (Turkey)

^d MARMARA UNIVERSITY   (Turkey)

Author keywords

eLearning (web based learning distance learning); Information fusion; Machine learning; Sensitivity analysis; Severity index; Usability engineering

Indexed keywords

E-LEARNING SYSTEMS; MULTIPLE LINEAR REGRESSIONS; SENSITIVITY VALUES; SEVERITY INDEX; THREE MACHINE LEARNING METHODS; USABILITY EVALUATION METHODS; USABILITY FACTORS; WEB BASED LEARNING;

DECISION TREES; E-LEARNING; INFORMATION FUSION; LEARNING SYSTEMS; LINEAR REGRESSION; SENSITIVITY ANALYSIS;

USABILITY ENGINEERING;

EID: 84889088422     PISSN: 01679236     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dss.2013.05.003     Document Type: Article

References (74)

1. J.B. Arbaugha, and R. Benbunan-Fichb The importance of participant interaction in online environments Decision Support Systems 43 2007 853 865
2. J.S. Armstrong Combining forecasts J.S. Armstrong, Principles of Forecasting 2002 Kluwer Academic Publishers Norwell 418 439
3. E.M. Babiker, H. Fujihara, and C.D.B. Boyle A metric for hypertext usability Proceedings of 11th Annual International Conference on Systems Documentation 1991 ACM Press 95 104
4. R. Batchelor, and P. Dua Forecaster diversity and the benefits of combining forecasts Management Science 41 1995 68 75
5. M. Ben Ayed, H. Ltifi, C. Kolski, and A.M. Alimi A user-centered approach for the design and implementation of KDD-based DSS: a case study in the healthcare domain Decision Support Systems 50 2010 64 78
6. N. Bevan Measuring usability as quality of use Software Quality Journal 4 1995 115 140
7. N. Bevan Quality in use: meeting user needs for quality Journal of Systems and Software 49 1999 89 96
8. P. Bharatia, and A. Chaudhury An empirical investigation of decision-making satisfaction in web-based decision support systems Decision Support Systems 37 2004 187 197
9. L. Breiman, J.H. Friedman, R.A. Olshen, and C.J. Stone Classification and Regression Trees 1984 Wadsworth & Brooks/Cole Advanced Books Monterey, Ca.
10. J. Brooke SUS: a "Quick and Dirty" usability scale P. Jordan, B. Thomas, B. Weerdmeester, Usability Evaluation in Industry 1996 Taylor and Francis London 189 194
11. C.W. Chase Jr. Composite forecasting: combining forecasts for improved accuracy Journal of Business Forecasting Methods & Systems 19 2000 2 22
12. J.P. Chin, V.A. Diehl, and K.L. Norman Development of an instrument measuring user satisfaction of the human-computer interface Proceedings of CHI'88 1988 ACM Press Washington, D.C. 213 218
13. C.M. Chiu, M.H. Hsu, S.Y. Sun, T.C. Lin, and P.C. Sun Usability, quality, value and E-learning continuance decisions Computers in Education 45 2005 399 416
14. J. Cohen, P. Cohen, S.G. West, and L.S. Aiken Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences 2003 L. Erlbaum Associates Mahwah New Jersey
15. N. Cristianini, and J. Shawe-Taylor An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods 2000 Cambridge University Press London
16. G. Davis Sensitivity analysis in neural net solutions IEEE Transactions on Systems, Man, and Cybernetics 19 1989 1078 1082
17. D. Delen, and S. Hawamdeh A holistic framework for knowledge discovery and management Communications of the ACM 52 2009 141 145
18. D. Delen, and E. Sirakaya Determining the efficacy of data-mining methods in predicting gaming ballot outcomes Journal of Hospitality and Tourism Research 30 3 2006 313 332
19. D. Delen, R. Sharda, and P. Kumar Movie forecast guru: a web-based DSS for Hollywood managers Decision Support Systems 43 2007 1151 1170
20. S. Downey, R.M. Wentling, T. Wentling, and A. Wadsworth The relationship between national culture and the usability of an E-learning system Human Resource Development International 8 2005 47 64
21. B. Efron, and R. Tibshirani Statistical data analysis in the computer age Science 253 1991 390 395
22. X. Fang, and C.W. Holsapple An empirical study of web site navigation structures' impacts on web site usability Decision Support Systems 43 2007 476 491
23. S. Haykin Neural Networks and Learning Machines 3rd edition 2008 Prentice Hall New Jersey
24. A. Hope, and P. Guiton Strategies for Sustainable Open and Distance Learning 2006 Routledge Taylor and Francis Group London
25. K. Hornbaek Current practice in measuring usability: challenges to studies and research International Journal of Human Computer Studies 64 2006 79 102
26. K. Hornik, M. Stinchcombe, and H. White Universal approximation of an unknown mapping and its derivatives using multilayer feed-forward network Neural Networks 3 1990 359 366
27. C.M. Hsu, Y.C. Yeh, and J. Yen Development of design criteria and evaluation scale for web-based learning platforms International Journal of Industrial Ergonomics 39 2009 90 95
28. P.J.-H. Hu, and W. Hui Examining the role of learning engagement in technology-mediated learning and its effects on learning effectiveness and satisfaction Decision Support Systems 53 2012 782 792
29. IEEE IEEE Standard 161: Standard for a Software Quality Metrics Methodology 1998
30. ISO ISO 9241-11, 98: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTS)-Part 11: Guidance on Usability 1998
31. ISO ISO 13407, 99: Human-Centered Design Processes for Interactive Systems 1999
32. ISO/IEC ISO 9126: Information Technology - Software Quality Characteristics and Metrics 1991
33. N. Juristo, A.M. Moreno, and M.I. Sanchez-Segura Analyzing the impact of usability on software design Journal of Systems and Software 80 2007 1506 1516
34. G.V. Kass An exploratory technique for investigating large quantities of categorical data Applied Statistics 29 1980 119 127
35. B. Keevil Measuring the usability index of your web site Proceedings of the 16th Annual International Conference on Computer Documentation, SIGDOC 1998 ACM Press 271 277
36. J. Kirakowski The Software Usability Measurement Inventory (Sumi): background and usage P. Jordan, B. Thomas, B. Weerdmeester, Usability Evaluation in Industry 1996 Taylor and Francis London 169 178
37. J. Kirakowski, and M. Corbett SUMI: the software measurement inventory British Journal of Educational Technology 24 1993 210 212
38. R. Kohavi A study of cross-validation and bootstrap for accuracy estimation and model selection Proceedings of the 14th International Conference on AI (IJCAI) 1995 Morgan Kaufmann San Mateo, CA 1137 1145
39. Y. Lee, and K.A. Kozar Understanding of website usability: specifying and measuring constructs and their relationships Decision Support Systems 52 2012 450 463
40. J.R. Lewis Psychometric evaluation of the PSSUQ using data from five years of usability studies International Journal of Human Computer Interaction 14 2002 463 488
41. H.X. Lin, Y. Choong, and G. Salvendy A proposed index of usability: a method for comparing the relative usability of different software systems Behavior and Information Technology 16 1997 267 278
42. M. Macleod, and R. Rengger The development of drum: a software tool for video-assisted usability evaluation Proceedings of The HCI Conference on People and Computers VIII 1993 Cambridge University Press Loughborough, UK 293 309
43. S. Makridais, S.C. Wheelwright, and R.J. Hyndman Forecasting: Methods and Applications 1998 John Wiley and Sons New York
44. M. Matera, M.F. Costabile, F. Garzotto, and P. Paolini SUE inspection: an effective method for systematic usability evaluation of hypermedia IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 32 2002 93 103
45. M. Mcgee Master usability scaling: magnitude estimation and master scaling applied to usability measurement Proceedings of CHI 2004 2004 ACM Press Washington, DC 335 342
46. P. Mcnamara Usability and Elearning, Final Coursework Essay: Certificate in Online Education and Training, Institute of Education, University of London Available at: http://Nte.Unifr.Ch/Img/Pdf/Pmcnamarausabilityelearning.Pdf 2003
47. M.J. Miller Usability in Elearning, ASTD's Source for Elearning, Learning Circuit Available At: http://Www.Astd.Org/Lc/2005/0105-Miller.Htm 2005
48. T. Mitchell Machine Learning 1997 Mcgraw-Hill New York
49. R. Molich, and J. Nielsen Improving a human-computer dialogue Communications of the ACM 33 1990 338 348
50. Moodle A Free Open Source Course Management System for Online Learning 2011 (www.Moodle.Org and http://Moodle.Fatih.Edu.Tr)
51. J. Nielsen Usability Engineering 1993 Academic Press Professional Boston
52. J. Nielsen, and R.L. Mack Usability Inspection Methods 1994 John Wiley and Sons New York
53. K. Norman, and B. Schneiderman Questionnaire for User Interaction Satisfaction 1989 HCI Lab, College Park, University of Maryland
54. D.L. Olson, and D. Delen Advanced Data Mining Techniques 2008 Springer New York
55. A. Oztekin, Z.J. Kong, and O. Uysal UseLearn: a novel checklist and usability evaluation method for eLearning systems by criticality metric analysis International Journal of Industrial Ergonomics 40 2010 455 469
56. A. Oztekin, A. Nikov, and S. Zaim UWIS: an assessment methodology for usability of web-based information systems Journal of Systems and Software 82 2009 2038 2050
57. V. Pareto Manuale D'economia Politica (English Translation by A.M. Kelly) 1971
58. O. Parlangeli, E. Marchigiani, and S. Bagnara Multimedia systems in distance education: effects of usability on learning Interacting with Computers 12 1999 37 49
59. K. Pearson, and A. Lee On the generalized probable error in multiple normal correlation Biometrika 6 1908 59 68
60. J.C. Principe, D. Xu, and J.W. Fisher III Information-Theoretic Learning, in Unsupervised Adaptive Filtering 2000 John Wiley and Sons New York
61. J.C. Principe, N.R. Euliano, and W.C. Lefebvre Neural and Adaptive Systems 2001 John Wiley and Sons
62. J. Quinlan Induction of decision trees Machine Learning 1 1986 81 106
63. J. Quinlan C4.5: Programs for Machine Learning 1993 Morgan Kaufmann San Mateo
64. R. Rengger, M. Macleod, R. Bowden, A. Drynan, and M. Blaney Music Performance Measurement Handbook 1993 National Physical Laboratory Ditc, Teddington, UK
65. E. Ruiz, and F.H. Mieto A note on linear combination of predictors Statistics & Probability Letters 47 2000 351 356
66. A. Saltelli Making best use of model evaluations to compute sensitivity indices Computer Physics Communications 145 2002 280 297
67. A. Saltelli, S. Tarantola, F. Campolongo, and M. Ratto Sensitivity Analysis in Practice - A Guide to Assessing Scientific Models 2004 John Wiley and Sons
68. J. Sauro, and J.R. Lewis Correlations among prototypical usability metrics: evidence for the construct of usability Proceedings of The 27th International Conference on Human Factors in Computing Systems 2009 Association for Computing Machinery Boston, MA 1609 1618
69. B. Schneiderman Designing the User Interface: Strategies for Effective Human-Computer Interaction 1998 Addison Wesley Menlo Park, CA
70. A. Seffah, T. Mohamed, H. Habieb-Mammar, and A. Abran Reconciling usability and interactive system architecture using patterns Journal of Systems and Software 81 2008 1845 1852
71. D. Squires, and J. Preece Predicting quality in educational software: evaluating for learning, usability, and the synergy between them Interacting with Computers 11 1999 467 483
72. M.A. Storey, B. Phillips, M. Maczewski, and M. Wang Evaluating the usability of web-based learning tools Educational Technology & Society 5 2002 91 103
73. P.-C. Sun, H.K. Cheng, and G. Finger Critical functionalities of a successful e-learning system - an analysis from instructors' cognitive structure toward system usage Decision Support Systems 48 2009 293 302
74. S. Weisberg Applied Linear Regression 1980 John Wiley and Sons New York