Controlled experiment replication in evaluation of e-learning system's educational influence

Computers and Education

Volumn 53, Issue 3, 2009, Pages 591-602

  Grubišić, Ani ^a   Stankov, Slavomir ^a   Rosić, Marko ^a   Žitko, Branko ^a  

^a UNIVERSITY OF SPLIT   (Croatia)

Author keywords

Authoring tools and methods; Distributed learning environments; Evaluation methodologies; Intelligent tutoring systems

Indexed keywords

ARBITRARY NUMBER; AUTHORING TOOLS AND METHODS; CONTROLLED EXPERIMENT; DISTRIBUTED LEARNING ENVIRONMENTS; DOMAIN KNOWLEDGE; EFFECT SIZE; EFFECTIVENESS EVALUATION; EVALUATION METHODOLOGIES; EXPERIMENTAL DESIGN; INTELLIGENT TUTORING SYSTEMS; INTERMEDIATE STATE; POST TEST; REPLICATED EXPERIMENT;

COMPUTER AIDED INSTRUCTION; E-LEARNING; EXPERT SYSTEMS; INTERNET; MULTIMEDIA SYSTEMS; STATISTICS;

EXPERIMENTS;

EID: 67650233501     PISSN: 03601315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.compedu.2009.03.014     Document Type: Article

References (31)

1. Albert D. E-learning future - The contribution of psychology. In: Roth R., Lowenstein L., and Trent D. (Eds). Catching the future: Women and men in global psychology, proceedings of the 59th annual convention, international council of psychologists (2001), Winchester, England 30-53
2. Almqvist, J. P. F. (2006). Replication of controlled experiments in empirical software engineering - A survey. MS Thesis. Department of Computer Science, Faculty of Science, Lund University.
3. ASTD. (2001). A vision of e-learning for America's workforce. Report of the commission on technology and adult learning.
4. Becker, L. A. (2000). Online syllabus - Basic and applied research methods. Retrieved 14/09/2007 from .
5. Bloom B.S. The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher 13 (1984) 4-16
6. Cohen J. Statistical power analysis for the behavioral sciences (1988), Lawrence Erlbaum Ass, Hillsdale, NJ
7. Deligiannis I.S., Shepperd M., Webster S., and Roumeliotis M. A review of experimental investigations into objectoriented technology. Empirical Software Engineering 7 3 (2002) 193-231
8. Dempster, J. (2004). Evaluating e-learning developments: An overview. Retrieved 14/09/2007 from .
9. Farris, T. (2006). Experimental error. Pearson Custom Publishing, Retrieved 14/09/2007 from .
10. Gaines D.R. The role of ancillary variables in the design, analysis, and interpretation of animal experiments. ILAR Journal V43 4 (2002) 214-222
11. Grubišić, A., Stankov, S., & Žitko, B. (2006). An approach to automatic evaluation of educational influence. In Proceedings of the 6th WSEAS international conference on distance learning and web engineering (DIWEB 06) (pp. 20-25). Lisabon, Portugal.
12. Iqbal A., Oppermann R., Patel A., and Kinshuk. A classification of evaluation methods for intelligent tutoring systems. In: Arend U., Eberleh E., and Pitschke K. (Eds). Software Ergonomie 99 (1999), B.G. Teubner, Stuttgart, Leipzig 169-181
13. Litoiu M., Rolia J., and Serazzi G. Designing process replication and activation: A quantitative approach. IEEE Transactions on Software Engineering 26 12 (2000) 1168-1178
14. Lucas J.W. Theory-testing, generalization, and the problem of external validity. Sociological Theory 21 3 (2003) 236-253
15. Mark M.A., and Greer J.E. Evaluation methodologies for intelligent tutoring systems. Journal of Artificial Intelligence and Education 4 2/3 (1993) 129-153
16. Murray R.L., and Ehrenberg A.S.C. The design of replicated studies. American Statistician 47 3 (1993) 217-228
17. Murray, T. (1996). Having it all, maybe: Design tradeoffs in ITS authoring tools. In Proceedings of the third international conference on intelligent tutoring systems, Montreal.
18. Ohlsson S. Some principles of intelligent tutoring. In: Lawler, and Yazdani (Eds). Artificial intelligence and education Vol. 1 (1987), Ablex, Norwood, NJ 203-238
19. Pfahl D. Evaluating the learning effectiveness of using simulations in software project management education: Results from a twice replicated experiment. Information and software technology Vol. 46 (2004), Elsevier pp. 127-147
20. Pfleeger S.L. Experimental design and analysis in software engineering, part 2: How to set up an experiment. ACM SIGSOFT Software Engineering Notes 20 1 (1995) 22-26
21. Phillips, R., Gilding, T. (2003). Approaches to evaluating the effect of ICT on student learning. ALT starter guide 8.
22. Rodríguez D., Sicilia M.A., Cuadrado-Gallego J.J., and Pfahl D. E-learning in project management using simulation models: A case study based on the replication of an experiment. IEEE Transactions on Education 49 4 (2006) 451-463
23. Rosić, M. (2000) Establishing of distance education systems within the information infrastructure. Faculty of electrical engineering and computing, Zagreb, Croatia, MS Thesis (in Croatian).
24. Sleeman D., and Brown J.S. Introduction: Intelligent tutoring systems. In: Sleeman D., and Brown J.S. (Eds). Intelligent tutoring systems (1982), Academic Press, New York 1-11
25. Stankov, S. (1997). Isomorphic model of the system as the basis of teaching control principles in an intelligent tutoring system. Faculty of electrical engineering, Mechanical Engineering and Naval Architecture, Croatia, PhD Thesis (in Croatian).
26. Stankov S., Grubišić A., and Žitko B. E-learning paradigm and Intelligent tutoring systems. In: Kniewald Z. (Ed). Annual 2004 of the Croatian academy of engineering (2004), Croatian Academy of Engineering, Zagreb 21-31
27. Stankov, S., Glavinić, V., & Grubišić, A. (2004). What is our effect size: Evaluating the educational influence of a web-based intelligent authoring shell? In IEEE international conference on intelligent engineering systems 2004 - INES 2004, Cluj-Napoca, Romania.
28. Stankov, S., Grubišić, A., Žitko, B., Krpan, D. (2005). Vrednovanje učinkovitosti procesa učenja i poučavanja u sustavima za e-učenje, Školski Vjesnik - časopis za pedagoška i školska pitanja, 54 (2005), 1-2; 21-31 (in Croatian).
29. Stankov, S. (2005). Principal investigating project TP-02/0177-01 web oriented intelligent hypermedial authoring shell. Ministry of Science and Technology of the Republic of Croatia, 2003-2005.
30. Trochim, W. M. (2006). The research methods knowledge base (2nd ed.). Retrieved 14/09/2007 from .
31. Wenger E. Artificial intelligence and tutoring systems (1987), Morgan Kaufmann Publishers, Los Altos, California