Experiment replication and meta-analysis in evaluation of intelligent tutoring system's effectiveness

WSEAS Transactions on Computers

Volumn 7, Issue 4, 2008, Pages 304-315

  Grubišić, Ani ^a   Stankov, Slavomir ^a   Žitko, Branko ^a  

^a UNIVERSITY OF SPLIT   (Croatia)

Author keywords

E learning; Effect size; Effectiveness; Evaluation; Experiment; Intelligent tutoring systems; Meta analysis; Replication

Indexed keywords

COMPUTER AIDED INSTRUCTION; E-LEARNING; EDUCATION COMPUTING; HELIUM; INTERNET; LEARNING SYSTEMS; MULTIMEDIA SYSTEMS; RESEARCH;

COMMON GROUND; CONTROLLED EXPERIMENTS; E-LEARNING SYSTEMS; EFFECT SIZE; EFFECTIVENESS; EFFECTIVENESS EVALUATION; EVALUATION; EXPERIMENT; INFLUENCE EVALUATION; INTELLIGENT TUTORING SYSTEM; INTELLIGENT TUTORING SYSTEMS; META-ANALYSIS; REPLICATION; SCIENTIFIC METHODS;

EXPERIMENTS;

EID: 48749101006     PISSN: 11092750     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (32)

1. Almqvist J. P. F. (2006) Replication of controlled Experiments in Empirical Software Engineering - A Survey. MS thesis, Department of Computer Science, Faculty of Science, Lund University.
2. Baruque, L.B., Melo, N.R. (2006). "Towards Metrics for the Assessment of Web-Based Education", in International Journal WSEAS Transactions on Computers, Vol. 5, Issue 11, November 2006, ISSN: 1109-2750, pp.2668-2673.
3. Becker, L.A. (2000) Online syllabus - Basic and Applied Research Methods. Retrieved 14/09/2007 from web.uccs.edu/lbecker/Psy590/ default.html
4. Bloom, B.S. (1984) The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring. Educational Researcher, 13, pp. 4-16.
5. Cumming, G. (2006). Meta-analysis: Pictures that explain how experimental findings can be integrated. In A. Rossman and B. Chance (Eds.), Proceedings of the Seventh Interantional Conference on Teaching Statistics, Salvador, Brazil. Voorburg: The Netherlands: International Statistical Institute.
6. Deligiannis I. S., Shepperd M., Webster S., Roumeliotis M. (2002) A review of experimental investigations into object-oriented technology. Empirical Software Engineering, 7(3), pp. 193-231.
7. Dempster, J. (2004) Evaluating e-learning developments: An overview. Retrieved 14/09/2007 from warwick.ac.uk/go/cap/resources/eguides
8. Farris T. (2006) Experimental error. Pearson Custom Publishing, Retrieved 14/09/2007 from www2.volstate.edu/tfarris/PHYS2110-2120/ experimental_error.htm
9. Fletcher, J.D. (2003) Evidence for Learning from Technology-Assisted Instruction. In: Technology applications in education: a learning view, (H.F. O'Neal, R.S. Perez (Ed.)), Mahwah, NJ: Lawrence Erlbaum Associates, pp. 19-99.
10. Glass, G. V (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, pp. 3-8.
11. Grubišić, A., Stankov, S., Žitko, B. (2007). "EVEDIN: A System for Automatic Evaluation of Educational Influence", in International Journal WSEAS Transactions on Computers, Vol. 6, Issue 1, January 2007, ISSN: 1109-2750, pp.95-102.
12. Hedges, L.V., Olkin, T. (1985) Statistical methods for meta-analysis. New York: Academic Press.
13. Hunt, M. (1997). How Science Takes Stock. The Story of Meta-Analysis. New York: Sage.
14. Iqbal, A., Oppermann, R., Patel, A., Kinshuk. (1999) A Classification of Evaluation Methods for Intelligent Tutoring Systems., In: Arend, U., Eberleh, E., Pitschke, K. (eds.): Software Ergonomie '99, B. G. Teubner, Stuttgart, Leipzig, pp. 169-181.
15. Lipsey, M.W., Wilson, D.B. (2001) Practical meta-analysis. Sage Publications, Inc., Thousand Oaks, California.
16. Litoiu M., Rolia J., Serazzi G. (2000) Designing process replication and activation: A quantitative approach. IEEE Transactions on Software Engineering, 26(12), pp. 1168-1178.
17. Lucas J. W. (2003) Theory-testing, generalization and the problem of external validity. Sociological Theory, 21(3), pp. 236-253.
18. Murray, L. R., Ehrenberg, A. S. C. (1993) The design of replicated studies. American Statistician, 47(3): pp. 217-228
19. Murray, T. (1996) Having It All, Maybe: Design Tradeoffs in ITS Authoring Tools. In Proceedings of the Third International Conference on Intelligent Tutoring Systems, Montreal
20. Ohlsson, S. (1987) Some Principles of Intelligent Tutoring. In Lawler & Yazdani (Eds.), Artificial Intelligence and Education, Volume 1. Ablex: Norwood, NJ, pp. 203-238.
21. Perseus Digital Library Project. Ed. Gregory R. Crane. Tufts University. Accessed on 21.02.2008. http://www.perseus.tufts.edu
22. Pfahl, D. (2004) Evaluating the learning effectiveness of using simulations in software project management education: Results from a twice replicated experiment. Information and Software Technology (Elsevier), v46, pp. 127-147.
23. Pfleeger S.L. (1995) Experimental design and analysis in software engineering, part 2: How to set up an experiment. ACM SIGSOFT Software Engineering Notes, 20(1), pp. 22-26.
24. Phillips, R., Gilding, T. (2003) Approaches to evaluating the effect of ICT on student learning. ALT Starter Guide 8.
25. Rodriguez, D., Sicilia, M. A., Cuadrado-Gallego, J. J., Pfahl, D. (2006) e-Learning in Project Management Using Simulation Models: A Case Study Based on the Replication of an Experiment. IEEE Transactions on Education 49(4), pp. 451-463.
26. Rosić, M. (2000) Establishing of Distance Education Systems within the Information Infrastructure. Faculty of Electrical Engineering and Computing, Zagreb, Croatia, MS Thesis (in Croatian)
27. Schulze, R. (2004). Meta-analysis: A comparison of approaches. Cambridge, MA: Hogrefe & Huber.
28. Sleeman, D., Brown, J. S. (1982) Introduction: Intelligent Tutoring Systems. In D. Sleeman & J. S. Brown (Eds.), Intelligent Tutoring Systems. New York: Academic Press. pp. 1-11.
29. Stankov, S. (1997): Isomorphic Model of the System as the Basis of Teaching control Principles in an Intelligent Tutoring System. Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, Croatia, PhD Thesis (in Croatian)
30. Stankov, S., Glavinić, V., Grubišić, A. (2004) What is Our Effect Size: Evaluating the Educational Influence of a Web-Based Intelligent Authoring Shell?, Proceedings INES 2004 8th International Conference on Intelligent Engineering Systems, Nedevschi, Sergiu; Rudas, Imre (eds.). Cluj-Napoca: Faculty of Automation and Computer Science, Technical University of Cluj-Napoca, 2004., pp. 545-550
31. Stankov, S., Rosić, M., Žitko, B., Grabišić, A. (in press): TEx-Sys model for building intelligent tutoring systems, Computers & Education (2007), doi:10.1016/j.compedu.2007.10.002
32. Wenger, E. (1987) Artificial Intelligence and Tutoring Systems. Los Altos, California: Morgan Kaufmann Publishers.