An analysis of students' gaming behaviors in an intelligent tutoring system: Predictors and impacts

User Modeling and User-Adapted Interaction

Volumn 21, Issue 1-2, 2011, Pages 99-135

  Muldner, Kasia ^a   Burleson, Winslow ^b   Van De Sande, Brett ^b   Vanlehn, Kurt ^b  

^a ARIZONA STATE UNIVERSITY   (United States)

^b ARIZONA STATE UNIVERSITY   (United States)

Author keywords

Bayesian network parameter learning; Educational data mining; Gaming; Utility of hints

Indexed keywords

COGNITIVE TUTORS; DATA MINING TECHNIQUES; EDUCATIONAL DATA MINING; GAMING; HIGH SCHOOL STUDENTS; INSTRUCTIONAL SYSTEM; INTELLIGENT TUTORING SYSTEM; IT IMPACT; LEARNING OUTCOME; LOG DATA; MACHINE-LEARNING; NETWORK PARAMETERS; RESEARCH QUESTIONS; UTILITY OF HINTS;

BAYESIAN NETWORKS; COMPUTER AIDED INSTRUCTION; DATA MINING; DISTRIBUTED PARAMETER NETWORKS; EDUCATION COMPUTING; INFERENCE ENGINES; INTELLIGENT NETWORKS; POPULATION STATISTICS; PROBLEM SOLVING;

STUDENTS;

EID: 79955795644     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-010-9086-0     Document Type: Article

References (68)

1. Aleven, V.: Helping students to become better help seekers: towards supporting metacognition in a cognitive tutor. In: Paper Presented at German-USA Early Career Research Exchange Program: Research on Learning Technologies and Technology-Supported Education (2001)
2. V. Aleven B. McLaren I. Roll K. Koedinger 2006 Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor Int. J. Artif. Intell. Educ. 16 101 128
3. Arroyo I., Woolf B.: Inferring learning and attitudes from a Bayesian network of log file data. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED'05), Amsterdam, Netherlands, pp. 33-40 (2005)
4. Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf, B.: Repairing disengagement with non-invasive interventions. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education (AIED'07), Los Angeles, United States, pp. 195-202 (2007)
5. Baker, R.: Is gaming the system state-or-trait? educational data mining through the multi-contextual application of a validated behavioral model. In: Proceedings of the Workshop on Data Mining for User Modeling, Corfu, Greece, pp. 76-80 (2007)
6. Baker, R., Corbett, A., Koedinger, K.: Detecting student misuse of intelligent tutoring systems. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS'04), Maceio, Brazil, pp. 531-540 (2004a)
7. Baker, R., Corbett, A., Koedinger, K., Wagner, A.: Off-task behavior in the cognitive tutor classroom: when students "game the system". In: Proceedings of the ACM CHI 2004: Computer-Human Interaction (CHI'04), Vienna, Austria, pp. 383-390 (2004b)
8. Baker, R.S., Roll, I., Corbett, A., Koedinger, K.: Do performance goals lead students to game the system. In: Proceedings of the 12th International Conference on Artificial Intelligence and Education (AIED2005), Amsterdam, Netherlands, pp. 57-64 (2005)
9. Baker, R., Corbett, A., Koedinger, K., Evenson, E., Roll, I., Wagner, A., Naim, M., Raspat, J., Baker, D., Beck, J.: Adapting to when students game an intelligent tutoring system. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 392-401 (2006a) (Pubitemid 44117611)
10. Baker, R.S.J.d., Corbett, A.T., Koedinger, K., Roll, I.: Generalizing detection of gaming the system across a tutoring curriculum. In: Proceedings of the 11th International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 402-411 (2006b) (Pubitemid 44117612)
11. R. Baker A. Corbett I. Roll K. Koedinger 2008 Developing a generalizable detector of when students game the system User Model. User-Adap. Inter. 18 3 287 314 10.1007/s11257-007-9045-6
12. R. Baker J. Walonoski N. Heffernan I. Roll A. Corbett K. Koedinger 2008 Why students engage in "gaming the system". Behavior in interactive learning environments J. Interact. Learn. Res. 19 2 185 224
13. Baker, R., Corbett, A., Koedinger, K.: Educational software features that encourage and discourage "gaming the system". In: Proceedings of the 14th international conference on artificial intelligence in education (AIED'09), Brighton, UK, pp. 475-482 (2009)
14. Baker R.: Data mining for education. In: McGaw, B., Peterson, P., Baker, E., (eds) International Encyclopedia of Education, vol. 7, 3rd edn, pp. 112-118. Elsevier Oxford, UK (2010)
15. J.E. Beck P. Jia J. Mostow 2004 Using automated questions to assess reading comprehension, vocabulary, and effects of tutorial interventions Technol. Instr. Cogn. Learn. (TICL) 2 97 134
16. Beck, J., Chang, K., Mostow, J., Corbett, A.: Does help? Introducing the Bayesian evaluation and assessment methodology. In: Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS'08), Montreal, Canada, pp. 383-394 (2008)
17. Bunt, A., Conati, C., Muldner, K.: Scaffolding self-explanation to improve learning in exploratory learning environments. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS'04), Maceio, Brazil, pp. 656-667 (2004)
18. Carnegie Learning Inc.: Retrieved April 1, 2010, from http://www.carnegielearning.com/ (2010)
19. Chang, K., Beck, J., Mostow, J., Corbett, A.: A Bayes net toolkit for student modeling in intelligent tutoring systems. In: Proceedings of the 11th International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 104-113 (2006) (Pubitemid 44117583)
20. Chi, M.T.H.: How adaptive is an expert human tutor? In: Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS'10), Pittsburgh, United States, pp. 401-113 (2010)
21. Chi, M., VanLehn, K.: The impact of explicit strategy instruction on problem-solving behaviors across intelligent tutoring systems. In: Proceedings of the 29th annual conference of the cognitive science society, Nashville, Tennessee, pp. 167-172 (2007a)
22. Chi, M., VanLehn, K.: Accelerated future learning via explicit instruction of a problem solving strategy. In: Proceedings of the 13th international conference on artificial intelligence in education (AIED'07), Los Angeles, United States, pp. 409-416 (2007b)
23. M.T.H. Chi M. Bassok M. Lewis P. Reimann R. Glaser 1989 Self-explanations: how students study and use examples in learning to solve problems Cogn. Sci. 15 145 182
24. M.T.H. Chi S.A. Siler H. Jeong T. Yamauchi R.G. Hausmann 2001 Learning from human tutoring Cogn. Sci. 25 471 533 10.1207/s15516709cog2504-1
25. M.T.H. Chi M. Roy R. Hausmann 2008 Observing tutoring collaboratively: insights about tutoring effectiveness from vicarious learning Cogn. Sci. 32 2 301 341 10.1080/03640210701863396
26. Chi, M., VanLehn, K., Litman, D.: Do micro-level tutorial decisions matter: applying reinforcement learning to induce pedagogical tutorial tactics. In: Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS'10), Pittsburgh, United States, pp. 184-193 (2010a)
27. Chi, M., VanLehn, K., Litman, D., Jordan, P.: Inducing effective pedagogical strategies using learning context features. In: Proceedings of the 18th International Conference on User Modeling, Adaptation and Personalization (UMAP'10), Kona, Hawaii, pp. 147-158 (2010b)
28. Chi, M., VanLehn, K., Litman, D., Jordan P.: Empirically evaluating the Application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User model User-Adap. Inter. (to appear). Special issue on educational data mining for personalized educational systems
29. P. Cohen C. Beal 2009 Temporal data mining for educational applications Int. J. Softw. Inform. 3 1 31 46
30. C. Conati A. Gertner K. VanLehn 2002 Using Bayesian networks to manage uncertainty in student modeling User Model. User-Adap. Inter. 12 4 371 417 10.1023/A:1021258506583 1030.68630
31. C. Conati H. Maclaren 2009 Empirically building and evaluating a probabilistic model of user affect User Model. User-Adap. Inter. 19 3 267 303 10.1007/s11257-009-9062-8
32. A.T. Corbett J.R. Anderson 1995 Knowledge tracing: modeling the acquisition of procedural knowledge User Model. User-Adap. Inter. 4 4 253 278 10.1007/BF01099821
33. S. D'Mello S. Craig J. Sullins A. Graesser 2006 Predicting affective states expressed through an emote-aloud procedure from autotutor's mixed-initiative dialogue Int. J. Artif. Intell. Educ. 16 1 3 28
34. W. Davis C. Carson A. Ammeter D. Treadway 2005 The interactive effects of goal orientation and feedback specificity on task performance Human Perform. 18 409 426 10.1207/s15327043hup1804-7 (Pubitemid 41634648)
35. T. Dean K. Kanazawa 1989 A model for reasoning about persistence and causation Comput. Intell. 5 3 142 150 10.1111/j.1467-8640.1989.tb00324.x
36. L. Dempster N. Laird D. Rubin 1977 Maximum likelihood from incomplete data via the EM algorithm J. R. Stat. Soc. 39 1 1 38 0364.62022 501537
37. Feng, M., Beck, J., Heffernan, N.T.: Using learning decomposition and bootstrapping with randomization to compare the impact of different educational interventions on learning. In: Proceedings of the 2nd International Conference on Educational Data Mining (EDM'09), Cordoba, Spain, pp. 51-60 (2009)
38. L. Getoor J. Rhee D. Koller P. Small 2004 Understanding tuberculosis epidemiology using probabilistic relational models J. Artif. Intell. Med. 30 233 256 10.1016/j.artmed.2003.11.003 (Pubitemid 38471895)
39. Hays, M., Lane, C., Auerbach, D., Core, M., Gomboc, D., Rosenberg, M.: Feedback specificity and the learning of intercultural communication skills. In: Proceedings of the 14th International Conference on Artificial Intelligence in Education (AIED'09), Brighton, UK, pp. 391-398 (2009)
40. D. Heckerman 1997 Bayesian networks for data mining Data Min. Knowl. Discov. 1 79 119 10.1023/A:1009730122752 (Pubitemid 127078360)
41. Koedinger, K.R., Anderson, Hadley, W.H., Mark, M.A.: Intelligent tutoring goes to school in the big city. In: Proceedings of the 3rd International Conference on Artificial Intelligence and Education (AIED'95), Washington, DC, United States, pp. 421-428 (1995)
42. M. Mayo A. Mitrovic 2001 Optimizing ITS behaviour with Bayesian networks and decision theory Int. J. Artif. Intell. Educ. 12 124 153
43. Muldner, K., Conati, C.: Using similarity to infer meta-cognitive behaviors during analogical problem solving. In: Proceedings of the 10th International Conference on User Modeling (UM'05), Edinborough, Scottland, pp. 134-143 (2005)
44. Muldner, K., Burleson, W., de Sande, B., VanLehn, K.: An analysis of gaming behaviors in an intelligent tutoring system. In: Proceedings of the 10th International Conference on Intelligent Tutoring Systems (ITS'10), Pittsburgh, United States, pp. 195-205 (2010)
45. K. Muldner C. Conati 2010 Scaffolding meta-cognitive skills for effective analogical problem solving via tailored example selection Int. J. Artif. Intell. Educ. 20 2 99 136
46. Murphy, K.: Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. Thesis, UC, Berkeley (2002)
47. Murphy, K.: Bayes Net Toolbox for Matlab. http://bnt.sourceforge.net. Accessed 2010 April 1 (2004)
48. Murray, R.C., VanLehn, K.: Effects of dissuading unnecessary help requests while providing proactive help. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED'05), Amsterdam, Netherlands, pp. 887-889 (2005)
49. S. Nelson-Le Gall 1985 Help-seeking behavior in learning review of research in education Rev. Res. Educ. 12 55 90
50. Netica Reference Manual. Retrieved March 1, 2010, from www.norsys.com/netica-j/docs/NeticaJ-Man.pdf (2010)
51. G.D. Phye C. Sanders 1994 Advice and feedback: elements of practice for problem solving Contemp. Educ. Psychol. 19 3 286 301 10.1006/ceps.1994.1022
52. Rai, D., Gong, Y., Beck, J.: Using Dirichlet priors to improve model parameter plausibility. In: Proceedings of the international conference on educational data mining (EDM'09), pp. 141-150 (2009)
53. Razzaq, L., Heffernan, N.: To tutor or not to tutor: that is the question. In: Proceedings of th 2nd International Conference on Artificial Intelligence in Education (AIED'09), Cordoba, Spain, pp. 457-464 (2009)
54. A. Renkl 1997 Learning from worked-examples: a study on individual differences Cogn. Sci. 21 1 1 30 10.1207/s15516709cog2101-1
55. J. Reye 2004 Student modeling based on belief networks Int. J. Artif. Intell. Educ. 14 1 33
56. Ritter, S., Harris, T., Nixon, T., Dickison, D., Murray, R.C., Towle, B.: Reducing the knowledge tracing space. In: Proceedings of the 1st International Conference On Educational Data Mining (EDM'08), Montreal, Canada, pp. 151-160 (2008)
57. A. Rodin T.H. Mosley A.G. Clark C.F. Sing E. Boerwinkle 2005 Mining genetic epidemiology data with Bayesian networks application to APOE gene variation and plasma lipid levels J. Comput. Biol. 12 1 1 11 10.1089/cmb.2005.12.1 (Pubitemid 40204478)
58. Rodrigo, M., Baker, R., d'Mello, S., Gonzalez, M.C.T., Lagud, M., Lim, S., Macapanpan, A., Pascua, S., Santillano, J., Sugay, J., Tep, S., Viehland, N.: Comparing learners affect while using an intelligent tutoring systems and a simulation problem solving game. In: Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS'08), Montreal, Canada, pp. 40-49 (2008)
59. Roll, I., Aleven, V., McLaren, B., Ryu, E., Baker, R.S.J.d., Koedinger, K.: The help tutor: does metacognitive feedback improve students' help-seeking actions, skills and learning? In: Proceedings of the Eight International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 360-369 (2006) (Pubitemid 44117608)
60. Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Los Altos, CA, Morgan-Kaufman (2009)
61. Shih, B., Koedinger, K., Scheines, R.: A response time model for bottom-out hints as worked examples. In: Proceedings of the 1st International Conference on Educational Data Mining (EDM'08), Montreal Canada, pp. 117-126 (2008)
62. V. Shute 2008 Focus on formative feedback Rev. Educ. Res. 78 1 153 189 10.3102/0034654307313795
63. K. VanLehn 1998 Analogy events: how examples are used during problem solving Cogn. Sci. 22 3 347 388 10.1207/s15516709cog2203-4
64. K. VanLehn C. Lynch K. Schulze J.A. Shapiro R. Shelby L. Taylor D. Treacy A. Weinstein system WintersgillM.TheAndesphysics tutoring 2005 Lessons learned Int. J. Artif. Intell. Educ. 15 3 1 47
65. VassarStats: Website for Statistical Computation. Retrieved April 1, 2010, from http://faculty.vassar.edu/lowry/VassarStats.html
66. Walonoski, J.A., Heffernan, N.T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 382-391 (2006a) (Pubitemid 44117610)
67. Walonoski, J.A., Heffernan, N.T.: Prevention of off-task gaming behavior in intelligent tutoring systems. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems (ITS'06), Jhongli, Taiwan, pp. 722-724 (2006b) (Pubitemid 44117652)
68. Zhang, X., Mostow, J., Beck, J.: A case study empirical comparison of three methods to evaluate tutorial behaviors. In: Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS'08), Montral, Canada, pp. 122-131 (2008)