Developing a generalizable detector of when students game the system

User Modeling and User-Adapted Interaction

Volumn 18, Issue 3, 2008, Pages 287-314

  Baker, Ryan S J D ^a   Corbett, Albert T ^a   Roll, Ido ^a   Koedinger, Kenneth R ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

Behavior detection; Cognitive tutors; Gaming the system; Generalizable models; Interactive learning environments; Latent response models; Machine learning; Student modeling

Indexed keywords

(PL) PROPERTIES; BUSINESS MEDIA; GENERALIZABILITY; INTERACTIVE LEARNING ENVIRONMENTS; LEARNING OUTCOMES; MATHEMATICS CURRICULA; SPRINGER (CO); TUTOR LESSONS;

COMPUTER NETWORKS; CURRICULA; DETECTORS; EDUCATION; STUDENTS;

GAME THEORY;

EID: 45449102857     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-007-9045-6     Document Type: Article

References (40)

1. Aleven, V.: Helping students to become better help seekers: towards supporting metacognition in a cognitive tutor. Paper presented at German-USA Early Career Research Exchange Program: Research on Learning Technologies and Technology-Supported Education. Tubingen, Germany (2001)
2. Aleven, V., McLaren, B.M., Roll, I., Koedinger, K.R.: Toward tutoring help seeking: applying cognitive modeling to meta-cognitive skills. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS 2004), pp. 227-239. Maceió, Brazil (2004)
3. Anderson, J.R., Corbett A.T., Koedinger, K.R., Koedinger, K.R., Pelletier, R.: Cognitive tutors: lessons learned. J. Learn. Sci. 4(2), 167-207 (1995)
4. Arroyo, I., Woolf, B.: Inferring learning and attitudes from a Bayesian Network of log file data. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 33-40. Amsterdam (2005)
5. Arroyo, I., Ferguson, K., Johns, J., Dragon, T., Meheranian, H., Fisher, D., Barto, A., Mahadevan, S., Woolf. B.P.: Repairing disengagement with non-invasive interventions. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education, pp. 195-202. Marina del Rey, CA (2007)
6. Baker, R.S., Corbett, A.T., Koedinger, K.R.: Detecting student misuse of intelligent tutoring systems. In: Proceedings of the 7th International Conference on Intelligent Tutoring Systems, pp. 531-540. Maceió, Brazil (2004)
7. Baker, R.S., Corbett, A.T., Koedinger, K.R., Wagner, A.Z.: Off-Task Behavior in the Cognitive tutor classroom: when students "Game The System". In: Proceedings of ACM CHI 2004: Computer-Human Interaction, pp. 383-390. Vienna, Austria (2004)
8. Baker, R.S.J.d., Corbett, A.T., Koedinger, K.R., Evenson, S.E., Roll, I., Wagner, A.Z., Naim, M., Raspat, J., Baker, D.J., Beck, J.: Adapting to when students game an intelligent tutoring system. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems, pp. 392-401. Jhongli, Taiwan (2006)
9. Baker, R.S.J.d., Walonoski, J.A., Heffernan, N.T., Roll, I., Corbett, A.T., Koedinger, K.R.: Why students engage in "Gaming the System" behavior in interactive learning environments. To appear in J. Interact. Learn. Res. (in press)
10. Beal, C.R., Mitra, S., Cohen, P.R.: Modeling learning patterns of students with a tutoring system using Hidden Markov Models. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education, pp. 238-245. Marina del Rey, CA (2007)
11. Beal, C.R., Qu, L., Lee, H.: Classifying learner engagement through integration of multiple data sources. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 2-8. Boston (2006)
12. Beck, J.: Engagement tracing: using response times to model student disengagement. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED 2005), pp. 88-95. Amsterdam (2005)
13. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge, UK (2004)
14. Cheng, R., Vassileva, J.: Adaptive reward mechanism for sustainable online learning community. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 152-159. Amsterdam (2005)
15. Conati, C., McLaren, H.: Data-driven refinement of a probabilistic model of user affect. In: Proceedings of the Tenth International Conference on User Modeling (UM2005), pp. 40-49. Edinburgh, Scotland (2005)
16. Corbett, A.T., Anderson, J.R.: Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction 4, 253-278 (1995)
17. Corbett, A.T., Koedinger, K.R., Hadley, W.H.: Cognitive tutors: from the research classroom to all classrooms. In: Goodman, P.S.(eds) Technology Enhanced Learning: Opportunities for Change, pp. 235-263. Lawrence Erlbaum Associates, Mahwah, NJ (2001)
18. D'Mello, S.K., Craig, S.D., Witherspoon, A.W., McDaniel, B.T., and Graesser, A.C.: Automatic detection of learner's affect from conversational cues. To appear in User Modeling and User-Adapted Interaction. (in press)
19. de Vicente, A. and Pain, H.: Informing the detection of the students' motivational state: an empirical study. In: Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, pp. 933-943. Biarritz, France (2002)
20. Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve. Radiology 143, 29-36 (1982)
21. Johns, J., Woolf, B.: A dynamic mixture model to detect student motivation and proficiency. In: Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-06), pp. 163-168. Boston (2006)
22. Koedinger, K.R.: Toward evidence for instructional design principles: Examples from Cognitive Tutor Math 6. In: Proceedings of PME-NA XXXIII (the North American Chapter of the International Group for the Psychology of Mathematics Education), pp. 21-49. Athens, GA (2002)
23. Liu, H., Singh, P.L.: MAKEBELIEVE: using commonsense knowledge to generate stories. In: Proceedings of the 18th National Conference on Artificial Intelligence, AAAI 2002, pp. 957-958. Edmonton, Canada (2002)
24. Magnussen, R., and Misfeldt, M.: Player transformation of educational multiplayer games. In: Proceedings of Other Players. Copenhagen, Denmark. Available at http://www.itu.dk/op/proceedings.htm (2004). Accessed 13 May 2007.
25. Maris, E.: Psychometric latent response models. Psychometrika 60(4), 523-547 (1995)
26. Mostow, J., Aist, G., Beck, J., Chalasani, R., Cuneo, A., Jia, P., Kadaru, K.: A La Recherche du Temps Perdu, or As Time Goes By: where does the time go in a Reading Tutor that listens? In: Proceedings of the Sixth International Conference on Intelligent Tutoring Systems (ITS'2002), pp. 320-329. Biarritz, France (2002)
27. Murray, R.C., vanLehn, K.: Effects of dissuading unnecessary help requests while providing proactive help. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 887-889. Amsterdam (2005)
28. Ramsey, F.L., Schafer, D.W.: The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press, Belmont, CA (1997)
29. Rodrigo, M.M.T., Baker, R.S.J.d., Lagud, M.C.V., Lim, S.A.L., Macapanpan, A.F., Pascua, S.A.M.S., Santillano, J.Q., Sevilla, L.R.S., Sugay, J.O., Tep, S., Viehland, N.J.B.: Affect and usage choices in simulation problem solving environments. In: Proceedings of the 13th International Conference on Artificial Intelligence in Education, pp. 145-152. Marina del Rey, CA (2007)
30. Roll, I., Baker, R.S., Aleven, V., McLaren, B.M., Koedinger, K.R.: Modeling students' metacognitive errors in two intelligent tutoring systems. In: Proceedings of User Modeling 2005, pp. 379-388. Edinburgh, Scotland (2005)
31. Roll, I., Aleven, V., McLaren, B.M., and Koedinger, K.R.: Can help seeking be tutored? Searching for the secret sauce of metacognitive tutoring. In; Proceedings of the 13th International Conference on Artificial Intelligence in Education, pp. 203-210. Marina del Rey, CA (2007)
32. Rosenthal, R., Rosnow, R.: Essentials of Behavioral Research: Methods and Data Analysis. McGraw-Hill, Boston, MA (1991)
33. Schofield, J.W.: Computers and Classroom Culture. Cambridge University Press, Cambridge, UK (1995)
34. Strube, M.J.: Combining and comparing significance levels from nonindependent hypothesis tests. Psychol. Bull. 97, 334-341 (1985)
35. Tait, K., Hartley, J., Anderson, R.C.: Feedback procedures in computer-assisted arithmetic instruction. Br. J. Educ. Psychol. 43, 161-171 (1973)
36. van Lehn, K., Lynch, C., Shulze, K., Shapiro, J.A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., Wintersgill, M.: The Andes physics tutoring system: Five years of evaluations. In: Proceedings of the 12th International Conference on Artificial Intelligence in Education, pp. 678-685. Amsterdam (2005)
37. Walonoski, J.A., Heffernan, N.T.: Detection and analysis of off-task gaming behavior in intelligent tutoring systems. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems, pp. 382-391. Jhongli, Taiwan (2006a)
38. Walonoski, J.A., Heffernan, N.T.: Prevention of off-task gaming behavior in intelligent tutoring systems. In: Proceedings of the 8th International Conference on Intelligent Tutoring Systems, pp. 722-724. Jhongli, Taiwan (2006b)
39. Wood, H., Wood, D.: Help seeking, learning, and contingent tutoring. Comp. Educ. 33, 153-169 (1999)
40. Yu, L., Liu, H.: Feature selection for high-dimensional data: a fast correlation-based filter solution. In: Proceedings of the International Conference on Machine Learning, pp. 856-863. Washington, DC (2003)