Game-based assessment: An integrated model for capturing evidence of learning in play

International Journal of Learning Technology

Volumn 9, Issue 2, 2014, Pages 111-138

  Halverson, Richard ^a   Owen, V Elizabeth ^a  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

ADAGE; Digital media and learning; Educational data mining; Educational games; Embedded assessment; Evidence centred design; Game based assessment; Game based learning; Learning analytics; Technology based assessment

Indexed keywords

EID: 84928478783     PISSN: 14778386     EISSN: 17418119     Source Type: Journal    
DOI: 10.1504/IJLT.2014.064489     Document Type: Article

References (64)

1. Aarseth, E. (2007) 'I fought the law: transgressive play and the implied player', in Akira, B. (Ed.): Proceedings of DiGRA 2007 Conference: Situated Play, pp.130-133, The University of Tokyo, Tokyo.
2. Anaya, A.R. and Boticario, J.G. (2011) 'Content-free collaborative learning modeling using data mining', User Modeling and User-Adapted Interaction, Vol. 21, Nos. 1-2, pp.181-216.
3. Baker, R., Scheines, R. and Koedinger, K. (2008) Educational Data Mining [online] http://www.learnlab.org/opportunities/summer/presentations/2008/ 3.DataMiningTues315-v3.ppt (accessed 23 February 2013).
4. Baker, R.S., Corbett, A.T. and Koedinger, K.R. (2004) 'Detecting student misuse of intelligent tutoring systems', in Intelligent Tutoring Systems, pp.531-540, Springer, Berlin, Heidelberg.
5. Baker, R.S., Pardos, Z.A., Gowda, S.M., Nooraei, B.B. and Heffernan, N.T. (2011) 'Ensembling predictions of student knowledge within intelligent tutoring systems', in Konstan, J.A., Conejo, R., Marzo, J.L. and Oliver, N. (Eds.): User Modeling, Adaption and Personalization, pp.13-24, Springer-Verlag, Berlin, Heidelberg.
6. Baker, R.S.J.D. and de Carvalho, A.M.J.A. (2008) 'Labeling student behavior faster and more precisely with text replays', in Proceedings of the 1st International Conference on Educational Data Mining, pp.38-47.
7. Baker, R.S.J.D. and Yacef, K. (2009) 'The state of educational data mining in 2009: a review and future visions', Journal of Educational Data Mining, Vol. 1, No. 1, pp.3-17.
8. Beal, C.R. and Cohen, P.R. (2008) 'Temporal data mining for educational applications', in Proceedings of the 10th Pacific Rim International Conference on Artificial Intelligence, pp.66-77.
9. Beall, M., Farajian, R., Owen, V.E., Slater, S., Smith, A., Solis, E., Steinkuehler, C. and Davidson, R. (2013) 'Games for mindfulness and pro-social behavior: the tenacity project collaboration', Presented at the 9th Annual Games+Learning+Society Conference, Madison, 11-14 June.
10. Behrens, J.T., Mislevy, R.J., DiCerbo, K.E. and Levy, R. (2012) 'Evidence centered design for learning and assessment in the digital world', in Mayrath, M.C., Clarke-Midura, J., Robinson, D.H. and Schraw, G. (Eds.): Technology-based Assessments for 21st Century Skills: Theoretical and Practical Implications from Modern Research, pp.13-53, Information Age Publishing, Charlotte, NC.
11. Brown, J.S., Collins, A. and Duguid, P. (1989) 'Situated cognition and the culture of learning', Educational Researcher, Vol. 18, No. 1, pp.32-42.
12. Clark, D.B. and Martinez-Garza, M. (2012) 'Prediction and explanation as design mechanics in conceptually integrated digital games to help players articulate the tacit understandings they build through game play', in Steinkuehler, C., Squire, K. and Barab, S. (Eds.): Games, Learning, and Society: Learning and Meaning in the Digital Age, pp.279-305, Cambridge University Press, Cambridge.
13. Clark, D.B., Martinez-Garza, M.M., Biswas, G., Luecht, R.M. and Sengupta, P. (2012) 'Driving assessment of students' explanations in game dialog using computer-adaptive testing and hidden Markov modeling', Assessment in Game-Based Learning, pp.173-199, Springer, New York.
14. Clarke-Midura, J., Code, J., Dede, C., Mayrath, M. and Zap, N. (2012) 'Thinking outside the bubble: virtual performance assessments for measuring complex learning', in Mayrath, M.C., Clarke-Midura, J., Robinson, D.H. and Schraw, G. (Eds.): Technology-based Assessments for 21st Century Skills: Theoretical and Practical Implications from Modern Research, pp.125-148, Information Age Publishing, Charlotte, NC.
15. Dankoff, J. (2011) Game Telemetry with Playtest DNA on Assassin's Creed [online] http://engineroom.ubi.com/game-telemetry-with-playtest-dna-on-assassins (accessed 28 March 2013).
16. Feng, M. and Heffernan, N. (2006) 'Informing teachers live about student learning: reporting in the assessment system', Technology, Instruction, Cognition, and Learning Journal, Vol. 3, Nos. 1/2, p.63.
17. Feng, M., Beck, J.E. and Heffernan, N.T. (2009) 'Using learning decomposition and bootstrapping with randomization to compare the impact of different educational interventions on learning', in International Conference on Educational Data Mining, pp.51-60.
18. Feng, M., Heffernan, N.T. and Koedinger, K.R. (2010) 'Student modeling in an intelligent tutoring system', in Stankov, S., Glavinc, V. and Rosic, M. (Eds.): Intelligent Tutoring Systems in E-learning Environments: Design, Implementation and Evaluation, pp.208-236, Information Science Reference, Hershey, PA.
19. Fok, A.W.P., Wong, H.S. and Chen, Y.S. (2005) 'Hidden Markov model based characterization of content access patterns in an e-learning environment', in IEEE International Conference on Multimedia and Expo, Amsterdam, Netherlands, pp.201-204.
20. Gagné, A.R., Seif El-Nasr, M. and Shaw, C.D. (2012) 'Analysis of telemetry data from a real-time strategy game: a case study', Computers in Entertainment (CIE), Vol. 10, No. 3, p.2.
21. Gee, J.P. (2005a) 'Learning by design: good video games as learning machines', E-learning and Digital Media, Vol. 2, No. 1, pp.5-16.
22. Gee, J.P. (2005b) Why Video Games are Good for your Soul: Pleasure and Learning, Common Ground Publishing, Australia.
23. Gee, J.P. (2007) What Video Games have to Teach us about Learning and Literacy, Palgrave Macmillan, New York, NY.
24. Gee, J.P. (2008) 'Learning and games', in Salen, K. (Ed.): The Ecology of Games: Connecting Youth, Games, and Learning, pp.21-40, MIT Press, Cambridge, MA.
25. Gee, J.P. (2012) 'Games can drive assessment to a new place' [online] http://gamesandimpact.org/wp-content/uploads/2012/09/Games-Can-Drive-Assessment-to-a-New-Place.pdf (accessed 3 April 2013).
26. Halverson, E. and Gibbons, D. (2010) 'Key moments' as pedagogical windows into the video production process', Journal of Computing in Teacher Education, Vol. 26, No. 2, pp.69-74.
27. Hickey, D.T., Ingram-Goble, A.A. and Jameson, E.M. (2009) 'Designing assessments and assessing designs in virtual educational environments', Journal of Science Education and Technology, Vol. 18, No. 2, pp.187-208.
28. Huang, C., Tsai, P., Hsu, C. and Pan, R. (2006) 'Exploring cognitive difference in instructional outcomes using text mining technology', in IEEE Conference on System, Man and Cybernertics, Taipei, Taiwan, pp.2116-2120.
29. Ibrahim, Z. and Rusli, D. (2007) 'Predicting students' academic performance: comparing artificial neural network, decision tree and linear regression', in Annual SAS Malaysia Forum, Kuala Lumpur, pp.1-6.
30. Jenkins, H. (2006) Fans, Bloggers, and Gamers: Exploring Participatory Culture, NYU Press, New York, NY.
31. Kafai, Y., Fields, D. and Giang, M. (2009) 'Transgressive gender play: profiles and portraits of girl players in a tween virtual world', in Proceedings of DiGRA 2009 Conference: Breaking New Ground, Brunel University, London.
32. Kerr, D. and Chung, G. (2012) 'Identifying key features of student performance in educational video games and simulations through cluster analysis', Journal of Educational Data Mining, Vol. 4, No. 1, pp.144-182.
33. Kim, Y.J., Ventura, M., Shute, V. and Almond, R. (2012) 'Newton's playground: how to use evidence centered design (ECD) to develop game-based assessment', in Ochsner, A. (Ed.): Proceedings of the 8th Annual Games+Learning+Society Conference, ETC Press, Pittsburg, PA.
34. Koedinger, K.R. and Corbett, A.T. (2006) 'Cognitive tutors: technology bringing learning science to the classroom', in Sawyer, K. (Ed.): The Cambridge Handbook of the Learning Sciences, pp.61-78, Cambridge University Press, New York, NY.
35. Lave, J. and Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge University Press, Cambridge.
36. Lee, M.W., Chen, S.Y., Chrysostomou, K. and Liu, X. (2009) 'Mining student's behavior in web-based learning programs', Expert Systems with Applications, Vol. 36, No. 2, pp.3459-3464.
37. Martin, C., Anton, G., Ochsner, A., Elmgergreen, J. and Steinkuehler, C. (2012) 'Information literacy and online reading comprehension in WoW and school', in Martin, C., Ochsner, A. and Squire, K. (Eds.): Proceedings of the 8th Annual Games+Learning+Society Conference, ETC Press, Pittsburgh, PA.
38. Mavrikis, M. (2010) 'Modeling student interactions in intelligent learning environments: constructing Bayesian networks from data', International Journal on Artificial Intelligence Tools, Vol. 19, No. 6, pp.733-753.
39. Mayrath, M.C., Clarke-Midura, J. and Robinson, D.H. (2012) 'Introduction to technology-based assessments for 21st century skills', in Mayrath, M.C., Clarke-Midura, J., Robinson, D.H. and Schraw, G. (Eds.): Technology-based Assessments for 21st Century Skills: Theoretical and Practical Implications from Modern Research, pp.1-11, Information Age Publishing, Charlotte, NC.
40. Millán, E., Loboda, T. and Pérez-de-la-Cruz, J.L. (2010) 'Bayesian networks for student model engineering', Computers & Education, Vol. 55, No. 4, pp.1663-1683.
41. Mislevy, R.J. (2011) Evidence-Centered Design for Simulation-Based Assessment, CRESST Report 800, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
42. Mislevy, R.J. and Haertel, G.D. (2006) 'Implications of evidence-centered design for educational testing', Educational Measurement: Issues and Practice, Vol. 25, No. 4, pp.6-20.
43. Niwinski, T. and Randall, D.J. (2010) 'Using telemetry to improve zombie killing', Paper presented at Game Developer's Conference, Canada, March.
44. Owen, V. and Halverson, R. (2013) 'ADAGE: assessment data aggregator for game environments', in Ochsner, A. (Ed.): Proceedings of the 9th Annual Games+Learning+Society Conference, ETC Press, Pittsburgh, PA.
45. Owen, V.E., Shapiro, R.B. and Halverson, R. (2013) 'Gameplay as assessment: analyzing event-stream player data and learning using ADAGE (assessment data aggregator for game environments)', Presented at the Tenth International Conference on Computer-Supported Collaborative Learning, Madison, WI, June.
46. Owen, V.E., Steinkuehler, C. and Harris, S. (2012) 'Epistemological beliefs in games vs. school: a mixed-methods approach', in Ochsner, A. (Ed.): Proceedings of the 8th Annual Games+Learning+Society Conference, ETC Press, Pittsburg PA.
47. Rabiner, L.R. (1989) 'A tutorial on hidden Markov models and selected applications in speech recognition', Proceedings of the IEEE, Vol. 77, No. 2, pp.257-286.
48. Romero, C. and Ventura, S. (2010) 'Educational data mining: a review of the state of the art', IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 40, No. 6, pp.601-618.
49. Salen, K. and Zimmerman, E. (2004) Rules of Play: Game Design Fundamentals, MIT Press, Cambridge, MA.
50. Sheard, J., Ceddia, J., Hurst, J. and Tuovinen, J. (2003) 'Inferring student learning behaviour from website interactions: a usage analysis', Journal of Education and Information Technologies, Vol. 8, No. 3, pp.245-266.
51. Shute, V.J. (2011) 'Stealth assessment in computer-based games to support learning', Computer Games and Instruction, Vol. 55, No. 2, pp.503-524.
52. Song, D., Lin, H. and Yang, Z. (2007) 'Opinion mining in e-learning system', in Proceedings of the International Conference on Network and Parallel Computing.
53. Squire, K. (2006) 'From content to context: videogames as designed experience', Educational Researcher, Vol. 35, No. 8, pp.19-29.
54. Squire, K. (2011) Video Games and Learning: Teaching and Participatory Culture in the Digital Age: Technology, Education-Connections (The TEC Series), Teachers College Press, New York, NY.
55. Squire, K. and Patterson, N. (2010) Games and Simulations in Informal Science Education, WCER Working Paper No. 2010-14 [online] http://www.wcer.wisc.edu/publications/workingPapers/Working-Paper-No-2010-14.pdf (accessed 11 March 2013).
56. Srivastava, J. (2008) 'Data mining for social network analysis', in Intelligence and Security Informatics, IEEE, pp.33-34).
57. Steinkuehler, C.A. (2004) 'Learning in massively multiplayer online games', in Kafai, Y.B., Sandoval, W.A., Enyedy, N., Nixon, A.S. and Herrera, F. (Eds.): Proceedings of the 6th International Conference on Learning Sciences, pp.521-528, Erlbaum, Mahwah, NJ.
58. Steinkuehler, C.A. (2006) 'Massively multiplayer online video gaming as participation in a discourse', Mind, Culture, and Activity, Vol. 13, No. 1, pp.38-52.
59. Steinkuehler, C.A. and Duncan, S. (2008) 'Scientific habits of mind in virtual worlds', Journal of Science Education and Technology, Vol. 17, No. 6, pp.530-543.
60. US Department of Education (2012) Enhancing Teaching and Learning through educational data mining and Learning Analytics, US Government Printing Office, Washington, DC.
61. Xu, B. and Recker, M. (2012) 'Teaching analytics: a clustering and triangulation study of digital library user data', Educational Technology & Society, Vol. 15, No. 3, pp.103-115.
62. Yu, C.H., Jannasch-Pennell, A., Digangi, S. and Wasson, B. (1999) 'Using on-line interactive statistics for evaluating web-based instruction', Journal of Educational Media International, Vol. 35, pp.157-161.
63. Yu, C.H., Digangi, S., Jannasch-Pennell, A.K. and Kaprolet, C. (2008) 'Profiling students who take online courses using data mining methods', Online Journal of Distance Learning Administration, Vol. 11, No. 2, pp.1-14.
64. Zapata-Rivera, D. and Bauer, M. (2012) 'Exploring the role of games in educational assessment', in Mayrath, M.C., Clarke-Midura, J., Robinson, D.H. and Schraw, G. (Eds.): Technology-based Assessments for 21st Century Skills: Theoretical and Practical Implications from Modern Research, pp.149-172, Information Age Publishing, Charlotte, NC.