Assessment in and of serious games: An overview

Advances in Human-Computer Interaction

Volumn 2013, Issue , 2013, Pages

  Bellotti, Francesco ^a   Kapralos, Bill ^b   Lee, Kiju ^c   Moreno Ger, Pablo ^d   Berta, Riccardo ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b UNIVERSITY OF ONTARIO INSTITUTE OF TECHNOLOGY   (Canada)

^c Case Western Reserve University   (United States)

^d UNIVERSIDAD COMPLUTENSE DE MADRID   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84875498804     PISSN: 16875893     EISSN: 16875907     Source Type: Journal    
DOI: 10.1155/2013/136864     Document Type: Review

References (90)

1. Gee J. P., What Video Games Have to Teach Us about Learning and Literacy 2007 New York, NY, USA Palgrave MacMillan
2. Greitzer F. L., Kuchar O. A., Huston K., Cognitive science implications for enhancing training effectiveness in a serious gaming context. ACM Journal on Educational Resources in Computing 2007 7 3, article 2 2-s2.0-36448952741 10.1145/1281320.1281322 (Pubitemid 350169468)
3. De Grove F., Mechant P., Van Looy J., Uncharted waters? Exploring experts' opinions on the opportunities and limitations of serious games for foreign language learning. Proceedings of the 3rd International Conference on Fun and Games September 2010 Leuven, Belgium 107 115 2-s2.0-78349288141 10.1145/1823818.1823830
4. van Eck R., Digital game-based learning: it's not just the digital natives who are restless. EDUCAUSE Review 2006 41 2 16 30
5. Cannon-Bowers J., The state of gaming and simulation. Proceedings of the Training Conference and Expo March 2006 Orlando, Fla, USA
6. Shute V., Ventura M., Bauer M., Zapata-Rivera D., Ritterfeld U., Cody M., Vorderer P., Melding the power of serious games and embedded assessment to monitor and foster learning: flow and grow. Serious Games: Mechanisms and Effects 2009 Mahwah, NJ, USA Routledge, Taylor and Francis 295 321
7. Gosen J., Washbush J., A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming 2004 35 2 270 293 2-s2.0-33749576028 10.1177/1046878104263544
8. Hays R. T., The effectiveness of instructional games: a literature review and discussion. 2005 2005-004 Naval Air Warfare Center, Training Systems Division
9. Kulik A. A., School mathematics and science programs benefit from instructional technology. United States National Science Foundation (NSF), National Center for Science and Engineering Statistics (NCSES), InfroBrief NSF-03-301, November 2002, http://www.nsf.gov/statistics/infbrief/nsf03301/
10. Blunt R., Do serious games work? Results from three studies. eLearn Magazine 2009 2009 12 10.1145/1661377.1661378
11. Bergeron B., Developing Serious Games 2006 Hingham, Mass, USA Thomson Delmar Learning
12. Prensky M., Don't Bother Me Mom-I'm Learning! 2006 Paragon House
13. Livingston S., Fennessey G., Coleman J., Edwards K., Kidder S., The Hopkins games program: final report on seven years of research. Report No. 155 1973 Baltimore, Md, USA Johns Hopkins University, Center for Social Organization of Schools
14. Chin J., Dukes R., Gamson W., Assessment in simulation and gaming: a review of the last 40 years. Simulation & Gaming 2009 40 4 553 568 2-s2.0-69749124117 10.1177/1046878109332955
15. Connolly T. M., Boyle E. A., MacArthur E., Hainey T., Boyle J. M., A systematic literature review of the empirical evidence on computer games and serious games. Computers and Education 2012 59 2 661 686
16. Kato P. M., Cole S. W., Bradlyn A. S., Pollock B. H., A video game improves behavioral outcomes in adolescents and young adults with cancer: a randomized trial. Pediatrics 2008 122 2 e305 e317 2-s2.0-49849084908 10.1542/peds.2007-3134
17. Cole S. W., Yoo D. J., Knutson B., Interactivity and reward-related neural activation during a serious videogame. PLoS ONE 2012 7 3 e33909 10.1371/journal.pone.0033909
18. Dandeneau S. D., Baldwin M. W., The inhibition of socially rejecting information among people with high versus low self-esteem: the role of attentional bias and the effects of bias reduction training. Journal of Social and Clinical Psychology 2004 23 4 584 602 2-s2.0-4444279484 10.1521/jscp.23.4. 584.40306 (Pubitemid 39203274)
19. Bellotti F., Berta R., De Gloria A., D'Ursi A., Fiore V., A serious game model for cultural heritage. Journal on Computing and Cultural Heritage 2012 5 4 1 27
20. Bellotti F., Berta R., De Gloria A., Designing effective serious games: opportunities and challenges for research. International Journal of Emerging Technologies in Learning 2010 5 22 35
21. Bente G., Breuer J., Ritterfield U., Cody M. J., Vorderer P., Making the implicit explicit: embedded measurement in serious games. Serious Games: Mechanisms and Effects 2009 New York, NY, USA Routledge 322 343
22. Michael D., Chen S., Proof of learning: assessment in serious games. October 2005, http://www.gamasutra.com/view/feature/2433/proof-of-learning- assessment-in-.php
23. Enfield J., Myers R. D., Lara M., Frick T. W., Innovation diffusion: assessment of strategies within the diffusion simulation game. Simulation & Gaming 2012 43 2 188 214 1177/1046878111408024
24. Boston C., The concept of formative assessment. Practical Assessment, Research & Evaluation 2002 8 9
25. Moreno-Ger P., Burgos D., Torrente J., Digital games in eLearning environments: current uses and emerging trends. Simulation & Gaming 2009 40 5 669 687
26. Sebastian C., Anantachai A., Byun J. H., Lenox J., Assessing what players learned in serious games: in-situ data collection, information trails, and quantitative analysis. Proceedings of the 10th International Conference on Computer Games: AI, Animation, Mobile, Educational and Serious Games 2007 10 19
27. Becker K., Parker J. R., The Guide to Computer Simulations and Games 2011 Indianapolis, Ind, USA John Wiley & Sons
28. Dugard P., Todman J., Analysis of pre-test-post-test control group designs in educational research. Educational Psychology 1995 15 2 181 198
29. National Center for Technology Innovation (NCTI), Experimental Study Design. 2012, http://www.nationaltechcenter.org/index.php/products/at-research- matters/experimental-study-design/
30. Allen L., Seeney M., Boyle L., Hancock F., The implementation of team based assessment in serious games. Proceedings of the 1st Conference in Games and Virtual Worlds for Serious Applications (VS-GAMES '09) March 2009 Coventry, UK 28 35 2-s2.0-70350075493 10.1109/VS-GAMES.2009.12
31. Corti K., Game-Based Learning: A Serious Business Application 2006 Coventry, UK PIXELearning
32. Brockmyer J. H., Fox C. M., Curtiss K. A., McBroom E., Burkhart K. M., Pidruzny J. N., The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. Journal of Experimental Social Psychology 2009 45 4 624 634 2-s2.0-67651015209 10.1016/j.jesp.2009.02.016
33. IJsselsteijn W. A., van de Hoogen W., Klimmt C., de kort Y., Lindley C., Mathiak K., Poels K., Ravaja N., Turpeinen M., Vorderer P., Measuring the experience of digital game enjoyment. Proceedings of Measuring Behavior August 2008 Maastricht, The Netherlands 88 89
34. Csikszentmihalyi M., Flow: The Psychology of Optimal Experience 1990 New York, NY, USA Harper & Row
35. Cowley B., Charles D., Black M., Hickey R., Toward an understanding of flow in video games. Computers in Entertainment 2008 6 2 1 28 2-s2.0-49449083303 10.1145/1371216.1371223
36. Sweetser P., Wyeth P., GameFlow: a model for evaluating player enjoyment in games. ACM Computers in Entertainment 2005 3 3 1 24
37. Fu F. L., Su R. C., Yu S. C., EGameFlow: a scale to measure learners' enjoyment of e-learning games. Computers and Education 2009 52 1 101 112 2-s2.0-56249103760 10.1016/j.compedu.2008.07.004
38. Janicke S. H., Ellis A., Psychological and physiological differences between the 3D and 2D gaming experience. Proceedings of the 3D Entertainment Summit September, 2011 Hollywood, Calif, USA
39. Jelinek H. F., August K., Imam H., Khandoker A. H., Koenig A., Riener R., Heart rate asymmetry and emotional response to robot assist task challenges in stroke patients. Proceedings of the Computing in Cardiology Conference September 2011 Hangzhou, China
40. Kivikangas J. M., Chanel G., Cowley B., Ekman I., Salminen M., Järvelä S., Ravaja N., A review of the use of psychophysiological methods in game research. Journal of Gaming & Virtual Worlds 2011 3 3 181 199
41. Nacke L. E., Affective ludology: scientific measurement of user experience in interactive entertainment [Ph.D. thesis] 2009 Karlskrona, Sweden Blekinge Institute of Technology
42. Plotnikov A., Stakheika N., De Gloria A., Schatten C., Bellotti F., Berta R., Fiorini C., Ansovini F., Exploiting real-time EEG analysis for assessing flow in games. Workshop: "Game Based Learning for 21st Century Transferable Skills" at iCalt 2012 June 2012 Rome, Italy
43. Nacke L. E., Physiological game interaction and psychophysiological evaluation in research and industry. Gamasutra Article, June 2011, http://www.gamasutra.com/blogs/LennartNacke/20110628/7867/Physiological-Game- Interaction-and-Psychophysiological-Evaluation-in-Research-and-Industry.php
44. Salminen M., Ravaja N., Oscillatory brain responses evoked by video game events: the case of super monkey ball 2. Cyberpsychology & Behavior 2007 10 3 330 338 2-s2.0-34347237292 10.1089/cpb.2006.9947 (Pubitemid 47001173)
45. Loh C., Gibson D., Aldrich C., Prensky M., Designing online games assessment as information trails. Games and Simulations in Online Learning: Research and Development Frameworks 2007 Hershey, Pa, USA Information Science Publishing 323 348
46. Yannakakis G. N., Hallam J., Evolving opponents for interesting interactive computer games. Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design and Education 2004
47. Malone T. W., Toward a theory of intrinsically motivating instruction. Cognitive Science 1981 5 4 333 369 2-s2.0-49049148395
48. Iida H., Takeshita N., Yoshimura J., Nakatsu R., Hoshino J., A metric for entertainment of boardgames: its implication for evolution of chess variants. Proceeding of: Entertainment Computing: Technologies and Applications, IFIP First International Workshop on Entertainment Computing (IWEC '02) 2003 Boston, Mass, USA Kluwer Academic 65 72
49. Bellotti F., Berta R., De Gloria A., Primavera L., Enhancing the educational value of video games. Computers in Entertainment 2009 7 2 23 41 2-s2.0-68049105384 10.1145/1541895.1541903
50. Hassenzahl M., Wessler R., Capturing design space from a user perspective: the repertory grid technique revisited. International Journal of Human-Computer Interaction 2000 12 3-4 441 459 2-s2.0-23044524335
51. Zoanetti N., Software for online testing and quizzes. 2011, http://www.assessmentfocus.com/online-testing.php
52. Flynn A., Concannon F., Campbell M., An evaluation of undergraduate students' online assessment performances. Advanced Technology for Learning 2006 3 1 15 51
53. Hewson C., Can online course-based assessment methods be fair and equitable? Relationships between students' preferences and performance within online and offline assessments. Journal of Computer Assisted Learning 2001 28 5 488 498 10.1111/j.1365-2729.2011.00473.x
54. Guzmán E., Conejo R., Pérez-de-la-Cruz J. L., Improving student performance using self-assessment tests. IEEE Intelligent Systems 2007 22 4 46 52 2-s2.0-34547981295 10.1109/MIS.2007.71 (Pubitemid 47274705)
55. Hattie J., Masters D., asTTle-Assessment Tools for Teaching and Learning. HEFCE JISC, 2006, http://www.jisc.ac.uk/media/documents/projects/asttle- casestudy.pdf
56. Hattie J., Brown G., Keegan P., Irving S., MacKay A., Sutherland T., Mooyman D., Patel P., Validation evidence of asTTle reading assessment results: norms and criteria. Asttle Tech. Rep. November 2003 22 University of Auckland/Ministry of Education
57. Hattie J., Large-scale assessment of student competencies. Symposium: Working in Today's World of Testing and Measurement: Required Knowledge and Skills (Joint ITC/CPTA Symposium); the 26th International Congress of Applied Psychology July 2006 Athens, Greece
58. Questionmark Corporation, Questionmark Perception Measure Knowledge, Skills and Attitudes Securely for Certification, Regulatory Compliance and successful Learning Outcomes. 2012
59. Bull J., Stephens D., The use of question mark software for formative and summative assessment in two universities. Innovations in Education and Teaching International 1999 36 2 128 135 2-s2.0-0033412369
60. Velan G. M., Kumar R. K., Dziegielewski M., Wakefield D., Web-based self-assessments in pathology with Questionmark Perception. Pathology 2002 34 3 282 284 2-s2.0-0035985940 10.1080/00313020220131372 (Pubitemid 34628192)
61. HEFCE JISC, Case study 5: making the most of a computer-assisted assessment system University of Manchester. 2010, http://www.jisc.ac.uk/media/ documents/programmes/elearning/digiassess-makingthemost.pdf
62. Wood M., Human Computer Collaborative Assessment-Access by Computer (ABC)-University of Manchester 2009 HEFCE JISC
63. HEFCE JISC, Short answer marking engines. 2009, http://www.jisc.ac.uk/ media/documents/projects/shorttext.pdf
64. Jordan S., Mitchell T., e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology 2009 40 2 371 385 2-s2.0-60349104573 10.1111/j.1467-8535. 2008.00928.x
65. Noorbehbahani F., Kardan A. A., The automatic assessment of free text answers using a modified BLEU algorithm. Computers & Education 2011 56 2 337 345 2-s2.0-78149468676 10.1016/j.compedu.2010.07.013
66. Beatty I. D., Gerace W. J., Technology-enhanced formative assessment: a research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology 2009 18 2 146 162 2-s2.0-64549150952 10.1007/s10956-008-9140-4
67. Fies C., Marshall J., Classroom response systems: a review of the literature. Journal of Science Education and Technology 2006 15 1 101 109 2-s2.0-33751025689 10.1007/s10956-006-0360-1 (Pubitemid 44757013)
68. 3 framework: evaluating classroom response system interactions in university classrooms. Journal of Science Education and Technology 2008 17 5 483 499 2-s2.0-55149115667 10.1007/s10956-008-9116-4
69. IMS Global Learning Consortium, IMS Question & Test Interoperability Specification (QTI). 2012, http://www.imsglobal.org/question/
70. Smythe C., Roberts P., An Overview of the IMS Question & Test Interoperability Specification 2000 Leicestershire, UK Computer Aided Assessment
71. Codility Ltd., Codility: WE TEST CODERS. 2009, http://codility.com/
72. About IKM, Overview. 2011, http://www.ikmnet.com/about/overview.cfm
73. Clarke M. M., Madaus G. F., Horn C. L., Ramos M. A., Retrospective on educational testing and assessment in the 20th century. Journal of Curriculum Studies 2000 32 2 159 181 2-s2.0-33749086731
74. Resnick L. B., Resnick D. P., Assessing the Thinking Curriculum: New Tools for Educational Reform 1989 Pittsburgh, Pa, USA Learning Research and Development Center: University of Pittsburgh and Carnegie Mellon University
75. Lipman M., Baron J. B., Sternberg R. J., Some thoughts on the formation of reflective education. Teaching-Thinking Skills: Theory and Practice 1987 New York, NY, USA W. H. Freeman 151 161
76. Tribune C., Standardized testing will limit students' future. April 2010, http://articles.chicagotribune.com/2010-04-21/news/chi-100421shafer-briefs-1- standardized-test-scores-teacher-and-principal-evaluations
77. Fairtest, What's Wrong with Standardized Tests? May 2012, http://www.fairtest.org/facts/whatwron.htm
78. Short E. J., Noeder M., Gorovoy S., Manos M. J., Lewis B., Russ S., Niec L., The importance of play in both the assessment and treatment of young children. An Evidence-Based Approach to Play in Intervention and Prevention: Integrating Developmental and Clinical Science London, UK Guilford
79. Kaugars A. S., Russ S. W., Assessing preschool children's pretend play: preliminary validation of the affect in play scale-preschool version. Early Education and Development 2009 20 5 733 755 2-s2.0-70349939624 10.1080/10409280802545388
80. Bellotti F., Berta R., De Gloria A., Primavera L., Adaptive experience engine for serious games. IEEE Transactions on Computational Intelligence and AI in Games 2009 1 4 264 280 2-s2.0-77955666872 10.1109/TCIAIG.2009.2035923
81. Tan C. H., Tan K. C., Tay A., Dynamic game difficulty scaling using adaptive behavior-based AI. IEEE Transactions on Computational Intelligence and AI in Games 2011 3 4 289 301 10.1109/TCIAIG.2011.2158434
82. Doucet L., Srinivasany V., Designing entertaining educational games using procedural rhetoric: a case study. Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games July 2010 Los Angeles, Calif, USA 5 10 2-s2.0-77956270453 10.1145/1836135.1836136
83. Froschauer J., Seidel I., Gärtner M., Berger H., Merkl D., Design and evaluation of a serious game for immersive cultural training. Proceedings of the 16th International Conference on Virtual Systems and Multimedia (VSMM '10) October 2010 Seoul, Republic of Korea IEEE CS Press 253 260 2-s2.0-78651458103 10.1109/VSMM.2010.5665978
84. Kelly H., Howell K., Glinert E., Holding L., Swain C., Burrowbridge A., Roper M., How to build serious games. Communications of the ACM 2007 50 7 44 49 2-s2.0-34547361803 10.1145/1272516.1272538 (Pubitemid 47152821)
85. Swarz J., Ousley A., Magro A., Rienzo M., Burns D., Lindsey A. M., Wilburn B., Bolcar S., CancerSpace: a simulation-based game for improving cancer-screening rates. IEEE Computer Graphics and Applications 2010 30 1 90 94 2-s2.0-75149160070 10.1109/MCG.2010.4
86. Zielke M. A., Evans M. J., Dufour F., Christopher T. V., Donahue J. K., Johnson P., Jennings E. B., Friedman B. S., Ounekeo P. L., Flores R., Serious games for immersive cultural training: creating a living world. IEEE Computer Graphics and Applications 2009 29 2 49 60 2-s2.0-63149086704 10.1109/MCG.2009.30
87. King M., Newman R., Evaluating business simulation software: approach, tools and pedagogy. On the Horizon 2009 17 4 368 377 2-s2.0-70449095576 10.1108/10748120910998399
88. Stainton A. J., Johnson J. E., Borodzicz E. P., Educational validity of business gaming simulation: a research methodology framework. Simulation & Gaming 2010 41 5 705 723 2-s2.0-77956952381 10.1177/1046878109353467
89. Ritterfeld U., Cody M., Vorderer P., Serious Games: Mechanisms and Effects 2009 New York, NY, USA Routledge
90. Bellotti F., Berta R., De Gloria A., Primavera L., A task annotation model for SandBox Serious Games. Proceedings of IEEE Symposium on Computational Intelligence and Games (CIG '09) September 2009 Milano, Italy 233 240 2-s2.0-71549116088 10.1109/CIG.2009.5286471