Three things game designers need to know about assessment

Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives

Volumn , Issue , 2012, Pages 59-81

  Mislevy, Robert J ^a   Behrens, John T ^b   Dicerbo, Kristen E ^b   Frezzo, Dennis C ^c   West, Patti ^d  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b PEARSON   (United States)

^c CISCO SYSTEMS   (United States)

^d Cisco Networking Academy   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84894813510     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-3546-4_5     Document Type: Chapter

References (69)

1. Alexander, C., Ishikawa, S., & Silverstein, M. (1977). A pattern language: Towns, buildings, construction. New York: Oxford University Press.
2. Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. Journal of Technology, Learning, andAssessment, 1(5). Retrieved May 1, 2011, from http://www.bc.edu/research/intasc/jtla/journal/v1n5.shtml.
3. Bagley, E., & Shaffer, D. W. (2009). When people get in the way: Promoting civic thinking through epistemic gameplay. International Journal of Gaming and Computer-mediated Simulations, 1, 36-52.
4. Barab S. A., Dodge T., & Gee, J. P. (in press). The worked example: Invitational scholarship in service of an emerging field. Educational Researcher.
5. Behrens, J. T., Frezzo, D. C., Mislevy, R. J., Kroopnick, M., & Wise, D. (2007). Structural, functional, and semiotic symmetries in simulation-based games and assessments. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 59-80). New York: Erlbaum.
6. Behrens, J. T., Mislevy, R. J., Bauer, M., Williamson, D. M., & Levy, R. (2004). Introduction to evidence centered design and lessons learned from its application in a global e-learning program. International Journal of Testing, 4, 295-301.
7. Behrens, J. T., Mislevy, R. J., DiCerbo, K. E., & Levy, R. (2012). An evidence centered design for learning and assessment in the digital world. In M. C. Mayrath, J. Clarke-Midura, & D. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 13-53). Charlotte, NC: Information Age.
8. Bejar, I.I, & Braun, H. (1999). Architectural simulations: From research to implementation. Final report to the National Council of Architectural Registration Boards (ETS RM-99-2). Princeton, NJ: Educational Testing Service.
9. Bennett, R. E., & Bejar, I. I. (1998). Validity and automated scoring: It’s not only the scoring. Educational Measurement: Issues and Practice, 17(4), 9-17.
10. Cheng, B. H., Ructtinger, L., Fujii, R., & Mislevy, R. (2010). Assessing systems thinking and complexity in science (Large-Scale Assessment Technical Report 7). Menlo Park, CA: SRI International.
11. Chi, M. T. H., Glaser, R., & Farr, M. J. (Eds.). (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.
12. Chung, G. K. W. K., Baker, E. L., Delacruz, G. C., Bewley, W. L., Elmore, J., & Seely, B. (2008). A computational approach to authoring problem-solving assessments. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 289-307). Mahwah, NJ: Erlbaum.
13. Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research on Technology in Education, 42, 309-328.
14. Claxton, G. (2002). Education for the learning age: A sociocultural approach to learning to learn. In G. Wells & G. Claxton (Eds.), Learning for life in the 21st century (pp. 19-33). Oxford, UK: Blackwell.
15. Conejo, R., Guzman, E., Millan, E., Trella, M., PerezDeLaCruz, J. L., & Rios, A. (2004). A web-based tool for adaptive testing. International Journal of Artificial Intelligence in Education, 14, 29-61.
16. Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass.
17. Embretson, S. E. (Ed.). (1985). Test design: Developments in psychology and psychometrics. Orlando: Academic.
18. Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 380-396.
19. Ericsson, A. K., Charness, N., Feltovich, P., & Hoffman, R. R. (2006). Cambridge handbook on expertise and expert performance. Cambridge, UK: Cambridge University Press.
20. Fletcher, J. D., & Morrison, J. E. (2007). Representing cognition in games and simulations. In E. Baker, J. Dickieson, W. Wulfeck, & H. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 107-137). New York: Lawrence Erlbaum.
21. Frezzo, D. C. (2009). Using activity theory to understand the role of a simulation-based interactive learning environment in a computer networking course. Doctoral dissertation, ProQuest. Retrieved August 29, 2011, from http://gradworks.umi.com/33/74/3374268.html.
22. Frezzo, D. C., Behrens, J. T., & Mislevy, R. J. (2009). Design patterns for learning and assessment: Facilitating the introduction of a complex simulation-based learning environment into a community of instructors. The Journal of Science Education and Technology. Retrieved April 10, 2012, from Springer Open Access http://www.springerlink.com/content/566p6g4307405346/.
23. Fullerton, T., Swain, C., & Hoffman, S. S. (2008). Game design workshop: Designing, prototyping, and playtesting games (2nd ed.). Burlington, MA: Morgan Kaufmann.
24. Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design patterns. Reading, MA: Addison-Wesley.
25. Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave/Macmillan.
26. Greeno, J. G. (1998). The situativity of knowing, learning, and research. American Psychologist, 53, 5-26.
27. Katz, I. R. (1994). Coping with the complexity of design: Avoiding conflicts and prioritizing constraints. In A. Ram, N. Nersessian, & M. Recker (Eds.), Proceedings of the sixteenth annual meeting of the Cognitive Science Society (pp. 485-489). Mahwah, NJ: Erlbaum.
28. Koster, R. (2005). A theory offunfor game design. Scottsdale, AZ: Paraglyph.
29. Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.
30. Leighton, J., & Gierl, M. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. New York, NY: Cambridge University Press.
31. Levy, R., Behrens, J. T., & Mislevy, R. J. (2006). Variations in adaptive testing and their online leverage points. In D. D. Williams, S. L. Howell, & M. Hricko (Eds.), Online assessment, measurement, and evaluation (pp. 180-202). Hershey, PA: Information Science Publishing.
32. Loftus, E. F., & Loftus, G. R. (1983). Mind at play: The psychology of video games. New York: Basic Books.
33. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Mahwah, NJ: Erlbaum.
34. Luecht, R. M. (2006). Assessment engineering: An emerging discipline. Paper presented in the Centre for Research in Applied Measurement and Evaluation, University of Alberta, Edmonton.
35. Malone, T. W. (1981). What makes computer games fun? Byte, 6, 258-277.
36. Margolis, M. J., & Clauser, B. E. (2006). A regression-based procedure for automated scoring of a complex medical performance assessment. In D. M. Williamson, R. J. Mislevy, & I. I. Bejar (Eds.), Automated scoring for complex tasks in computer-based testing (pp. 123-167). Hillsdale, NJ: Lawrence Erlbaum.
37. Mayer, R. E. (1981). Frequency norms and structural analysis of algebra story problems into families, categories, and templates. International Science, 10, 135-175.
38. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
39. Mislevy, R. J. (2004). Can there be reliability without “reliability”? Journal of Educational and Behavioral Statistics, 29, 241-244.
40. Mislevy, R. J., & Riconscente, M. M. (2006). Evidence-centered assessment design: Layers, concepts, and terminology. In S. Downing & T. Haladyna (Eds.), Handbook of test development (pp. 61-90). Mahwah, NJ: Erlbaum.
41. Mislevy, R. J., Riconscente, M. M., & Rutstein, D. W. (2009). Design patterns for assessing model-based reasoning (Large-Scale Assessment Technical Report 6). Menlo Park, CA: SRI International.
42. Mislevy, R. J., Steinberg, L. S., & Almond, R. A. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-67.
43. Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Johnson, L., & Almond, R. A. (2002). Making sense of data from complex assessments. Applied Measurement in Education, 15, 363-378.
44. Moss, P. (1994). Can there be validity without reliability? Educational Researcher, 23(2), 5-12.
45. Nelson, B. C., Erlandson, B., & Denham, A. (2011). Global channels of evidence for learning and assessment in complex game environments. British Journal of Educational Technology, 42, 88-100.
46. Pausch, R., Gold, R., Skelly, T., & Thiel, D. (1994). What HCI designers can learn from video game designers. In Conference on human factors in computer systems (pp. 177-178). Boston, MA: ACM.
47. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks ofplausible inference. San Mateo, CA: Kaufmann.
48. Quellmalz, E., & Pellegrino, J. W. (2009). Technology and testing. Science, 323, 75-79.
49. Rollings, A., & Morris, D. (2000). Game architecture and design. Scottsdale, AZ: Coriolis.
50. Roschelle, J. (1996). Designing for cognitive communication: Epistemic fidelity or mediating collaborative inquiry? In D. L. Day & D. K. Kovacs (Eds.), Computers communication and mental models (pp. 13-25). Bristol, PA: Taylor and Francis.
51. Rupp, A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence-centered design of epistemic games: Measurement principles for complex learning environments. Journal of Technology, Learning, and Assessment, 5(4). Retrieved April 10, 2012, from http://ejournals.bc.edu/ojs/index.php/jtla/article/download/1623/1467.
52. Rupp, A., Templin, J., & Henson, R. (2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford.
53. Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge: MIT.
54. Salthouse, T. A. (1991). Expertise as the circumvention of human processing limitations. In K. A. Ericcson & J. Smith (Eds.), Toward a general theory of expertise (pp. 286-300). Cambridge, UK: Cambridge University Press.
55. Scalise, K., & Gifford, B. (2006). Computer-based assessment in E-learning: A framework for constructing “Intermediate Constraint” questions and tasks for technology platforms. Journal of Technology, Learning, and Assessment, 4(6). Retrieved July 17, 2009, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1653/1495.
56. Schmit, M. J., & Ryan, A. (1992). Test-taking dispositions: A missing link? Journal of Applied Psychology, 77, 629-637.
57. Schwartz, D. L., & Arena, D. (2009). Choice-based assessments for the digital age. Palo Alto: Stanford University.
58. Shaffer, D. W. (2006). How computer games help children learn. New York: Palgrave/Macmillan.
59. Shaffer, D. W., & Gee, J. P. (2012). The Right Kind of GATE: Computer games and the future of assessment. In M. Mayrath, J. Clarke-Midura, & D. H. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 211-228). Charlotte: Information Age Publishing.
60. Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., et al. (2009). Epistemic network analysis: A prototype for 21st century assessment of learning. The International Journal of Learning and Media, 1, 33-53.
61. Shao, J., & Tu, D. (1995). The jackknife and bootstrap. New York: Springer.
62. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503-524). Charlotte, NC: Information Age Publishers.
63. Shute, V. J., & Torres, R. (2012). Where streams converge: Using evidence-centered design to assess Quest to Learn. In M. Mayrath, J. Clarke-Midura, & D. H. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 91-204). Charlotte, NC: Information Age Publishing.
64. Sundre, D. L., & Wise, S. L. (2003). ‘Motivation filtering’: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago.
65. Vendlinski, T. P., Baker, E. L., & Niemi, D. (2008). Templates and objects in authoring problem solving assessments. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O’Neil (Eds.), Assessment of problem solving using simulations (pp. 309-333). New York: Erlbaum.
66. Vygotsky, L. S. (1978). Mind and society: The development ofhigherpsychologicalprocesses. Cambridge, MA: Harvard University Press.
67. Wainer, H., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., et al. (2000). Computerized adaptive testing: A primer (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
68. Wertsch, J. (1998). Mind as action. New York: Oxford University Press.
69. Williamson, D. M., Bauer, M., Steinberg, L. S., Mislevy, R. J., Behrens, J. T., & DeMark, S. (2004). Design rationale for a complex performance assessment. International Journal of Measurement, 4, 303-332.