Applying Evidence-Centered Design for the Development of Game-Based Assessments in Physics Playground

International Journal of Testing

Volumn 16, Issue 2, 2016, Pages 142-163

  Kim, Yoon Jeon ^a   Almond, Russell G ^b   Shute, Valerie J ^b  

^a WORCESTER POLYTECHNIC INSTITUTE   (United States)

^b FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Bayesian networks; evidence centered design; game based assessment; stealth assessment

Indexed keywords

EID: 84951288802     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2015.1108322     Document Type: Article

References (37)

1. R.G.Almond, (2015). An IRT-based parameterization for conditional probability tables. In J.M.Augusta and R.N.Carvalho, (Eds.), Proceedings of the 2015 Bayesian Modeling Application Workshop at the 2015 Uncertainty in Artificial Intelligence conference. Retrieved from http://pluto.coe.fsu.edu/RNetica/.
2. R.G.Almond, (2010). “I can name that Bayesian network in two matrixes!”. International Journal of Approximate Reasoning, 51(2), 167–178.
3. R.G.Almond,, L.DiBello,, F.Jenkins,, R.J.Mislevy,, D.Senturk,, L.S.Steinberg,, & D.Yan, (2001). Models for conditional probability tables in educational assessment. In Proceedings of Artificial Intelligence and Statistics 2001 (pp. 137–143).
4. R.G.Almond,, Y.J.Kim,, V.J.Shute,, & M.Ventura, (2013). Debugging the evidence chain. In R.G.Almond and O.Mengshoel (Eds.), Proceedings of the 2013 UAI Application Workshops: Big Data meet Complex Models and Models for Spatial, Temporal and Network Data (UAI2013AW), CEUR Conference Proceedings, 16 (pp. 1–10). Retrieved from http://ceur-ws.org/Vol-1024/paper-01.pdf
5. R.G.Almond,, Y.J.Kim,, G.Velasquez,, & V.J.Shute, (2014). How task features impact evidence from assessments embedded in simulations and games. Measurement: Interdisciplinary Research & Perspectives, 12(1–2), 1–33
6. R.G.Almond, & R.J.Mislevy, (1999). Graphical models and computerized adaptive testing. Applied Psychological Measurement, 23, 223–238.
7. R.G.Almond,, R.J.Mislevy,, L.S.Steinberg,, D.Yan,, & D.M.Williamson, (2015). Bayesian networks in educational assessment. New York, NY: Springer.
8. R.G.Almond,, L.S.Steinberg,, & R.J.Mislevy, (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. Journal of Technology, Learning, and Assessment, 1(5). Retrieved from https://ejournals.bc.edu/ojs/index.php/jtla/article/view/1671
9. R.Almond,, D.Yan,, & L.Hemat, (2008). Parameter recovery studies with a diagnostic Bayesian network model. Behaviormetrika, 35, 159–185.
10. E.L.Baker,, & G.C.Delacruz, (2008). A framework for the assessment of learning games. In H.F.O’Neil & R.S.Perez (Eds.), Computer games and team and individual learning (pp. 21–37). Oxford, UK: Elsevier.
11. J.Behrens,, R.Mislevy,, K.DiCerbo,, & R.Levy, (2010). An evidence centered design for learning and assessment in the digital world. In M.C.Mayrath, J.Clarke-Midura, & D.Robinson (Eds.), Technology-based assessments for 21st century skills: theoretical and practical implications from modern research (pp. 13–53). Charlotte, NC: Information Age.
12. E.Catto, (2013). Box2d: A 2d physics engine for games.
13. K.DiCerbo, (2014). Game-based assessment of persistence. Journal of Educational Technology and Society, 17(1), 17–28.
14. M.P.J.Habgood,, & S.E.Ainsworth, (2011). Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games. Journal of the Learning Sciences, 20(2), 169–206.
15. Y.J.Kim, (2014). Search for the optimal balance among learning, psychometric qualities, and enjoyment in game-based assessment. Unpublished doctoral dissertation, Florida State University, Tallahassee, FL.
16. D.Madigan,, K.Mosurski,, & R.G.Almond, (1997). Graphical explanation in belief networks. Journal of Computational Graphics and Statistics, 6, 160–181.
17. R.J.Mislevy,, R.G.Almond,, & J.F.Lukas, (2003). A brief introduction to evidence-centered design (ETS Research Report RR-03-16). Princeton, NJ: Educational Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/RR-03-16.pdf
18. R.J.Mislevy,, J.T.Behrens,, K.E.DiCerbo,, D.C.Frezzo,, & P.West, (2012). Three things game designers need to know about assessment. In D.Ifenthaler, D.Eseryel, & X.Ge (Eds.), Assessment in game-based learning: foundations, innovations, and perspectives (pp. 59–81). New York: Springer.
19. R.J.Mislevy,, L.Hamel,, R.G.Fried,, T.Gaffney,, & G.Haertel, (2003). Design patterns for assessing science inquiry (PADI Technical Report No.1). Menlo Park, CA: SRI International.
20. R.J.Mislevy,, A.Oranje,, M.Bauer,, A.A.von Davier,, J.Hao,, S.Corrigan,, … M.John, (2014). Psychometric considerations in game-based assessment. Redwood City, CA: GlassLab.
21. R.J.Mislevy,, L.S.Steinberg,, & R.G.Almond, (2003). Focus article: on the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3–62.
22. E.Muraki, (1992). A Generalized partial credit model: application of an EM algorithm. Applied Psychological Measurement, 16(2), 159–176.
23. Norsys (2012). Netica Application: A complete software package to solve problems using Bayesian Belief Networks and Influence diagrams. Computer software retrieved 2015-02-27 from http://www.norsys.com/.
24. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. San Mateo, CA: Morgan Kaufmann
25. R Core Team. (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org/
26. F.Samejima, (1997). Graded response model. In W.van der Linden & R.Hambleton (Eds.), Handbook of modern item response theory (pp. 85–100). New York: Springer.
27. V.J.Shute, (2011). Stealth assessment in computer-based games to support learning. In S.Tobias & J.D.Fletcher (Eds.), Computer games and instruction (pp. 503–524). Charlotte, NC: Information Age Publishers.
28. V.J.Shute,, E.G.Hansen,, & R.G.Almond, (2008). You can't fatten a hog by weighing it—or can you? Evaluating an assessment for learning system called ACED. International Journal of Artificial Intelligence in Education, 18(4), 289–316.
29. V.J.Shute,, & M.Ventura, (2013). Measuring and supporting learning in games: Stealth assessment. Cambridge, MA: The MIT Press.
30. V.J.Shute,, M.Ventura,, M.Bauer,, & D.Zapata-Rivera, (2009). Melding the power of serious games and embedded assessment to monitor and foster learning. In U.Ritterfeld, M.J.Cody, & P.Vorderer (Eds.), The social science of serious games: theories and applications (pp. 295–321). Philadelphia: Routledge/LEA.
31. V.J.Shute,, M.Ventura,, & Y.J.Kim, (2013). Assessment and learning of qualitative physics in Newton's Playground. The Journal of Educational Research, 106(6), 423–430.
32. Slota, S. T. (2014). Project TECHNOLOGIA: A game-based approach to understanding situated intentionality (Doctoral dissertation). University of Connecticut, Storrs, CT.
33. D.J.Spiegelhalter,, & S.L.Lauritzen, (1990). Sequential updating of conditional probabilities on directed graphical structures. Networks, 20, 579–605.
34. D.L.Sundre,, & S.L.Wise, (2003). ‘Motivation filtering’: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the Annual meeting of the National Council on Measurement in Education, Chicago, IL.
35. K.K.Tatsuoka, (1983). Rule space: an approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20(4), 345–354.
36. H.Wainer,, J.L.Veva,, F.Camacho,, B.B.ReeveIII,, K.Rosa,, L.Nelson,, K.A.Swygert,, & D.Thissen, (2001). Augmented scores—“borrowing strength” to compute scores based on a small number of items. In D.Thissen & H.Wainer (Eds.), Test scoring (pp. 343–388). Hillsdale, NJ: Lawrence Erlbaum Associates.
37. D.Zapata-Rivera,, & M.Bauer, (2012). Exploring the role of games in educational assessment. In M.Mayrath, J.Clarke-Midura, D.Robinson, & G.Shraw (Eds.), Technology-based assessments for twenty-first-century skills: theoretical and practical implications from modern research (pp. 147–169). Charlotte, NC: Information Age Publishing.