Psychometrics and game-based assessment

Technology and Testing: Improving Educational and Psychological Measurement

Volumn , Issue , 2015, Pages 23-48

  Mislevy, Robert J ^a   Corrigan, Seth ^f   Oranje, Andreas ^b   DiCerbo, Kristen ^c   Bauer, Malcolm I ^b   Von Davier, Alina ^d,e   John, Michael ^f  

^a UNIVERSITY OF MARYLAND   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

^c Center for Learning Science and Technology   (United States)

^d FORDHAM UNIVERSITY   (United States)

^e ETS   (United States)

^f GlassLab Games ^*

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84949444740     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9781315871493     Document Type: Chapter

References (66)

1. Achieve, Inc. (2013). Next generation science standards. Washington, DC: Achieve, Inc.
2. Adams, R. J., Wilson, M., & Wang, W. C. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1-23.
3. Almond, R. G., & Mislevy, R. J. (1999). Graphical models and computerized adaptive testing. Applied Psychological Measurement, 23, 223-237.
4. Almond, R. G., Steinberg, L. S., & Mislevy, R. J. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. Journal of Technology, Learning, and Assessment, 1(5). Downloaded May 8, 2015, from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.198.5129&rep=rep1&type=pdf
5. Avouris, N., Dimitracopoulou, A., & Komis, V. (2003). On analysis of collaborative problem solving: An object-oriented approach. Computers in Human Behavior, 19, 147-167.
6. Bagley, E., & Shaffer, D. W. (2009). When people get in the way: Promoting civic thinking through epistemic gameplay. International Journal of Gaming and Computer-Mediated Simulations, 1, 36-52.
7. Baker, R.S.J.D. Corbett, A. T., & Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. Paper 6. Pittsburgh: Human-Computer Interaction Institute, Carnegie-Mellon University. Available online at http://repository.cmu.edu/hcii/6
8. Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253-278.
9. Cronbach, L. J. (1989). Construct validation after thirty years. In R. L. Linn (Ed.), Intelligence: Measurement, theory, and public policy (pp. 147-171). Urbana: University of Illinois Press.
10. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
11. Csíkszentmihályi, M. (1975). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass.
12. De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
13. DiCerbo, K. E. & Kidwai, K. (2013). Detecting player goals from game log files. Poster presented at the Sixth International Conference on Educational Data Mining, Memphis, TN.
14. Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991). Quality control in the development and use of performance assessments. Applied Measurement in Education, 4, 289-303.
15. Gee, J. P. (2007). What video games have to teach us about learning and literacy (2nd ed.). New York: Palgrave.
16. Gee, J. P. (2008). Learning and games. In K. Salen (Ed.), The ecology of games: Connecting youth, games, and learning (pp. 21-40). Cambridge: MIT Press.
17. Geerlings, H., Glas, C. A., & van der Linden, W. J. (2011). Modeling rule-based item generation. Psychometrika, 76(2), 337-359.
18. Gibson, J. J. (1977). The theory of affordances. In R. Shaw & J. Bransford (Ed.), Perceiving, acting, and knowing (pp. 67-82). Mahwah, NJ: Erlbaum.
19. Gierl, M. J., & Lai, H. (2012). The role of item models in automatic item generation. International Journal of Testing, 12, 273-298.
20. Gitomer, D. H., Steinberg, L. S., & Mislevy, R. J. (1995). Diagnostic assessment of trouble-shooting skill in an intelligent tutoring system. In P. Nichols, S. Chipman, & R. Brennan (Eds.), Cognitively diagnostic assessment (pp. 73-101). Hillsdale, NJ: Erlbaum.
21. Gobert, J. D., Sao Pedro, M., Baker, R.S.J.D., Toto, E., & Montalvo, O. (2012). Leveraging educational data mining for real time performance assessment of scientific inquiry skills within microworlds. Journal of Educational Data Mining, 5, 153-185.
22. Greeno, J. G. (1998). The situativity of knowing, learning, and research. American Psychologist, 53, 5-26.
23. Gulliksen, H. (1950/1987). Theory of mental tests. New York: Wiley. Reprint, Hillsdale, NJ: Erlbaum.
24. Herman, J. L., Osmundson, E., & Silver, D. (2010). Capturing quality in formative assessment practice: Measurement challenges. CRESST Report 770. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
25. Hsieh, I.-L., & O'Neil, H. F., Jr. (2002). Types of feedback in a computer-based collaborative problem solving group task. Computers in Human Behavior, 18, 699-715.
26. Kerr, D. & Chung, G.K.W.K. (2012). Using cluster analysis to extend usability testing to instructional content. CRESST Report 816. Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
27. Kimball, R. (1982). A self-improving tutor for symbolic integration. In D. Sleeman & J. S. Brown (Eds.), Intelligent tutoring systems (pp. 283-307). London: Academic Press.
28. Kolen, M. J., & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer.
29. Levy, R. (2012, May). Psychometric advances, opportunities, and challenges for simulation-based assessment. Princeton, NJ: K-12 Center at ETS. Retrieved from www.k12center.org/rsc/pdf/session2-levy-paper-tea2012.pdf
30. Linn, R. L. (1994). Performance assessment: Policy promises and technical measurement standards. Educational Researcher, 23(9), 4-14.
31. Lord, F. M. (1970). Some test theory for tailored testing. In W. H. Holtzman (Ed.), Computer-assisted instruction, testing and guidance (pp. 139-183). New York: Harper & Row.
32. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
33. Luecht, R. M. (2003). Multistage complexity in language proficiency assessment: A framework for aligning theoretical perspectives, test development, and psychometrics. Foreign Language Annals, 36, 527-535.
34. Margolis, M. J., & Clauser, B. E. (2006). A regression-based procedure for automated scoring of a complex medical performance assessment. In. D. W. Williamson, I. I. Bejar, & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123-168). Mahwah, NJ: Lawrence Erlbaum.
35. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education/Macmillan.
36. Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
37. Mislevy, R. J. (2006). Cognitive psychology and educational assessment. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 257-305). Phoenix, AZ: Greenwood.
38. Mislevy, R. J. (in press). Missing responses in item response theory. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory, (2nd ed., Vol. 1). Chapman & Hall/CRC Press.
39. Mislevy, R. J. Behrens, J. T., DiCerbo, K. E., Frezzo, D. C., & West, P. (2012). Three things game designers need to know about assessment. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning: Foundations, innovations, and perspectives (pp. 59-81). New York: Springer.
40. Mislevy, R. J., Behrens, J. T., DiCerbo, K., & Levy, R. (2012). Design and discovery in educational assessment: Evidence centered design, psychometrics, and data mining. Journal of Educational Data Mining, 4, 11-48. Available online at www.educationaldatamining.org/JEDM/images/articles/vol4/issue1/MislevyEtAlVol4Issue1P11_48.pdf
41. Mislevy, R. J., & Gitomer, D. H. (1996). The role of probability-based inference in an intelligent tutoring system. User-Modeling and User-Adapted Interaction, 5, 253-282.
42. Mislevy, R. J., Steinberg, L. S., & Almond, R. A. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspective, 1, 3-67.
43. Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Johnson, L., & Almond, R. A. (2002). Making sense of data from complex assessments. Applied Measurement in Education, 15, 363-378.
44. Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression: A second course in statistics. Reading, MA: Addison-Wesley.
45. Moustaki, I. & Knott, M. (2000). Generalised latent trait models. Psychometrika, 65, 391-411.
46. O'Neil, H. F., Jr., & Chuang, S. H. (2008). Measuring collaborative problem solving in low stakes tests. In E. L. Baker, J. Dickieson, W. Wulfeck, & H. F. O'Neil (Eds.), Assessment of problem solving using simulations (pp. 177-199). Mahwah, NJ: Lawrence Erlbaum Associates.
47. Reckase, M. D. (2009). Multidimensional item response theory. New York, NY: Springer.
48. Riconscente, M., Mislevy, R., Hamel, L., & PADI Research Group. (2005). An introduction to PADI task templates (PADI Technical Report 3). Menlo Park, CA: SRI International. Available online at http://padi.sri.com/downloads/TR3_Templates.pdf
49. Riconscente, M. M., & Vattel, L. (2013, April). Extending ECD to the design of learning experiences. In M. M. Riconscente (Chair), ECD from A to Z: Applying evidence-centered design across the assessment continuum. Invited session presented at the National Council on Measurement in Education, San Francisco, CA.
50. Ruiz-Primo, M. A., & Shavelson, R. J. (1996). Rhetoric and reality in science performance assessments: An update. Journal of Research in Science Teaching, 33, 1045-1063.
51. Rupp, A. A., Nugent, R., & Nelson, B. (2012). Evidence-centered design for diagnostic assessment within digital learning environments: Integrating modern psychometrics and educational data mining. Journal of Educational Data Mining, 4(1), 1-10. Available online at www.educationaldatamining.org/JEDM/images/articles/vol4/issue1/IntroVol4Issue1P1_10.pdf
52. Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: Guilford Press.
53. Salen, K., & Zimmerman, E. (2004). Rules of play: Game design fundamentals. Cambridge: MIT Press.
54. Scalise, K., & Gifford, B. (2006). Computer-based assessment in e-learning: A framework for constructing "intermediate constraint" questions and tasks for technology platforms. Journal of Technology, Learning, and Assessment, 4(6), Retrieved July 16, 2013, from http://ejournals.bc.edu/ojs/index.php/jtla/article/view/1653
55. Segall, D. O. (2010). Principles of multidimensional adaptive testing. In W. J. van der Linden & C.A.W. Glas (Eds.), Elements of adaptive testing (pp. 57-75). New York: Springer.
56. Shaffer, D. W. (2006). Epistemic frames for epistemic games. Computers and Education, 46(3), 223-234.
57. Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503-524). Charlotte, NC: Information Age Publishers.
58. Shute, V. J., Ventura, M., Bauer, M. I., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M. Cody, & P. Vorder (Eds.), Serious games: Mechanisms and effects (pp. 295-321). Mahwah, NJ: Routledge, Taylor and Francis.
59. Ting, C.-Y., Phon-Amnuaisuk, S., & Chong, Y.-K. (2008). Modeling and intervening across time in scientific inquiry exploratory learning environment. Educational Technology & Society, 11, 239-258.
60. Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.
61. Von Davier, A. A., & Halpin, P. F. (2013, May). Modeling the dynamics in dyadic interactions in collaborative problem solving. Paper presented at an invited symposium at the annual meeting of the National Council of Measurement in Education, San Francisco, CA.
62. Von Davier, M. (2005). A class of models for cognitive diagnosis. Research Report RR-05-17. Princeton, NJ: ETS.
63. Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
64. Wainer, H., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., & Thissen, D. (2000). Computerized adaptive testing: A primer (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
65. Yen, W. M., & Fitzpatrick, A. R. (2006). Item response theory. In R. Brennan (Ed.), Educational measurement (pp. 111-153). Portsmouth, NH: Praeger/Greenwood.
66. Zapata-Rivera, D., & Bauer, M. (2011). Exploring the role of games in educational assessment. In M. C. Mayrath, J. Clarke-Midura, & D. Robinson (Eds.), Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research (pp. 149-172). Charlotte, NC: Information Age.