Designing assessments and assessing designs in virtual educational environments

Journal of Science Education and Technology

Volumn 18, Issue 2, 2009, Pages 187-208

  Hickey, Daniel T ^a   Ingram Goble, Adam A ^a   Jameson, Ellen M ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

Design based research; Educational videogaming; Formative assessment; Formative feedback; Virtual environments

Indexed keywords

EID: 64549161091     PISSN: 10590145     EISSN: 15731839     Source Type: Journal    
DOI: 10.1007/s10956-008-9143-1     Document Type: Article

References (69)

1. RG Almond RJ Mislevy 1999 Graphical models and computerized adaptive testing Appl Psychol Meas 23 3 223 10.1177/01466219922031347
2. Arici AB (2008) Unpublished dissertation. Indiana University, Departments of Learning and Development Science and Cognitive Science
3. Barab S (2006) Design-based research: a methodological toolkit for the learning scientist. In: Sawyer K (ed) The handbook of the learning sciences. Cambridge University Press, Cambridge, pp 153-170
4. S Barab C Dede 2007 Games and immersive participatory simulations for science education: an emerging type of curricula J Sci Educ Technol 16 1 1 3 10.1007/s10956-007-9043-9
5. S Barab T Sadler C Heiselt D Hickey S Zuiker 2007 Relating narrative, inquiry, and inscriptions: a framework for socioscientific inquiry J Sci Educ Technol 16 59 82 10.1007/s10956-006-9033-3
6. SA Barab S Zuiker S Warren D Hickey A Ingram-Goble E-J Kwon I Kouper SC Herring 2007 Embodied curriculum: relating formalisms to contexts Sci Educ 91 5 750 782 10.1002/sce.20217
7. Bass KM, Glaser R (2004) Developing assessments to inform teaching and learning. National Center for Research on Evaluation Standards and Student Testing, Los Angeles
8. Bell P, Hoadley CM, Linn MC (2004) Design-based research in education. In: Linn MC, Davis EA, Bell P (eds) Internet environments for science education. Erlbaum, Mahwah, pp 73-85
9. Bereiter C, Scardamalia M (1989) Intentional learning as a goal of instruction. In: Resnick L (ed) Cognition and instruction: issues and agendas. Lawrence Erlbaum Associates, Hillsdale, pp 361-379
10. P Black D Wiliam 1998 Assessment and classroom learning Assess Educ 5 1 7 74 10.1080/0969595980050102
11. H Burkhardt AH Schoenfeld 2003 Improving educational research; toward a more useful, more influential, and better-funded enterprise Educ Res 32 9 3 14 10.3102/0013189X032009003
12. S Burroughs E Groce ML Webeck 2005 Social studies education in the age of testing and accountability Educ Meas 24 13 20 10.1111/j.1745-3992.2005.00015.x
13. P Cobb J Confrey A diSessa R Lehrer L Schauble 2003 Design experiments in educational research Educ Res 32 1 9 10.3102/0013189X032001009
14. J Cohen 1992 Quantitative methods in psychology: a power primer Psychol Bull 12 1 155 10.1037/0033-2909.112.1.155
15. LJ Cronbach RL Linn RL Brennan EH Haertel 1997 Generalizability analysis for performance assessments of student achievement or school effectiveness Educ Psychol Meas 57 3 373 10.1177/0013164497057003001
16. RA Duschl DH Gitomer 1997 Strategies and challenges to changing the focus of assessment and instruction in science classrooms Educ Assess 4 37 73 10.1207/s15326977ea0401-2
17. Federation of American Scientists (2006) Harnessing the power of video games for learning. Federation of American Scientists, Washington
18. Feng M, Heffernan NT, Koedinger KR (2006) Addressing the testing challenge with a web-based E-assessment system that tutors as it assesses. In: Proceedings of the 15th international conference on the World Wide Web, Edinburgh, pp 307-316
19. MJ Feuer L Towne RJ Shavelson 2002 Scientific culture and educational research Educ Res 31 8 4 10.3102/0013189X031008004
20. JR Frederiksen A Collins 1989 A systems approach to educational testing Educ Res 18 9 27 32
21. Gee JP (2003a) What video games have to teach us about learning and literacy, 1st edn. Palgrave Macmillan, New York
22. JP Gee 2003 Opportunity to learn: a language-based perspective on assessment Assess Educ 10 1 27 47 10.1080/09695940301696
23. Gee JP (2004) Situated language and learning: a critique of traditional schooling. Routledge, New York
24. Gee JP (2007) Learning and games. In: Salen K (ed) The ecology of games: connecting youth, games, and learning. MIT Press, Cambridge, MA, pp 21-40
25. RL Goldstone JY Son 2005 The transfer of scientific principles using concrete and idealized simulations J Learn Sci 14 69 110 10.1207/ s15327809jls1401-4
26. JG Greeno 1997 On claims that answer the wrong questions Educ Res 26 1 5
27. JG Greeno Middle School Mathematics through Applications Project Group 1998 The situativity of knowing, learning, and research Am Psychol 53 5 26 10.1037/0003-066X.53.1.5
28. Greeno JG, Collins AM, Resnick LB (1996) Cognition and learning. In: Berliner DC, Calfee RC (eds) Handbook of educational psychology. Macmillan, New York, pp 15-46
29. L Habib L Wittek 2007 The portfolio as artifact and actor Mind Cult Act 14 4 266 282 10.1080/10749030701623763
30. EH Haertel 1999 Performance assessment and educational reform Phi Delta Kappan 80 9 662 667
31. EH Haertel JG Greeno 2003 A situative perspective: broadening the foundations of assessment Measurement 1 154 161
32. RD Hannafin WR Foshay 2008 Computer-based instruction's (CBI) rediscovered role in K-12: an evaluation case study of one high-school's use of CBI to improve pass rates on high-stakes tests Educ Technol Res Dev 56 147 160 10.1007/s11423-006-9007-4
33. Harlen W (2007) On the relationship between assessment for formative and summative purposes. In: Gardner J (ed) Assessment and learning. Sage Publications Ltd., London, pp 103-118
34. H Hattie H Timperley 2007 The power of feedback Rev Educ Res 77 81 112 10.3102/003465430298487
35. Hickey DT, Anderson K (2007) Situative approaches to assessment for resolving problems in educational testing and transforming communities of educational practice. In: Moss P (ed) Evidence and decision making. The 103rd NSSE Yearbook. National Society for the Study of Education/University of Chicago Press, Chicago, pp 269-293
36. Hickey DT, Pellegrino JW (2005) Theory, level, and function: three dimensions for understanding the connections between transfer and student assessment. In: Mestre JP (ed) Transfer of learning from a modern multidisciplinary perspective. Information Age Publishers, Greenwich, pp 251-253
37. DT Hickey SJ Zuiker 2005 Engaged participation: a sociocultural model of motivation with implications for assessment Educ Assess 10 277 305 10.1207/s15326977ea1003-7
38. DT Hickey ACH Kindfield P Horwitz MA Christie 2003 Integrating instruction, assessment, & evaluation in a technology-supported genetics environment Am Educ Res J 40 2 495 538 10.3102/00028312040002495
39. DT Hickey SJ Zuiker G Taasoobshirazi NJ Schafer MA Michael 2006 Three is the magic number: a design-based framework for balancing formative and summative functions of assessment Stud Educ Eval 32 180 201 10.1016/j.stueduc.2006.08.006
40. D Ketelhut 2007 The impact of student self-efficacy on scientific inquiry skills: an exploratory investigation in river city, a multi-user virtual environment J Sci Educ Technol 16 1 99 111 10.1007/s10956-006-9038-y
41. KR Koedinger JR Anderson WH Hadley MA Mark 1997 Intelligent tutoring goes to school in the big city Int J Artif Intell Educ 8 30 43
42. J Lemke 2000 Across the scales of time Mind Cult Act 7 4 273 290 10.1207/S15327884MCA0704-03
43. T Martindale C Pearson L Curda J Pilcher 2005 Effects of an online instructional application on reading and mathematics standardized test scores J Res Technol Educ 37 349 360
44. S Messick 1994 The interplay of evidence and consequences in the validation of performance assessments Educ Res 23 2 13 18
45. S Messick 1995 Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning Am Psychol 50 9 741 749 10.1037/0003-066X.50.9.741
46. PA Moss 1992 Shifting conceptions of validity in educational measurement: implications for performance assessment Rev Educ Res 62 229 258
47. National Research Council (2001a) In: Atkin JM, Black P, Coffey J (eds) Classroom assessment and the national science education standards. National Academies Press, Washington
48. National Research Council (2001b) In: Pellegrino JW, Chudowski N, Glaser RW (eds) Knowing what students know: the science and design of educational assessment. National Academy Press, Washington
49. National Research Council (2006) Systems for state science assessment. The National Academies Press, Washington
50. B Nelson DJ Ketelhut J Clarke C Bowman C Dede 2005 Design-based research strategies for developing a scientific inquiry curriculum in a multi-user virtual environment Educ Technol 45 1 21 27
51. Phelps RP (ed) (2005) Defending standardized testing. Erlbaum, Matewan
52. DR Sadler 1989 Formative assessment and the design of instructional systems Instr Sci 18 2 119 144 10.1007/BF00117714
53. TD Sadler 2004 Informal reasoning regarding socioscientific issues: a critical review of research J Res Sci Teach 41 5 513 536 10.1002/tea.20009
54. RJ Shavelson GP Baxter J Pine 1991 Performance assessment in science Appl Meas Educ 4 4 347 362 10.1207/s15324818ame0404-7
55. LA Shepard 1993 Evaluating test validity Rev Educ Res 19 405 450
56. VJ Shute 2008 Focus on formative feedback Rev Educ Res 78 153 189 10.3102/0034654307313795
57. Shute VJ, Hansen EG, Almond RG (2008) You can't fatten a hog by weighing it-or can you? Evaluating an assessment system for learning called ACED. Int J Artif Intell Educ 18:289-316
58. K Squire 2006 From content to context: videogames as designed experience Educ Res 35 8 19 10.3102/0013189X035008019
59. CA Steinkuehler 2006 Massively multiplayer online video gaming as participation in a discourse Mind Cult Act 13 1 38 52 10.1207/s15327884mca1301-4
60. Sternberg RJ (2006) Real improvement for real students: test smarter, serve better. Harv Educ Rev 76(4):557-563
61. G Taasoobshirazi KA Anderson SJ Zuiker DT Hickey 2006 Enhancing inquiry, understanding, and achievement in an astronomy multimedia astronomy learning environment J Sci Educ Technol 15 383 395 10.1007/s10956-006-9028-0
62. United States House of Representatives (2002) House resolution 3801, Educational Sciences Reform Act of 2002
63. CE Von Secker RW Lissitz 1999 Estimating the impact of instructional practices on student's achievement in science J Res Sci Teach 36 10 1110 1126 10.1002/(SICI)1098-2736(199912)36:10<1110::AID-TEA4>3.0.CO;2-T
64. Wenger E (1998) Communities of practice: learning, meaning, and identity. Cambridge University Press, Cambridge
65. Wiggins G (2001) Assessing student performance: exploring the purpose and limits of testing. Jossey-Bass, San Francisco
66. Wilson M (ed) (2004) Towards coherence in classroom assessment and accountability. The 103rd Yearbook of the National Society for the Study of Education. University of Chicago Press, Chicago
67. Winerip M (2005) Are schools passing or failing? Now there's a third choice...both. New York Times, November 2, p 1
68. JE Ysseldyke S Tardrew 2007 Use of a progress monitoring system to enable teachers to differentiate mathematics instruction J Appl Sch Psychol 24 1 28 10.1300/J370v24n01-01
69. Zuiker SJ (2007) Transforming practice: designing for liminal transitions along trajectories of participation. Unpublished doctoral dissertation, Indiana University