Validity evidence for learning progression-based assessment items that fuse core disciplinary ideas and science practices

Journal of Research in Science Teaching

Volumn 50, Issue 5, 2013, Pages 597-626

  Gotwals, Amelia Wenk ^a   Songer, Nancy Butler ^b  

^a MICHIGAN STATE UNIVERSITY   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

assessment; learning progressions; validity

Indexed keywords

ASSESSMENT; ASSESSMENT SYSTEM; ASSESSMENT TASKS; EVIDENCE-BASED; ITEM DIFFICULTIES; LEARNING PROGRESSIONS; VALIDITY; VALIDITY EVIDENCE;

EDUCATION; RESEARCH;

STUDENTS;

EID: 84876292291     PISSN: 00224308     EISSN: 10982736     Source Type: Journal    
DOI: 10.1002/tea.21083     Document Type: Article

References (47)

1. Adams, R. J., &, Wilson, M., (1996). Formulating the Rasch model as a mixed coefficients multinomial logit. In, G. Engelhard, &, M. Wilson, (Eds.), Objective measurement III: Theory into practice (pp. 143-166). Norwood, NJ: Ablex.
2. AERA, APA, & NCME. (1999). Standards for educational and psychological testing. Washington, DC: AERA.
3. Alonzo, A. C., (2012). Eliciting student responses relative to a learning progression. In, A. C. Alonzo, &, A. W. Gotwals, (Eds.), Learning progressions in science: Current challenges and future directions (pp. 241-254). Rotterdam, The Netherlands: Sense Publishing.
4. Alonzo, A. C., &, Steedle, J. T., (2009). Developing and assessing a force and motion learning progression. Science Education, 93, 389-421.
5. American Association for the Advancement of Science [AAAS]. (1993). Benchmarks for science literacy. New York: Oxford University Press.
6. Anderson, C. W., Alonzo, A. C., Smith, C., &, Wilson, M., (2007, August). NAEP Pilot Learning Progression Framework. Report to the National Assessment Governing Board.
7. Ayala, C. C., (2002). On The Cognitive Validity Of Science Performance Assessments Using A Research Based Knowledge Framework. Unpublished Doctoral Dissertation, Stanford University, Palo Alto, CA.
8. Berland, L. K., &, McNeill, K. L., (2010). A learning progression for scientific argumentation: Understanding student work and designing supportive instructional contexts. Science Education, 94, 765-793.
9. Berland, L. K., &, Reiser, B. J., (2010). Classroom communities adaptations of the practice of scientific argumentation. Science Education, 95, 191-216.
10. Bond, T. G., &, Fox, C. M., (2001). Applying the Rasch model: Fundamental measurement in human sciences. Mahwah, NJ: Erlbaum.
11. Bricker, L., &, Bell, P., (April, 2007). Um. since I argue for fun, I don't remember what I argue about: Using children's argumentation across social contexts to inform science instruction. Paper presented at the National Association of Research in Science Teaching, New Orleans, LA.
12. Chi, M. T. H., (1997). Quantifying qualitative analyses of verbal data: A practical guide. The Journal of the Learning Sciences, 6 (3), 271-315.
13. Chi, M. T. H., Feltovich, P. J., &, Glaser, R., (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5 (2), 121-152.
14. College Board. (2009). Science College Board Standards for College Success. Available: http://professionals.collegeboard.com/profdownload/cbscs- sciencestandards-2009.pdf [June 2011].
15. Corcoran, T., Mosher, F. A., &, Rogat, A., (2009, May). Learning progressions in science: An evidence-based approach to reform (CPRE Research Report #RR-63). Philadelphia, PA: Consortium for Policy Research in Education.
16. DeBarger, A. H., Quellmalz, E., Fried, R., &, Fujii, R., (2006). Examining the validities of science inquiry assessments with cognitive analyses. Paper presented at the Annual meeting of the American Educational Research Association, San Francisco, CA.
17. Duschl, R., Maeng, S., &, Sezen, A., (2011). Learning progression and teaching sequences: A review and analysis. Studies in Science Education, 47 (2), 123-182.
18. Ericsson, K. A., &, Simon, H. A., (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.
19. Gotwals, A. W., &, Songer, N. B., (2006). Measuring students' scientific content and inquiry reasoning. In, S. A. Barab, K. E. Hay, &, D. T. Hickey, (Eds.), The Proceedings of the Seventh International Conference of the Learning Sciences (pp. 196-202). Mahwah, NJ: Lawrence Earlbaum Assoc.
20. Gotwals, A. W., &, Songer, N. B., (2010). Reasoning up and down a food chain: using an assessment framework to investigate students' middle knowledge. Science Education, 94, 259-281.
21. Gotwals, A. W., Songer, N. B., &, Bullard, L., (2012). Assessing students' progressing abilities to construct scientific explanations. In, A. C. Alonzo, &, A. W. Gotwals, (Eds.), Learning progressions in science (pp. 183-210). The Netherlands: Sense Publishing.
22. Kane, M. T., (1992). An argument-based approach to validity. Psychological Bulletin, 112 (3), 527-535.
23. Kane, M. T., (2001). Current concerns in validity theory. Journal of Educational Measurement, 38 (4), 319-342.
24. Kupermintz, H., Le, V.-N., &, Snow, R. E., (1999). Construct validation of mathematics achievement: Evidence from interview procedures. Los Angeles, CA: Stanford University: National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
25. Liu, O. L., Lee, H.-S., Hofstedder, C., &, Linn, M. C., (2008). Assessing knowledge integration in science: Construct, measures and evidence. Educational Assessment, 13, 33-55.
26. McNeill, K., (2011). Elementary students' views of explanation, argumentation, and evidence, and their abilities to construct arguments over the school year. Journal of Research in Science Teaching, 48 (7), 793-823.
27. McNeill, K., &, Krajcik, J., (2007). Middle school students' use of appropriate and inappropriate evidence in writing scientific explanations. In, M. Lovett, &, P. Shah, (Eds.), Thinking with data (pp. 233-265). New York: Taylor & Francis.
28. McNeill, K., Lizotte, D. J., Krajcik, J., &, Marx, R. W., (2006). Supporting students' construction of scientific explanations by fading scaffolds in instructional materials. Journal of the Learning Sciences, 15 (2), 153-191.
29. Messick, S., (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23 (2), 13-23.
30. Mislevy, R. J., (2012). The case for informal argument. Measurement, 10, 93-96.
31. Mislevy, R. J., Steinberg, L. S., &, Almond, R. G., (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1 (1), 3-62.
32. National Assessment Governing Board. (2008). Science framework for the 2009 National Assessment of Educational Progress. Retrieved from National Assessment Governing Board website: http://www.nagb.org/publications/frameworks/ science-09.pdf
33. National Research Council. (2001). Knowing what students know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.
34. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: The National Academies Press.
35. National Research Council. (2011). A Framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: The National Academies Press.
36. Newton, P. E., (2012). Clarifying the consensus definition of validity. Measurement, 10, 1-29.
37. Pellegrino, J. W., (2012). Comment: Assessment of science learning: Living in interesting times. Journal of Research in Science Teaching, 49 (6), 831-841.
38. Rasch, G., (1960). Probabalistic models for some intelligence and attainment tests. Chicago: University of Chicago Press.
39. Ruiz-Primo, M. A., Li, M., Tsai, S.-P., &, Schneider, J., (2010). Testing one premise of scientific literacy in classrooms: Examining students' scientific explanations and student learning. Journal of Research in Science Teaching, 47 (5), 583-608.
40. Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L., &, Klein, S., (2002). On the evaluation of systemic education reform: Searching for instructional sensitivity. Journal of Research in Science Teaching, 39 (5), 369-393.
41. Songer, N. B., &, Gotwals, A. W., (2012). Guiding explanation construction by children at the entry points of learning progressions. Journal for Research in Science Teaching, 49, 141-165.
42. Songer, N. B., Kelcey, B., &, Gotwals, A. W., (2009). How and when does complex reasoning occur? Empirically driven development of a learning progression focused on complex reasoning in biodiversity. Journal of Research in Science Teaching, 46 (6), 610-631.
43. Songer, N. B., Shah, A. M., &, Fick, S., (in press). Characterizing teachers' verbal scaffolds to guide elementary students' creation of scientific explanations. School Science and Mathematics.
44. Steedle, J. T., &, Shavelson, R. J., (2009). Supporting valid interpretations of learning progression level diagnoses. Journal of Research in Science Teaching, 46 (6), 699-715.
45. White, B., &, Frederiksen, J. R., (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition and Instruction, 16 (1), 3-118.
46. Wilson, M., (2005). Constructing measures: An item response modeling approach. Mahwah, New Jersey: Laurence Erlbaum Associates, Publishers.
47. Yue, Y., Ayala, C. C., &, Shavelson, R. J., (2002). Student's problem solving strategies in performance assessments: Hands on minds on. Paper presented at the Paper presented at the American Educational Research Association, New Orleans, LA.