The Nation's report card: A vision of large-scale science assessment

Science

Volumn 326, Issue 5960, 2009, Pages 1637-1638

  Fu, Alice C ^a   Raizen, Senta A ^b   Shavelson, Richard J ^a  

^a STANFORD UNIVERSITY   (United States)

^b West Ed   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITION; EDUCATIONAL ATTAINMENT; EDUCATIONAL DEVELOPMENT; INNOVATION; SCIENCE AND TECHNOLOGY; STUDENT;

ACADEMIC ACHIEVEMENT; ASTRONOMY; BEHAVIOR; BIOMEDICINE; COGNITION; COMPUTER; EDUCATION; EMPIRICISM; EVALUATION; HUMAN COMPUTER INTERACTION; MODEL; NATURAL SCIENCE; PREDICTION; PRIORITY JOURNAL; PSYCHOMETRY; RELIABILITY; SCIENCE; SCORING SYSTEM; SHORT SURVEY; SIMULATION; STANDARD; STUDENT; TASK PERFORMANCE; TECHNOLOGY; VALIDITY; ACHIEVEMENT; ADOLESCENT; ARTICLE; CHILD; GEOLOGY; HUMAN; UNITED STATES;

UNITED STATES;

ACHIEVEMENT; ADOLESCENT; BIOLOGICAL SCIENCE DISCIPLINES; CHILD; EARTH SCIENCES; EDUCATIONAL MEASUREMENT; HUMANS; NATURAL SCIENCE DISCIPLINES; UNITED STATES;

EID: 72949104690     PISSN: 00368075     EISSN: 10959203     Source Type: Journal    
DOI: 10.1126/science.1177780     Document Type: Short Survey

References (33)

1. NAEP, http://nces.ed.gov/nationsreportcard/.
2. Over its 40-year history, NAEP has tracked students' science achievement roughly every four years (3). NAEP frameworks are generally used for three or four assessment cycles.
3. Different purposes of assessment include diagnosis, evaluation, program improvement, accountability, and monitoring. As large-scale snapshots of achievement, NAEP and international assessments, such as (9) and ( 10), fall primarily into the category of monitoring tools. This contrasts, e.g., with state-level science assessments in the United States, which typically measure achievement of individual students and schools for accountability purposes.
4. National Assessment Governing Board (NAGB), www.nagb.org/.
5. NAGB, Science Framework for the 2009 National Assessment of Educational Progress (U.S. Government Printing Offi ce, Washington, DC, 2008); http://nagb.org/publica-tions/frameworks/science-09.pdf.
6. National Research Council, National Science Education Standards (National Academy Press, Washington, DC, 1996).
7. AAAS, Benchmarks for Science Literacy (Oxford Univ. Press, New York, 1993).
8. To develop the 2009 NAEP Science Framework, an independent, bipartisan board (4) appointed by the Secretary of Education convened scientists, educators, education researchers, assessment experts, government offi cials, industry representatives, and members of the general public (530); all authors were actively involved in these efforts.
9. Programme for International Student Assessment (PISA), www.pisa.oecd.org.
10. Trends in International Mathematics and Science Study (TIMSS), http://timss.bc.edu.
11. J. W. Pellegrino, N. Chudowsky, R. Glaser, Eds., Knowing What Students Know: The Science and Design of Educational Assessment (National Academy Press, Washington, DC, 2001).
12. J. D. Bransford, A. L. Brown, R. R. Cocking, Eds., How People Learn: Brain, Mind, Experience, and School (National Academy Press, Washington, DC, 1999).
13. It is standard NAEP procedure to collect psychometric information on items and subject them to review and fi eld testing, with revisions as necessary. See also: http://nces. ed.gov/nationsreportcard/researchcenter/ nvspapers.asp.
14. M. A. Ruiz-Primo, R. J. Shavelson, J. Res. Sci. Teach. 33, 569 (1996).
15. J. R. McClure, B. Sonak, H. K. Suen, J. Res. Sci. Teach. 36, 475 (1999).
16. J. Clement, Am. J. Phys. 50, 66 (1982).
17. D. Hestenes, M. Wells, G. Swackhamer, Phys. Teach. 30, 141 (1992).
18. J. Minstrell et al., Diagnoser Tools (2004); www.diag-noser.com.
19. J. T. Steedle, R. J. Shavelson, J. Res. Sci. Teach. 46, 699 (2009).
20. R. T. White, R. Gunstone, Probing Understanding (Falmer Press, London, 1992).
21. Y. Yin et al., Appl. Meas. Educ. 21, 335 (2008).
22. M. A. Ruiz-Primo, R. J. Shavelson, J. Res. Sci. Teach. 33, 1045 (1996).
23. R. J. Shavelson, M. A. Ruiz-Primo, E. W. Wiley, J. Educ. Meas. 36, 61 (1999).
24. G. P. Baxter, R. Glaser, Educ. Meas. 17, 37 (1998).
25. R. J. Shavelson, G. P. Baxter, J. Pine, Appl. Meas. Educ. 4, 347 (1991).
26. E. S. Quellmalz, J. W. Pellegrino, Science 323, 75 (2009).
27. Neither ICTs nor HOTs are administered to the entire student sample in recognition of the additional time required to administer, score, and analyze these more complex tasks.
28. Y. Yin, J. Vanides, M. A. Ruiz-Primo, C. C. Ayala, R. J. Shavelson, J. Res. Sci. Teach. 42, 166 (2005).
29. W. S. Grigg, M. A. Lauko, D. M. Brockway, The Nation's Report Card: Science 2005 (National Center for Education Statistics 2006-2466, U.S. Government Printing Offi ce, Washington, DC, 2006); http://nces.ed.gov/pub- search/pubsinfo.asp?pubid=2006466.
30. NAGB, Science Assessment and Item Specifi cations for the 2009 National Assessment of Educational Progress: Science NAEP 2009 (U.S. Department of Education, Washington, DC, 2007); http://nagb.org/publications/ frameworks/naep2009-science-specs07.doc.
31. HOTs and ICTs will be excluded from scaled scores for technical and practical reasons, though a special report is expected to provide extra information. Personal communication with S. Loomis, Assistant Director for Psycho-metrics at NAGB; www.nagb.org/who-we-are/staff.htm.
32. There likely will be discussion about the quality of the assessment, although this may be diffi cult to judge, because most test items are kept secure from the public to allow repeat use in the next round of testing.
33. The views expressed in this article are those of the authors and do not necessarily represent the views of the organizations or agencies that provided support for the framework development project.