How do American students measure up? Making sense of international comparisons

Future of Children

Volumn 19, Issue 1, 2009, Pages 37-51

  Koretz, Daniel ^a  

^a HARVARD GRADUATE SCHOOL OF EDUCATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACADEMIC ACHIEVEMENT; ACHIEVEMENT TEST; GEOGRAPHIC DISTRIBUTION; GOLD STANDARD; HIGH SCHOOL STUDENT; INTERMETHOD COMPARISON; MATHEMATICS; READING; REVIEW; SCIENCE; UNITED STATES;

ADOLESCENT; BIAS (EPIDEMIOLOGY); CROSS-CULTURAL COMPARISON; DATA COLLECTION; EDUCATIONAL MEASUREMENT; EDUCATIONAL STATUS; FEMALE; HUMANS; MALE; MATHEMATICS; MINORITY GROUPS; PEER GROUP; READING; SCIENCE; UNITED STATES;

EID: 66849119059     PISSN: 10548289     EISSN: 15501558     Source Type: Journal    
DOI: 10.1353/foc.0.0023     Document Type: Review

References (26)

1. National Commission on Excellence in Education, A Nation at Risk (Washington: U.S. Department of Education, April 1983).
2. S. Cavanagh and E. W. Robelen, "U.S. Students Fare Poorly in International Math Comparison," Education Week, December 7, 2004, www.edweek.org [August 20, 2008].
3. TIMSS initially stood for the "Third International Mathematics and Science Study," a reference to two earlier, related studies of mathematics conducted in the 1960s and 1970s. When a decision was made to repeat TIMSS at regular intervals, the name was changed to "Trends in International Mathematics and Science Study.quot;
4. R. Adams and M. Wu, eds., PISA 2000 Technical Report (Paris: Organisation for Economic Co-operation and Development, 2002).
5. The TIMSS sample is actually a bit more complex than this. TIMSS surveys students in the two grades that include the largest proportions of nine- and thirteen-year-olds. In most countries, that means grades three and four and seven and eight, but the primary reporting has usually been of the older of the two grades in each pair.
6. I. V. Mullis and others, Mathematics and Science Achievement in the Final Year of Secondary School: IEA's Third International Mathematics and Science Study (TIMSS) (Chestnut Hill, Mass.: TIMSS International Study Center, Boston College, 1998).
7. United States Department of Education, Policy Brief: What the TIMSS Means for Systemic School Improvement (Washington: November 1998). Archived information available at www.ed.gov/pubs/TIMSS Brief/student.html [June 2, 2002].
8. I. V. Mullis and others, TIMSS 1999 International Mathematics Report (Chestnut Hill, Mass.: International Study Center, Boston College, 2002).
9. For a non-technical explanation of this principle and more discussion of its implications, see D. Koretz, Measuring Up: What Educational Testing Really Tells Us (Harvard University Press, 2008).
10. Mullis and others, TIMSS 1999 International Mathematics Report (see note 8).
11. R. G. Wolfe, "Country-by-Item Interactions: Problems with Content Validity in Scaling," presented at a symposium on "Validity in Cross-National Assessments: Problems and Pitfalls," annual meeting of the American Educational Research Association, May 27, 1997, Chicago.
12. L. S. Gronmo and R. V. Olsen, "TIMSS versus PISA: The Case of Pure and Applied Mathematics," paper presented at the Second IEA International Research Conference, November 8-11, 2006, Washington, D.C.
13. Estimates of statistical significance in the reports of both TIMSS and PISA take into account measurement error as well as sampling error. However, these estimates of measurement error take the design of the test as a given and do not reflect changes in performance that would arise from altering it.
14. Mullis and others, Mathematics and Science Achievement in the Final Year of Secondary School (see note 6).
15. Mullis et al. Mathematics and Science Achievement in the Final Year of Secondary School Ibid.
16. Mullis et al. Mathematics and Science Achievement in the Final Year of Secondary School Ibid.
17. A. E. Lapointe, N. A. Mead, and J. M. Askew, Learning Mathematics (Princeton, N.J.: Educational Testing Service, 1992).
18. A. E. Lapointe, N. A. Mead, and G. W. Phillips, A World of Differences: An International Assessment of Mathematics and Science (Report No. 19-CAEP-01)(Princeton, N.J.: Educational Testing Service, 1989). A third study with the same finding was the International Assessment of Educational Progress, an extension of the U.S. National Assessment of Educational Progress, conducted in 1988 and 1991, but this study is now rarely noted.
19. In the case of international comparisons, standardization is not entirely straightforward. The standard deviation of the tested population and the average country-level standard deviation are functions of the group of countries tested and are therefore not useful for this purpose. The estimates of within-country standard deviations are not highly stable. The following text uses the 2003 estimates of the U.S. standard deviation. As explained below, choosing another nation's standard deviation would not have greatly changed the results. For a non-technical explanation of standardized score scales and their application to international comparisons and trends in the United States, see Koretz, Measuring Up (see note 9).
20. Compare I. V. Mullis and others, TIMSS 2003 International Mathematics Report (Chestnut Hill, Mass.: International Study Center, Boston College, 2004),
21. exhibit 1.1, with Organisation for Economic Co-operation and Development, Learning for Tomorrow's World: First Results from PISA 2003 (Paris: OECD, 2004), figure 2.15a.
22. Learning for Tomorrow's World: First Results from PISA 2003 (2004), Ibid.
23. Mullis and others, TIMSS 2003 International Mathematics Report (see note 19).
24. W. B. Elley, How in the World Do Children Read? (The Hague: International Association for the Evaluation of Educational Achievement, 1992);
25. I. V. Mullis and others, PIRLS 2001 International Report (Chestnut Hill, Mass.: International Study Center, Boston College, 2003).
26. National Center for Education Statistics, Highlights from the 2000 Program for International Student Assessment (PISA) (Washington: Office of Educational Research and Improvement, U.S. Department of Education, 2002).