Assessing literacy across a changing world

Science

Volumn 328, Issue 5977, 2010, Pages 433-434

  Schleicher, Andreas ^a  

^a OECD   (France)

Author keywords

[No Author keywords available]

Indexed keywords

EDUCATION; EDUCATION POLICY; EDUCATIONAL ATTAINMENT; INFORMATION SYSTEM; LABOR MARKET; LITERACY; STUDENT; WORLD WIDE WEB;

CONTROLLED STUDY; EDUCATION; HUMAN; KNOWLEDGE; OUTCOME ASSESSMENT; PRIORITY JOURNAL; READING; SCHOOL; SHORT SURVEY; SKILL; STUDENT; TASK PERFORMANCE; TRAINING; ARTICLE; EDUCATIONAL STATUS; INTERNATIONAL COOPERATION; LEARNING;

EDUCATIONAL MEASUREMENT; EDUCATIONAL STATUS; HUMANS; INTERNATIONALITY; LEARNING; READING;

EID: 77951618581     PISSN: 00368075     EISSN: 10959203     Source Type: Journal    
DOI: 10.1126/science.1183092     Document Type: Short Survey

References (24)

1. OECD, PISA 2006 Science Competencies for Tomorrow's World (OECD, Paris, 2007).
2. OECD and Statistics Canada, Literacy in the Information Age: Final Report of the International Adult Literacy Survey (OECD, Paris, and Statistics Canada, Ottawa, 2000).
3. D. Autor, F. Levy, R. J. Murnane, Q. J. Econ. 118, 1279 (2003).
4. F. Levy, "How technology changes demands for human skills" (OECD Education Working Paper 45, OECD, Paris, 2010); www.olis.oecd.org/olis/ 2010doc.nsf/ ENGDATCORPLOOK/NT00000D86/$FILE/JT03279582.PDF.
5. International Reading Association (IRA), "IRA position statement on new literacies and 21st-century technology" (IRA, Newark, DE, 2009).
6. For an overview of assessment systems, see (22).
7. The first international tests were established by the International Association for the Evaluation of Educational Achievement (IEA).
8. A. W. Foshay, R. L. Thorndike, F. Hotyat, D. A. Pidgeon, D. A. Walker, Educational Achievement of Thirteen-Year-Olds in Twelve Countries (UNESCO Institute for Education, Hamburg, Germany, 1962).
9. Other current major international assessments include the IEA Progress in Reading Literacy Study (PIRLS), the OECD Program for the International Assessment of Adult Competencies (PIAAC), the Assessment of the Southern Africa Consortium for Monitoring Educational Quality (SACMEQ), and the IEA Trends in Mathematics and Science Study (TIMSS).
10. OECD, The PISA 2009 Assessment Framework-Key Competencies in Reading, Mathematics and Science (OECD, Paris, 2010).
11. The part of the performance distribution into which most respondents fall is cut into five equally sized levels. Some respondents fall below the lower bound of Level 1. Level 2 is considered "baseline," at which students have acquired basic building blocks for competency. Students below level 2 face disproportionately higher risk of poor school-work transitions and labor-market outcomes.
12. R. Adams, A. Berezner, M. Jakubowski, "Analysis of PISA 2006 Preferred Items Ranking Using the Percent-Correct Method" (OECD Education Working Paper 46, OECD, Paris, 2010); www.pisa.oecd.org/ dataoecd/20/43/ 44919855.pdf.
13. Rankings were remarkably consistent even when countries were ranked by average performance across every other country's subset of preferred items (12).
14. Open-ended assessment tasks raise other challenges, in particular, the need to ensure that they are rated consistently and fairly across schools and countries. For PISA, subsamples of assessment tasks need to be rated independently by up to four experts. An international coding review needs to be implemented to check on the consistency of application of response coding standards across all participating countries (23).
15. J. Butler, R. J. Adams, Appl. Meas. 8, 279 (2007).
16. There is a tension between external validity and crossnational comparability that can be seen both in task formats (open-ended tasks aim at increasing external validity but are challenging in terms of achieving crosscultural comparability) and context-sensitivity (it is easier to assess recall of knowledge across countries than the application of knowledge, but the latter is critical for external validity). Trade-offs are made, for example, to protect against measuring in comparable ways things that may not be very important.
17. OECD, Pathways to Success: How Knowledge and Skills at Age 15 Shape Future Lives in Canada (OECD, Paris, 2010).
18. A study in Denmark led to similar results. The percentage of youth who completed postcompulsory, general, or vocational upper-secondary education by 19 increased significantly with reading ability assessed at age 15 (24).
19. The actual performance of a student in solving a task is observed and measured, rather than merely the degree of correctness of their response.
20. Such tests support inferences about the manipulations, strategies, et cetera that students used. For example, PISA will try this in 2012 with an assessment of dynamic problem-solving, where everything students do will be recorded by the computer.
21. Good tests reward divergent, "out-of-the-box" thinking rather than having students just reproduce and apply routines they have learned. Such tests allow students to devise their own solution strategies, rather than precoding those in the design of the tasks.
22. OECD, Education at a Glance 2009: OECD Indicators (OECD, Paris, 2009).
23. OECD, PISA 2006 Technical Report (OECD, Paris, 2009); www.oecd.org/dataoecd/0/47/42025182.pdf.
24. T. P. Jensen, D. Andersen, in Northern Lights on PISA 2003-A Reflection from the Nordic Countries, J. Mejding, A. Roe, Eds. (Nordic Council of Ministers, Copenhagen, 2006), pp. 239-248.