Reporting results of large-scale assessment in psychologically and educationally meaningful terms: Construct validation and proficiency scaling in TIMSS

European Journal of Psychological Assessment

Volumn 18, Issue 3, 2002, Pages 190-203

  Watermann, Rainer ^a   Klieme, Eckhard ^a  

^a MAX PLANCK INSTITUTE FOR HUMAN DEVELOPMENT   (Germany)

Author keywords

Large scale assessment; Mathematical problem solving; Proficiency scaling

Indexed keywords

EID: 0036397559     PISSN: 10155759     EISSN: None     Source Type: Journal    
DOI: 10.1027//1015-5759.18.3.190     Document Type: Article

References (35)

1. Adams, R.J., Wilson, M.R., & Wang, W.C. (1997). The multidimensional random coefficients multinomial logit. Applied Psychological Measurement, 21, 1-24.
2. Aebli, H. (1981). Denken, das Ordnen des Tuns: Bd. 2. Denkprozesse [Thought, the organization of action: Vol. 2. Thought processes]. Stuttgart: Klett-Cotta.
3. Baumert, J., Bos, W., Klieme, E., Lehmann, R., Lehrke, M., Hosenfeld, I., Neubrand, J., & Watermann, R. (Eds.). (1999). Test-aufgaben zu TIMSS/III. Mathematisch-naturwissenschaftliche Grundbildung und voruniversitäre Mathematik und Physik der Abschluβklassen der Sekundarstufe II (Population 3) [TIMSS mathematics and science items for the final year of secondary school]. Berlin: Max-Planck-Institut für Bildungsforschung (Materialien aus der Bildungsforschung, 62).
4. Baumert, J., Klieme, E., Lehrke, M., & Savelsbergh, E. (2000). Konzeption und Aussagekraft der TIMSS-Leistungstests. Zur Diskussion um TiMSS-Aufgaben aus der Mittelstufenphysik [Conception and diagnostic power of the TIMSS tests. On the discussion of the TIMSS physics items for the middle school]. Die deutsche Schule, 92, 102-115 (Part 1); 196-217 (Part 2).
5. Baumert, B., Klieme E., & Watermann, R. (1998). Jenseits von Gesamttest-und Untertestwerten: Analyse differentieller Item-funktionen am Beispiel des mathematischen Grundbildungstests der Dritten Internationalen Mathematik-und Naturwissenschaftsstudie der IEA (TIMSS) [Beyond combined test scores and subtest scores: An analysis of differential item functioning based on the basic mathematics test in the IEA's Third International Mathematics and Science Study (TIMSS)]. In H.-J. Herber & F. Hofmann (Eds.), Schulpädagogik und Lehrerbildung [Schooling and teacher training] (pp. 301-324). Innsbruck, Wien: Studien Verlag.
6. Barnett, K. (1979). The study of syntax variables. In G.A. Golding & C.E. McClintock (Eds.), Task variables in mathematical problem solving (pp. 23-68). Philadelphia, PA: Franklin Institute Press.
7. Beaton, A.E., & Allen, N.L. (1992). Interpreting Scales through Scale Anchoring. Journal of Educational Statistics, 17, 191-204.
8. Bloom, B.S. (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook 1. Cognitive domain. New York: Longman.
9. Freudenthal, H. (1983). Didactical phenomenology of mathematical structures. Dodrecht: Riedel.
10. Gagné, R.M., & Briggs, L.J. (1974). Principles of instructional design. New York: Holt, Rinehart & Winston.
11. Glaser, R. (1994). Instructional technology and the measurement of learning outcomes: Some questions. Educational Measurement: Issues & Practice, 13(4), 21-26.
12. Golding, G.A., & McClintock, C.E. (1979). Task variables in mathematical problem solving. Philadelphia, PA: Franklin Institute Press.
13. Grouws, D.A. (Ed.) (1992). Handbook of research on mathematics, teaching and learning (NCTM). New York: Macmillan.
14. Hambleton, R.K. (1994). The rise and fall of criterion-referenced measurement? Educational Measurement: Issues & Practice, 13(4), 21-26.
15. Kintsch, W., & Greeno, J.G. (1985). Understanding and solving word arithmetic problems. Psychological Review, 92, 109-129.
16. Klauer, K.C. (1987). Kriteriumsorientiertes Testen: Der Schluß auf den Itempool [Criterion-referenced testing: Inference regarding item pool]. Zeitschrift für Differentielle und Diagnostische Psychologie, 8, 141-147.
17. Klieme, E., & Baumert, J. (2001). Identifying national cultures of mathematics education: Analysis of cognitive demands and differential item functioning in TIMSS. European Journal of Psychology of Education, 16(3), 383-400.
18. Klieme, E., Bos, W. (2000). Mathematikleistungen und mathematischer Unterricht in Deutschland und Japan. Triangulation qualitativer und quantitativer Forschungsansätze im Rahmen der TIMS-Studie [Mathematics achievement and mathematics instruction in Germany and Japan: Triangulation of qualitative and quantitative research approaches in the framework of the TIMS study]. Zeitschrift für Erziehungswissenschaft, 3, 359-379.
19. Lesgold, A., Lajoie, S., Logan, D., & Eggan, G. (1990). Applying cognitive task analysis and research methods to assessment. In N. Frederiksen, R. Glaser, A. Lesgold, & M.G. Shafto (Eds.), Diagnostic monitoring of skill and knowledge acquisition (pp. 325-391). Hillsdale, NJ: Erlbaum.
20. Macaskill, G., Adams, R.J., & Wu, M.L. (1998). Scaling methodology and procedures for the mathematics and science literacy, advanced mathematics, and physics scales. In M.O. Martin & D.L. Kelly (Eds.), Third International Mathematics and Science Study. Technical report: Vol. III. Implementation and analysis: Final year of secondary school. (pp. 91-120). Chestnut Hill, MA: Boston College.
21. Martin, M.O., & Kelly, D.L. (Eds.). (1998). Third International Mathematics and Science Study. Technical report: Vol. III. Implementation and analysis: Final year of secondary school. Chestnut Hill, MA: Boston College.
22. Mullis, I.V.S., Martin, M.O., Beaton, A.E., Gonzalez, E.J., Kelly, D.L., & Smith, T.A. (1998). Mathematics and science achievement in the final year of secondary school. IEA's Third International Mathematics and Science Study. Chestnut Hill, MA: Boston College.
23. Neubrand, M., Biehler, R., Blum, W., Cohors-Fresenborg, E., Flade, L., Knoche, N., Lind, D., Löding, W., Möller, G., & Wynands, A. (2001). The German addition to the OECD-PISA mathematics assessment: Framework for the supplementary test and its connection to the international framework. International Reviews on Mathematical Education, 33, 45-59.
24. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute of Educational Research.
25. Renkl, A., & Helmke, A. (1992). Discriminant effects of performance-oriented and structure-oriented mathematics tasks on achievement growth. Contemporary Educational Psychology, 17, 47-55.
26. Reimann, P., & Chi, M.T.H. (1989). Human expertise. In K.J. Gilhooly (Ed.), Human and machine problem solving (pp. 161-191). New York: Plenum.
27. Resnick, L.B., & Ford, W.W. (1981). The psychology of mathematics for instruction. Hillsdale, NJ: Erlbaum.
28. Reusser, K. (1996). From cognitive modeling to the design of pedagogical tools. In S. Vosniadou, E. De Corte, R. Glaser, & H. Mandl (Eds.), International perspectives on the design of technology-supported learning environments (pp. 81-103). Mahwah, NJ: Erlbaum.
29. Shavelson, R.J., & Webb, N.M. (1991). Generalisability theory: Aprimer. Newbury Park, CA: Sage.
30. Stein, M.K., Grover, B.W., & Henningsen, M. (1996). Building student capacity for mathematical thinking and reasoning: An analysis of mathematical tasks used in reform classrooms. American Educational Research Journal, 32, 455-488.
31. Stern, E. (1998). Die Entwicklung des mathematischen Verständhisses im Kindesalter [The development of mathematical understanding during childhood]. Lengerich, Germany: Pabst Science Publishers.
32. Webb, N.M. (1979). Content and context variables in problem tasks. In G.A. Golding & C.E. McClintock (Eds.), Task variables in mathematical problem solving (pp. 69-102). Philadelphia, PA: Franklin Institute Press.
33. Williams, G., & Clarke, D. (1997). Mathematical task complexity and task selection. In D.M. Clarke, P. Clarkson, D. Gronn, M. Horne, L. Lowe, M. Mackinlay, & A. McDonough (Eds.), Mathematics. Imagine the possibilities (pp. 406-415). Branswick, Victoria: Mathematics Association of Victoria.
34. Wu, M.L. (1997). The development and application of a fit test for use with marginal estimation and generalized item response models. Unpublished Master's dissertation: University of Melbourne.
35. Wu, M.L., Adams, R.J., & Wilson, M. (1997). Conquest: Generalized item response modeling software - Manual. Melbourne: Australian Council for Educational Research.