Reading skills of students in different school tracks: Systematic (dis)advantages based on item formats in large scale assessments;Lesekompetenzen von Jugendlichen in unterschiedlichen Schulformen: Systematische Vor- und Nachteile bei verschiedenen Antwortformaten in Large-Scale Assessments

Zeitschrift fur Erziehungswissenschaft

Volumn 18, Issue 4, 2015, Pages 781-801

  Schwabe, Franziska ^a   McElvany, Nele ^a   Trendtel, Matthias ^b  

^a TU DORTMUND UNIVERSITY   (Germany)

^b Bundesinstitut für Bildungsforschung Innovation und Entwicklung des österreichischen Schulwesens ^*  (Austria)

Author keywords

Differential item functioning; Differential learning environments; Large scale assessments; Reading competence; School tracking

Indexed keywords

EID: 84945948431     PISSN: 1434663X     EISSN: 18625215     Source Type: Journal    
DOI: 10.1007/s11618-015-0645-3     Document Type: Article

References (64)

1. Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 3–23). Hillsdale: Lawrence Erlbaum Associates.
2. Ariga, K., & Brunello, G. (2007). Does secondary school tracking affect performance? Evidence from IALS, IZA Discussion Papers, No. 2643. http://nbn-resolving.de/urn:nbn:de:101:1–20080416197. Accessed: 4th April 2015.
3. Bacon, D. R. (2003). Assessing learning outcomes: A comparison of multiple choice and short answer questions in a marketing context. Journal of Marketing Education, 25, 31–36.
4. Bakker, M., van den Heuvel-Panhuizen, M., & Robitzsch, A. (2014). First-graders’ knowledge of multiplicative reasoning before formal instruction in this domain. Contemporary Educational Psychology, 39(1), 59–73.
5. Baumert, J., Stanat, P., & Watermann, R. (2006). Schulstruktur und die Entstehung differenzieller Lern- und Entwicklungsmilieus [School structure and the genesis of differential environments of learning and development]. In J. Baumert, P. Stanat & R. Watermann (Eds.), Herkunftsbedingte Disparitäten im Bildungswesen: Differenzielle Bildungsprozesse und Probleme der Verteilungsgerechtigkeit. Vertiefende Analysen im Rahmen von PISA 2000 (pp. 99–188). Wiesbaden: Springer VS.
6. Becker, M. (2009). Kognitive Leistungsentwicklung in differenziellen Lernumwelten: Effekte des gegliederten Sekundarschulsystems in Deutschland [Development of cognitive performance in differential environments of learning: Effects of the tracked system of secondary schools in Germany]. Berlin: Max-Planck-Institut für Bildungsforschung.
7. Becker, M., Lüdtke, O., Trautwein, U., Köller, O., & Baumert, J. (2012). The differential effects of school tracking on psychometric intelligence: Do academic-track schools make students smarter? Journal of Educational Psychology, 104(3), 682–699.
8. Becker, M., McElvany, N., Lüdtke, O., & Trautwein, U. (2014). Lesekompetenzen und schulische Lernumwelten: Besondere Fördereffekte des Frühübergangs in Gymnasien? [Reading skills and learning environment. Are there specific effects of early transition to academic tracks?]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 46(1), 35–50.
9. De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., & Tuerlinckx, F. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39(12), 1–28.
10. Bos, W., & Gröhlich, C. (eds.) (2010). KESS 8: Kompetenzen und Einstellungen von Schülerinnen und Schülern an Hamburger Schulen am Ende der Jahrgangsstufe 8 [KESS 8: Skills and attitudes of students in Hamburg’s schools at the end of grade 8]. Münster: Waxmann.
11. Bos, W., & Scharenberg, K. (2010). Lernentwicklung in leistungshomogenen und -heterogenen Schulklassen [Development of learning in classes with a homogenous and classes with a heterogenous achhievement level]. In W. Bos, E. Klieme, & O. Köller (eds.), Schulische Lerngelegenheiten und Kompetenzentwicklung: Festschrift für Jürgen Baumert (pp. 173–194). Münster: Waxmann.
12. Bos, W., Schwippert, K., & Stubbe, T. C. (2007). Die Koppelung von sozialer Herkunft und Schülerleistung im internationalen Vergleich [The connection between social background and achievement in international comparison]. In W. Bos, S. Hornberg, K.-H. Arnold, G. Faust, L. Fried, E.-M. Lankes, K. Schwippert, & R. Valtin (eds.), IGLU 2006. Lesekompetenzen von Grundschulkindern in Deutschland im internationalen Vergleich (pp. 225–247). Münster: Waxmann.
13. Brenner, A. D., & Crosnoe, R. (2011). The racial/ethnic composition of elementary schools and young children’s academic and socioemotional functioning. American Educational Research Journal, 48(3), 621–646.
14. van Buuren, S., & Groothuis-Oudshoorn, K. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67.
15. Connor, C. M., Morrison, F. J., Fishman, B., Giuliani, S., Luck, M., & Underwood, P. S. (2011). Testing the impact of child characteristics × instruction: Interactions on third graders’ reading comprehension by differentiating literacy instruction. Reading Research Quarterly, 46(3), 189–221.
16. DESI-Konsortium (eds.) (2008). Unterricht und Kompetenzerwerb in Deutsch und Englisch. Ergebnisse der DESI-Studie [Teaching and competency acquisition in German and English. Results of the DESI study]. Weinheim: Beltz.
17. van Ewijk, R., & Sleegers, P. (2010). The effect of peer socioeconomic status on student achievement: A meta-analysis. Educational Research Review, 5, 134–150.
18. Ferne, T., & Rupp, A. A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4(2), 113–148.
19. Grisay, A., & Monseur, C. (2007). Measuring the equivalence of item difficulty in the various versions of an international test. Studies in Educational Evaluation, 33, 69–86.
20. Guthrie, J. T., & Wigfield, A. (2005). Roles of motivation and engagement in reading comprehension assessment. In S. G. Paris & S. A. Stahl (eds.), Children’s reading comprehension and assessment (pp. 187–214). Mahwah: Lawrence Erlbaum.
21. Haag, N., Heppt, B., Stanat, P., Kuhl, P., & Pant, H. A. (2013). Second language learners’ performance in mathematics: Disentangling the effects of academic language features. Learning and Instruction, 28, 24–34.
22. Haberkorn, K., Pohl, S., Hardt, K., & Wiegand, E. (2012). NEPS technical report for reading—Scaling results of starting cohort 4 in ninth Grade (NEPS Working Paper No. 16). Bamberg: Otto-Friedrich-Universität (Nationales Bildungspanel).
23. Hadfield, J. D. (2010). MCMC methods for multi-response generalized linear mixed models: The MCMCglmm R package. Journal of Statistical Software, 33(2), 1–22.
24. Haladyna, T. M., & Rodriguez, M. C. (2013). Developing and validating test items. New York: Routledge.
25. Hanushek, E. A., & Wößmann, L. (2006). Does educational tracking affect performance and inequality? Differences-in-differences evidence across countries. The Economic Journal, 116, C63–C76.
26. Kiefer, T., Robitzsch, A., & Wu, M. (2013). TAM: test analysis modules. R package version 0.7–35.
27. Kleickmann, T., Richter, D., Kunter, M., Elsner, J., Besser, M., & Krauss, S. (2013). Teachers’ content knowledge and pedagogical content knowledge: The role of structural differences in teacher education. Journal of Teacher Education, 64(1), 90–106.
28. Klicpera, C., & Gasteiger-Klicpera, B. (1993). Lesen und Schreiben: Entwicklung und Schwierigkeiten [Reading and writing: Development and difficulties]. Bern: Huber.
29. Klieme, E., Artelt, C., Hartig, J., Jude, N., Köller, O., Prenzel, M., & Stanat, P. (eds.) (2010). PISA 2009. Bilanz nach einem Jahrzehnt [PISA 2009: Result after one decade]. Münster: Waxmann.
30. KMK (2014) = Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland. (2014). Basic structure of the education system in the Federal Republic of Germany. http://www.kmk.org/fileadmin/pdf/Bildung/AllgBildung/Schema-Bildungsgaenge_und_Schularten-Stand_2014.pdf. Accessed: 4th April 2015.
31. Köller, O., Schütte, K., Zimmermann, F., Retelsdorf, J., & Leucht, M. (2013). Starke Klasse, hohe Leistungen? Die Rolle der Leistungsstärke der Klasse für die individuellen Mathematik- und Leseleistungen in der Sekundarstufe I [Strong class. high performances? The role of class performance with regard to individual mathematic and reading performances at secondary level I]. Psychologie in Erziehung und Unterricht, 60, 184–197.
32. Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820.
33. Lafontaine, D., & Monseur, C. (2006). Impact of test characteristics on gender equity indicators in the assessment of reading comprehension. Liège: University of Liège.
34. Lafontaine, D., & Monseur, C. (2009). Gender gap in comparative studies of reading comprehension: To what extent do the test characteristics make a difference? European Educational Research Journal, 8(1), 69–79.
35. Luyten, H., Cremers-van Wees, L. M., & Bosker, R. J. (2003). The Matthew effect in Dutch primary education: Differences between schools, cohorts, and pupils. Research Papers in Education, 18, 167–195.
36. Maaz, K., Trautwein, U., Lüdtke, O., & Baumert, J. (2008). Educational transitions and differential learning environments: How explicit between-school tracking contributes to social inequality in educational outcome. Child Development Perspectives, 2(2), 99–106.
37. McElvany, N., & Schwabe, F. (2013). Fairness von Lesetestaufgaben für Kinder aus Familien mit unterschiedlichem sozioökonomischem Status bei Large-Scale-Studien [Fariness of large scale reading test items for children with different social backgrounds]. In N. McElvany & H. G. Holtappels (eds.), Empirische Bildungsforschung—Theorien, Methoden, Befunde und Perspektiven. Festschrift für Wilfried Bos (pp. 219–234). Münster: Waxmann.
38. Oakes, J. (1982). The reproduction of inequity: The content of secondary school tracking. The Urban Review, 14(2), 107–114.
39. OECD (2010a) = (Organization for Economic Co-operation and Development). (2010a). PISA 2009 results: What students know and can do—student performance in reading, mathematics and science (Volume I). http://dx.doi.org/ö.pl010.1787/9789264091450-en. Accessed: 4th April 2015.
40. OECD (2010b) = (Organization for Economic Co-operation and Development). (2010b). PISA 2009 Assessment framework: Key competencies in reading, mathematics and science. Pisa: OECD Publishing.
41. OECD (2012) = (Organization for Economic Co-operation and Development). (2012). PISA 2009 Technical Report. PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264167872-en. Accessed: 4th April 2015.
42. Pae, T.-I. (2004). DIF for examinees with different academic backgrounds. Language Testing, 21(1), 53–73.
43. Pfost, M., Karing, C., Lorenz, C., & Artelt, C. (2010). Schereneffekte im ein- und mehrgliedrigen Schulsystem: Differentielle Entwicklung sprachlicher Kompetenzen am Übergang von der Grund- in die weiterführende Schule [Fan spread effects in a tracked and a nontracked school system. Is there evidence for differential linguistic competence development at the transition from primary to secondary school?]. Zeitschrift für Pädagogische Psychologie, 24, 259–273.
44. Pfost, M., Dörfler, T., & Artelt, C. (2012). Reading competence development of poor readers in a German elementary school sample: An empirical examination of the Matthew effect model. Journal of Research in Reading, 35(4), 411–426.
45. R Core Team. (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. http://www.R-project.org/. Accessed: 4th April 2015.
46. Rauch, D., & Hartig, J. (2010). Multiple-choice versus open-ended response formats of reading test items: A two-dimensional IRT analysis. Psychological Test and Assessment Modeling, 52(4), 354–379.
47. Retelsdorf, J., Becker, M., Köller, O., & Möller, J. (2012). Reading development in a tracked school system: A longitudinal study over 3 years using propensity score matching. British Journal of Educational Psychology, 82, 647–671.
48. Rjosk, C., Richter, D., Hochweber, J., Lüdtke, O., Klieme, E., & Stanat, P. (2014). Socioeconomic and language minority classroom composition and individual reading achievement: The mediating role of instructional quality. Learning and Instruction, 32, 63–72.
49. Rodriguez, M. C. (2002). Choosing an item format. In G. Tindal & T. M. Haladyna (eds.), Large scale assessment programs for all students: Validity, technical adequacy, and implementation (pp. 213–231). Mahwah: Lawrence Erlbaum.
50. Routitsky, A., & Turner, R. (2003). Item format types and their influences on cross-national comparisons of student performance. Chicago: Paper presented at the annual meeting of the American Educational Research Association.
51. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.
52. Schroeder, S., & Tiffin-Richards, S. P. (2014). Kognitive Verarbeitung von Leseverständnisitems mit und ohne Text [Cognitive handling of reading comprehension test items with and without text]. Zeitschrift für Pädagogische Psychologie, 28(1–2), 21–30.
53. Schwabe, F., McElvany, N., & Trendtel, M. (2015). The school age gender gap in reading achievement: Examining the influences of item format and intrinsic reading motivation. Reading Research Quarterly, published online first, 1–14.
54. Schwippert, K., Bos, W., & Lankes, E. M. (2004). Lesen Mädchen anders? Vertiefende Analysen zu Geschlechtsdifferenzen auf Basis der Internationalen Grundschul-Lese-Untersuchung IGLU [Do girls read differently? In-depth analyses of gender differences based on the Progress in International Reading Literacy Study PIRLS]. Zeitschrift für Erziehungswissenschaft, 7(2), 219–234.
55. Simkin, M. G., & Kuechler, W. L. (2005). Multiple-choice tests and student understanding: What is the connection? Decision Sciences Journal of Innovative Education, 3(1), 73–97.
56. Snijders, T. A. B., & Bosker, R. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). London: Sage Publishers.
57. Snow, C. E. (2002). Reading for understanding: Toward a research and development program in reading comprehension. Santa Monica: RAND.
58. Snow, C. E., Burns, M. S., & Griffin, P. (eds.) (1998). Preventing reading difficulties in young children. Washington: National Academy Press.
59. Solheim, O. J. (2011). Impact of reading self-efficacy and task value on reading comprehension scores in different item formats. Reading Psychology, 32, 1–27.
60. Stanat, P., Rauch, D., & Segeritz, M. (2010). Schülerinnen und Schüler mit Migrationshintergrund [Students with immigrant backgrounds]. In E. Klieme, C. Artelt, J. Hartig, N. Jude, O. Köller, M. Prenzel, W. Schneider, & P. Stanat (eds.), PISA 2009. Bilanz nach einem Jahrzehnt (pp. 200–230). Münster: Waxmann.
61. Steinert, B., Hartig, J., & Klieme, E. (2008). Institutionelle Bedingungen der Sprachkompetenz [Institutional factors impact on language proficiency]. In E. Klieme (ed.), Unterricht und Kompetenzerwerb in Deutsch und Englisch (pp. 411–448). Weinheim: Beltz.
62. Taylor, C. S., & Lee, Y. (2012). Gender DIF in reading and mathematics tests with mixed item formats. Applied Measurement in Education, 25(3), 246–280.
63. Wainer, H., & Thissen, D. (1993). Combining multiple-choice and constructed response test scores: Toward a marxist theory of test construction. Applied Measurement in Education, 6(2), 103–118.
64. Walzebug, A. (2014). Is there a language-based social disadvantage in solving mathematical items? Learning, Culture and Social Interaction, 3(2), 159–169.