Educational measurement issues and implications of high stakes decision making in final examinations in secondary education in the Netherlands

Assessment in Education: Principles, Policy and Practice

Volumn 19, Issue 1, 2012, Pages 117-136

  van Rijn, P W ^a   Beguin A A ^b   Verstralen, H H F M ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b Cito group   (Netherlands)

Author keywords

classical test theory; high stakes testing; measurement precision; multiple tests

Indexed keywords

EID: 84858313662     PISSN: 0969594X     EISSN: 1465329X     Source Type: Journal    
DOI: 10.1080/0969594X.2011.591289     Document Type: Article

References (36)

1. Alberts, R.V.J. 2001. Equating exams as a prerequisite for maintaining standards: Experience with Dutch centralised secondary examinations. Assessment in Education: Principles, Policy & Practice, 8(3): 353-367.
2. Alberts, R.V.J., and B.J.M. Erens. 2010. Verslag van de examencampagne 2010 voortgezet onderwijs [Report of secondary education examinations 2010]. http://www.cito.nl/vo/ce/verslag/Cito_verslag_examencampagne_2010.pdf (accessed September 2, 2010).
3. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 1999. Standards for educational and psychological testing, Washington, DC: Author.
4. Bock, R.D. and Wood, R. 1971. Test theory. Annual Review of Psychology, 22: 193-224.
5. Borsboom, D., Romeijn, J.W. and Wicherts, J.M. 2008. Measurement invariance versus selection invariance: Is fair selection possible?. Psychological Methods, 13(2): 75-98.
6. Brennan, R.L., ed. 2006. Educational measurement, 4th ed., Westport, CT: American Council on Education/Praeger.
7. Buonaccorsi, J.P. 2010. Measurement error: Models, methods and applications, Boca Raton, FL: Chapman & Hall.
8. Campbell, D.T. 1976. Assessing the impact of planned social change. Dartmouth College Occasional Paper Series, No. 8. http://www.wmich.edu/evalctr/pubs/ops/ops08.pdf (accessed February 13, 2008).
9. Chester, M.D. 2003. Multiple measures and high-stakes decisions: A framework for combining measures. Educational Measurement: Issues and Practice, 22(2): 32-41.
10. Cronbach, L.J. 1951. Coefficient alpha and the internal structure of tests. Psychometrika, 16(3): 297-334.
11. Douglas, K.M. and Mislevy, R.J. 2010. Estimating classification accuracy for complex decision rules based on multiple scores. Journal of Educational and Behavioral Statistics, 35(3): 280-306.
12. Feldt, L.S., Steffen, M. and Gupta, N.C. 1985. A comparison of five methods for estimating the standard error of measurement at specific score levels. Applied Psychological Measurement, 9(4): 351-361.
13. Gulliksen, H. 1950. Theory of mental tests, New York: Wiley.
14. Haertel, E.H. 2006. "Reliability". In Educational measurement, 4th ed., Edited by: Brennan, R.L. 65-110. Westport, CT: American Council on Education/Praeger.
15. He, Q. 2010. Estimating the reliability of composite scores. Ofqual report 2010-02-01. Ofqual. http://www.ofqual.gov.uk/files/2010-02-01-composite-reliability.pdf (accessed January 21, 2011).
16. Heuvelmans, T. 2010. TiaPlus: Computer program and users' manual. Arnhem, The Netherlands: Cito. http://tiaplus.cito.nl/ (accessed September 2, 2010).
17. Kane, M.T. 2008. Errors of measurement, theory, and public policy. Twelfth annual William H. Angoff lecture presented at Educational Testing Service, November 18, in Princeton, NJ.
18. Kolen, M.J. 1988. Defining score scales in relation to measurement error. Journal of Educational Measurement, 25(2): 97-110.
19. Kolen, M.J. and Brennan, R.L. 2004. Test equating, scaling, and linking, 2nd ed., New York: Springer-Verlag.
20. Lee, W. 2010. Classification consistency and accuracy for complex assessments using item response theory. Journal of Educational Measurement, 47(1): 1-17.
21. Lee, W., Hanson, B.A. and Brennan, R.L. 2002. Estimating consistency and accuracy indices for multiple classifications. Applied Psychological Measurement, 26(4): 412-32.
22. Lord, F.M. 1962. Cutting scores and errors of measurement. Psychometrika, 27(1): 19-30.
23. Lord, F.M. and Novick, M.R. 1968. Statistical theories of mental test scores, Reading, MA: Addison-Wesley.
24. Meyer, J.P. 2010. Reliability, Oxford: Oxford University Press.
25. Onderwijsraad. 2007. Versteviging van kennis in het onderwijs II [Strengthening knowledge in Education II]. http://www.onderwijsraad.nl/upload/publicaties/299/documenten/versteviging_van_kennis_in_het_onderwijs_ii.pdf (accessed September 2, 2010).
26. Profielcommissie Natuur en Techniek/Natuur en Gezondheid and Profielcommissie Economie en Maatschappij/Cultuur en Maatschappij. 2007. Bruggen tussen Natuur en Maatschappij [Bridges between nature and society]. http://www.usgvox.nl/pdf/profielcommissie%20bruggen%20bouwen%20161206.pdf (accessed September 2, 2010).
27. R Development Core Team. 2011. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
28. Rekers-Mombarg, L.T.M., and G.J. Harms. 2008. Double standards? The discrepancy between grades on the school examination and central examination in secondary education of immigrant students [Meten met twee maten? De discrepantie tussen de cijfers op het schoolexamen en het centraal examen VO van allochtone leerlingen]. Groningen, The Netherlands: Gion.
29. Rudner, L. 2005. Expected classification accuracy. Practical Assessment Research & Evaluation 10, no. 13. http://pareonline.net/pdf/v10n13.pdf.
30. Sijtsma, K. 2009. On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74(1): 107-120.
31. Stalnaker, J.M. and Richardson, M.W. 1933. A note on the combination of test scores. Journal of General Psychology, 8(2): 460-463.
32. Tarkkonen, L. and Vehkalahti, K. 2005. Measurement errors in multivariate measurement scales. Journal of Multivariate Analysis, 96(1): 172-189.
33. Van der Linden, W.J. and Mellenbergh, G.J. 1977. Optimal cutting scores using a linear loss function. Applied Psychological Measurement, 1(4): 593-599.
34. Van Rijn, P.W., Béguin, A.A. and Verstralen, H.H.F.M. 2009. Failing or passing? The accuracy of examination results in secondary education [Zakken of slagen? De nauwkeurigheid van examenuitslagen in het voortgezet onderwijs]. Pedagogische Studiën, 86(3): 185-195.
35. Verhelst, N.D. 1998. Estimating the reliability of a test from a single test administration. Measurement and Research Department Reports, 98-02. Arnhem, The Netherlands: Cito.
36. Villasenor-Alva, J.A. and Gonzalez-Estrada, E. 2009. A generalisation of Shapiro-Wilk's test for multivariate normality. Communications in Statistics: Theory and Methods, 38(11): 1870-1883.