Trusting teachers’ judgement: Research evidence of the reliability and validity of teachers’ assessment used for summative purposes

Research Papers in Education

Volumn 20, Issue 3, 2005, Pages 245-270

  Harlen, Wynne ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Dependability; Reliability; Summative assessment; Teachers assessment; Validity

Indexed keywords

EID: 85015764904     PISSN: 02671522     EISSN: 14701146     Source Type: Journal    
DOI: 10.1080/02671520500193744     Document Type: Article

References (63)

1. Abbott, D., Broadfoot, P., Croll, P., Osborn, M. & Pollard, A. (1994) Some sink, some float: national curriculum assessment and accountability, British Educational Research Journal, 20, 155-174.
2. Black, H. D. (1986) Assessment for learning, in: D. L. Nuttall (Ed.) Assessing educational achievement (London, Falmer Press).
3. Black, P. (1993) Formative and summative assessment by teachers, Studies in Science Education, 21, 49-97.
4. Black, P. (1998) Testing: friend or foe? (London, Falmer Press).
5. Bennett, R. E., Gottesman, R. L., Rock, D. A. & Cerullo, F. (1993) Influence of behaviour perceptions and gender on teachers’ judgements of students’ academic skill, Journal of Educational Psychology, 85, 347-356.
6. Broadfoot, P., Murphy, R. & Torrance, H (Eds) (1990) Changing educational assessment: international perspectives and trends (London, Routledge).
7. Brown, C. R., Moor, J. L., Silkstone, B. E. & Botton, C. (1998) An evaluation of two different methods of assessing independent investigations in an operational pre-university level examination in biology in England, Studies in Educational Evaluation, 24, 87-98.
8. Brown, C. R., Moor, J. L., Silkstone, B. E. & Botton, C. (1996) The construct validity and context dependency of teacher assessment of practical skills in some pre-university level science examinations, Assessment in Education, 3, 377-391.
9. Choi, C. C. (1999) Public Examinations in Hong Kong, Assessment in Education, 6(3), 405-418.
10. Coladarci, T. (1986) Accuracy of teachers’ judgements of students responses to standardized test items, Journal of Educational Psychology, 78, 141-146.
11. Crawford, L., Tindal, G. & Steiber, S. (2001) Using oral reading rate to predict student performance on statewide achievement tests, Educational Assessment, 7, 303-323.
12. Crooks, T. J. (1988) The impact of classroom evaluation practices on students, Review of Educational Research, 58, 438-481.
13. Delap, M. R. (1994) An investigation into the accuracy of A level predicted grades, Educational Research, 26, 135-149.
14. Delap, M. R. (1995) Teachers’ estimates of candidates’ performance in public examinations, Assessment in Education, 2, 75-92.
15. DES (1987) Task Group on Assessment and Testing (TGAT): a report (London, Department of Education and Science and Welsh Office).
16. Donnelly, J. F., Buchan, A.S., Jenkins, E. W. & Welford, A. G. (1993) Policy, practice and teachers’ professional judgement: the internal assessment of practical work in GCSE science (Driffield, Nafferton Books).
17. Ecclestone, K. (2004) Learning in the comfort zone: cultural and social capital inside an outcome-based assessment regime, Assessment in Education, 11(1), 29-47.
18. EPPI-Centre (2002a) Core keywording strategy: data collection for a register of educational research version 0.9.7 (London, EPPI-Centre, Social Science Research Unit).
19. EPPI-Centre (2002b) Review guidelines for extracting data and quality assessing primary studies in educational research. Version 0.9.7. (London, EPPI-Centre, Social Science Research Unit).
20. Frederiksen, J. & White, B. (2004), Designing assessment for instruction and accountability: an application of validity theory to assessing scientific inquiry, in: M. Wilson (Ed.) Towards coherence between classroom assessment and accountability, 103rd Yearbook of the National Society for the Study of Education Part II (Chicago, National Society for the Study of Education), 74-104.
21. Gipps, C. (1994) Beyond testing (London, Falmer Press).
22. Gipps, C., McCallum, B. & Brown, M. (1996) Models of teacher assessment among primary school teachers in England, The Curriculum Journal, 7, 167-183.
23. Good, F. J. (1988a) Differences in marks awarded as a result of moderation: some findings from a teachers assessed oral examination in French, Educational Review, 40, 319-331.
24. Good, F. J. (1988b) A method of moderation of school-based assessments: some statistical considerations, The Statistician, 37, 33-49.
25. Good, F. J. & Cresswell, M. (1988) Can teachers enter candidates appropriately for examinations involving differentiated papers? Educational Studies, 14, 289-297.
26. Hall, K. & Harding, A. (2002) Level descriptions and teacher assessment in England: towards a community of assessment practice, Educational Research, 44, 1-15.
27. Hall, K., Webber, B., Varley, S., Young, V. & Dorman, P. (1997) A study of teacher assessment at Key Stage 1, Cambridge Journal of Education, 27, 107-122.
28. Hargreaves, D. J., Galton, M. J. & Robinson, S. (1996) Teachers’ assessments of primary children’s classroom work in the creative arts, Educational Research, 38, 199-211.
29. Harlen, W. (Ed.) (1994) Enhancing quality in assessment (London, Paul Chapman).
30. Harlen, W. (1995) Standards and science education in Scottish schools, Studies in Science Education, 26, 107-134.
31. Harlen, W. (2000) Teaching, learning and assessing science 5-12 (London, Paul Chapman).
32. Harlen, W. (2004a) A systematic review of the evidence of the reliability and validity of assessment by teachers for summative purposes, in: (EPPI-Centre Review) Research evidence in education library, 3 (London, EPPI-Centre, Social Science Research Unit). Available online at: http:// eppi.ioe.ac.uk/EPPIWEB/Home.aspx?page=/rael/review_groups/assessment/review_three.htm
33. Harlen, W. (2004b) A systematic review of the evidence of the impact on students, teachers and the curriculum of the process of using assessment by teachers for summative purposes, in: (EPPI-Centre Review) Research Evidence in Education Library, 4 (London, EPPI-Centre, Social Science Research Unit).
34. Harlen, W. & Deakin Crick, R. (2002) A systematic review of the impact of summative assessment and tests on students’ motivation for learning, in: (EPPI-Centre Review) Research evidence in education library, 1 (London, EPPI-Centre, Social Science Research Unit).
35. Hoge, R. D. & Butcher, R. (1984) Analysis of teacher judgements of pupil achievement levels, Journal of Educational Psychology, 76(5), 777-781.
36. Hoge, R. D. & Coladarci, T. (1989) Teacher-based judgements of academic achievement: a review of literature, Review of Educational Research, 59(3), 297-313.
37. Hopkins, K. D., George, C. A. & Williams, D. D. (1985) The concurrent validity of standardized achievement tests by content area using teachers’ ratings as criteria, Journal of Educational Measurement, 22, 177-182.
38. James, M. (1998) Using assessment for school improvement (Oxford, Heinemann Educational).
39. Klein, S. P., McCaffrey, B. & Koretz, D. (1995) The reliability of mathematics portfolio scores: lessons from the Vermont experience, Applied Measurement in Education, 8(3), 243-260.
40. Koretz, D. (1998) Large-scale portfolio assessment in the US: evidence pertaining to the quality of measurement, Assessment in Education, 5(3), 309-334.
41. Koretz, D., Stecher, B. M., Klein, S. P. & McCaffrey, D. (1994) The Vermont Portfolio Assessment Program: findings and implications, Educational Measurement: Issues & Practice, 13, 5-16.
42. Leonard, M. & Davey C. (2001) Thoughts on the 11 plus (Belfast, Save the Students Fund).
43. Levine, M. G., Haus, G. J. & Cort, D. (1987) The accuracy of teacher judgement of the oral proficiency of high school foreign language students, Foreign Language Annals, 20, 45-50.
44. Lubisi, R. C. & Murphy, R. J. L. (2002) Assessment in south African schools, Assessment in Education, 9(2), 255-268.
45. Massey, A., Green, S., Dexter, T. & Hamnett, L. (2003) Comparability of national tests over time: key stage test standards between 1996 and 2001 (London, Qualifications & Curriculum Authority).
46. Masters, G. N. & Forster, M. N. (1995) ARK guides to developmental assessment (Camberwell, The Australian Council for Educational Research).
47. Maxwell, G. (2004) Progressive assessment for learning and certification: some lessons from school-based assessment in Queensland, paper presented at The Third Conference of the Association of Commonwealth Examination and Assessment Boards, Nadi, Fiji, March.
48. Meisels, S. J., Bickel, D. D., Nicholson, J., Xue, Y. & Atkins-Burnett, S. (2001) Trusting teachers’ judgements: a validity study of a curriculum-embedded performance assessment in kindergarten to Grade 3, American Educational Research Journal, 38, 73-95.
49. National Research Council (NRC) (2001) Knowing what students know. The science and design of educational assessment (Washington, DC, National Academy Press).
50. Radnor, H. A. (1996) Evaluation of Key Stage 3 assessment arrangements for 1995. Final report (University of Exeter, Exeter).
51. Reeves, D. J., Boyle, W. F. & Christie, T. (2001) The relationship between teacher assessment and pupil attainments in standard test/tasks at Key Stage 2, 1996-1998, British Educational Research Journal, 27, 141-160.
52. Rowe, K. J. & Hill, P. W. (1996) Assessing, recording and reporting students’ educational progress: the case for ‘subject profiles’, Assessment in Education, 3, 309-352.
53. Satterley, D. (1994) The quality of external assessment, in: W. Harlen (Ed.) Enhancing quality in assessment (London, Paul Chapmen).
54. Shapley, K. S. & Bush, M. J. (1999) Developing a valid and reliable portfolio assessment in the primary grades: building on practical experience, Applied Measurement in Education, 12(2), 111-132.
55. Sharpley, C. F. & Edgar, E. (1986) Teachers’ ratings vs standardized tests: an empirical investigation of agreement between two indices of achievement, Psychology in the Schools, 23, 106-111.
56. Shavelson, R. J., Baxter, G. P. & Pine, J. (1992) Performance assessments: political rhetoric and measurement reality, Educational Researcher, 21, 22-27.
57. Shorrocks, D., Daniels, S., Staintone, R. & Ring, K. (1993) Testing and assessing 6- and 7-year-olds, The evaluation of the 1992 Key Stage 1 National Curriculum Assessment (National Union of Teachers).
58. Spear, M. G. (1984) Sex bias in science teachers’ ratings of work and pupil characteristics, European Journal of Science Education, 6(4), 368-377.
59. Thomas, S., Madaus, G. F., Raczek, A. E. & Smees, R. (1998) Comparing teacher assessment and the standard task results in England: the relationship between pupil characteristics and attainment, Assessment in Education, 5, 213-246.
60. Wiliam, D. (1993) Reconceptualising validity, dependability and reliability for National Curriculum assessment, paper given at the BERA Conference, Liverpool.
61. Wiliam, D. (1995) It’ll all end in tiers, British Journal of Curriculum and Assessment, 3, 21-24.
62. Wilson, J. & Wright, C. R. (1993) The predictive validity of student self-evaluations, teachers’ assessments, and grades for performance on the verbal reasoning and numerical ability scales of the differential aptitude test for a sample of secondary school students attending rural Appalachia schools, Educational & Psychological Measurement, 53, 259-270.
63. Wood, R. (1991) Assessment and testing: a survey of research (Cambridge, University Press).