Rater reliability in language assessment: The bug of all bears

System

Volumn 28, Issue 1, 2000, Pages 31-53

  Gamaroff, Raphael ^a  

^a UNIVERSITY OF FORT HARE   (South Africa)

Author keywords

Equivalent judgements; Equivalent scores; Errors; Interrater reliability; Levels of proficiency; Moderation; Reliability; Subjectivity; Validity

Indexed keywords

EID: 0042852125     PISSN: 0346251X     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0346-251X(99)00059-7     Document Type: Article

References (35)

1. Aitchison J. Words in the Mind: An introduction to the Mental Lexicon. 1987;Blackwell, Oxford.
2. Alderson, J.C., 1981. Report of the discussion on Communicative Language Testing. In: Alderson, J.C., Hughes, A. (Eds.), Issues in Language Testing: ELT Documents III. The British Council, London.
3. Alderson J.C. Who needs jam? Hughes A., Porter D. Current Developments in Language Testing. 1983;Academic Press, London.
4. Alderson J.C., Clapham C. Applied linguistics and language testing: a case study of the ELTS test. Applied Linguistics. 13:1992;149-167.
5. Alderson J.C., Clapham C., Wall D. Language test construction and evaluation. 1995;Cambridge University Press, Cambridge.
6. Bachman L.F., Clark J.L.D. The measurement of foreign/second language proficiency. American Academy of the Political and Social Science Annals. 490:1987;20-33.
7. Bereiter C., Scardemalia M. Does learning to write have to be so difficult? Freedman A., Pringle I., Yalden J. Learning to Write: First Language/Second Language. 1983;Longman, London.
8. Bley-Vroman R. The logical problem of foreign language learning. Linguistic Analysis. 20:1990;3-49.
9. Bock M. Teaching grammar in context. Angélil-Carter S. Access to Success: Literacy in Academic Contexts. 1998;University of Cape Town Press, Cape Town.
10. Bradbury, J., Damerell, C., Jackson, F., Searle, R., 1990. ESL issues arising from the "Teach-test-teach" programme. In: Chick, K. (Ed.), Searching for Relevance: Contextual Issues in Applied Linguistics. South African Applied Linguistics Association (SAALA), Johannesburg.
11. Brown A. The effect of rater variables in the development of an occupation-specific language performance test. Language Testing. 12:1995;1-15.
12. Coe R.M. An apology for form; or, who took the form out of the process. College English. 49:1987;13-28.
13. Cziko G.A. Improving the psychometric, criterion-referenced, and practical qualities of integrative testing. TESOL Quarterly. 16:1982;367-379.
14. Cziko G.A. Some problems with emprically-based models of communicative competence. Applied Linguistics. 5:1984;23-37.
15. Davies A. Principles of Language Testing. 1990;Blackwell Ltd, Oxford.
16. Douglas D. Developments in language testing. Annual Review of Applied Linguistics. 15:1995;167-187.
17. Dreyer C. Teacher-student style wars in South Africa: the silent battle. System. 26:1998;115-126.
18. Ebel R.L., Frisbie D.A. Essentials of Educational Measurement, 5th edition. 1991;Prentice Hall, Englewood Cliffs, NJ.
19. Gamaroff, R. 1996., Workshop on quantitative measurement in language testing. National Association of Educators of Teachers of English (NAETE) Conference, East London Teacher's Centre, South Africa, September.
20. Gamaroff R. "Old paradigm" language proficiency tests as predictors of longterm academic achievement. Per Linguam. 13:1997;1-22.
21. Gamaroff R. Language, content and skills in the testing of English for academic purposes. South African Journal of Higher Education. 12:1998;109-116.
22. Gamaroff R. Cloze tests as predictors of global language proficiency: a statistical analysis. South African Journal of Linguistics. 16:1998;7-15.
23. Hartog P., Rhodes E.C. The Marks of Examiners. 1936;Macmillan, New York.
24. Ingram E. Assessing proficiency: an overview on some aspects of testing. Hyltenstam K., Pienemann M. Modelling and Assessing Second Language Acquisition. 1985;Multilingual Matters Ltd, Clevedon, Avon.
25. Kaczmarek C.M. Scoring and rating essay tasks. Oller J.W. Jr., Perkins K. Research in Language Testing. 1980;Newbury House, Rowley, Massachusetts.
26. Lado R. Language Testing. 1961;McGraw-Hill, New York.
27. Moore R. How science educators construe student writing. Angélil-Carter S. Access to Success: Literacy in Academic Contexts. 1998;University of Cape Town Press, Cape Town.
28. Moss P. Can there be validity without reliability? Educational Researcher. 23:1994;5-12.
29. Oller J.W. Jr. Language Tests at School. 1979;Longman, London.
30. Oxford R.L., Ehrman M., Lavine R.Z. Style wars: teacher-student style conflicts in the language classroom. Magnan S.S. Challenges in the 1990s for College Foreign Language Programs. 1991;Heinle and Heinle, Boston, MA.
31. Pilliner A.E.G. Subjective and objective testing. Davies A. Language Testing Symposium. 1968;Oxford University Press, London.
32. Santos T. Professors' reactions to the academic writing of nonnative-speaking students. TESOL Quarterly. 22:1988;69-90.
33. Quine W., Ullian J. The Web of Belief. 1970;Random House, New York.
34. Vollmer H.J. The structure of foreign language competence. Hughes A., Porter D. Current Developments in Language Testing. 1983;Academic Press, London.
35. Widdowson H.G. New starts and different kinds of failure. Freedman A., Pringle I., Yalden J. Learning to Write: First Language/Second Language. 1983;Longman, London.