Diagnostic assessment of writing: A comparison of two rating scales

Language Testing

Volumn 26, Issue 2, 2009, Pages 275-304

  Knoch, Ute ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

Diagnostic writing assessment; Rating scale development; Rating scale validation; Rating scales; Second language writing; Second language writing assessment

Indexed keywords

EID: 68149158452     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1177/0265532208101008     Document Type: Article

References (37)

1. Alderson, C. (2005). Diagnosing foreign language proficiency. The interface between learning and assessment. London: Continuum.
2. Bachman, L. F. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press.
3. Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford: Oxford University Press.
4. Brindley, G. (1998). Describing language development? Rating scales and SLA. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 112-140). Cambridge: Cambridge University Press.
5. Creswell, J., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
6. Cumming, A., Kantor, R., & Powers, D. E. (2001). Scoring TOEFL essays and TOEFL 2000 prototype writing tasks: An investigation into raters' decision making and development of a preliminary analytic framework. TOEFL Monograph Series 22. Princeton, NJ: Educational Testing Service.
7. Davies, A., & Elder, C. (2005). Validity and validation in language testing. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning. Mahwah, NJ: Lawrence Erlbaum.
8. Elder, C. (2003). The DELNA initiative at the University of Auckland. TESOLANZ Newsletter, 12(1), 15-16.
9. Elder, C., Barkhuizen, G., Knoch, U., & von Randow, J. (2007). Evaluating rater responses to an online rater training program. Language Testing, 24(1), 37-64.
10. Elder, C., & Von Randow, J. (2008). Exploring the utility of a web-based English language screening tool. Language Assessment Quarterly, 5(3), 173-194.
11. Fulcher, G. (1996). Does thick description lead to smart tests? A databased approach to rating scale construction. Language Testing, 13(2), 208-238.
12. Fulcher, G. (2003). Testing second language speaking. London: Longman/Pearson Education.
13. Grabe, W., & Kaplan, R. B. (1996). Theory and practice of writing. New York: Longman.
14. Hamp-Lyons, L. (1991). Scoring procedures for ESL contexts. In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts. Norwood, NJ: Ablex.
15. Jolliffe, I. T. (1986). Principal component analysis. New York: Springer-Verlag.
16. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141-151.
17. Knoch, U. (2007a). Diagnostic writing assessment: The development and validation of a rating scale. Unpublished PhD, University of Auckland.
18. Knoch, U. (2007b). Do empirically developed rating scales function differently to conventional rating scales for academic writing? Spaan Fellow Working Papers in Second or Foreign Language Assessment, 5, 1-36.
19. Knoch, U. (2007c). 'Little coherence, considerable strain for reader': A comparison between two rating scales for the assessment of coherence. Assessing Writing, 12(2), 108-128.
20. Knoch, U. (2008). The assessment of academic style in EAP writing: The case of the rating scale. Melbourne Papers in Language Testing, 13(1), 34-67.
21. Knoch, U., Read, J., & von Randow, J. (2007). Re-training writing raters online: How does it compare with face-to-face training? Assessing Writing, 12, 26-43.
22. Linacre, J. M. (2006). Facets Rasch measurement computer program. Chicago, IL: Winsteps.
23. Lumley, T. (2002). Assessment criteria in a large-scale writing test: What do they really mean to the raters? Language Testing, 19(3), 246-276.
24. McNamara, T. (1996). Measuring second language performance. Harlow, Essex: Pearson Education.
25. McNamara, T. (2002). Discourse and assessment. Annual Review of Applied Linguistics, 22, 221-242.
26. Mickan, P. (2003). 'What's your score?' An investigation into language descriptors for rating written performance. Canberra: IELTS Australia.
27. Myford, C. M., & Wolfe, E. W. (2000). Monitoring sources of variability within the Test of Spoken English assessment system. Princeton, NJ: Educational Testing Service.
28. North, B., & Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15(2), 217-263.
29. Turner, C. E. (2000). Listening to the voices of rating scale developers: Identifying salient features for second language performance assessment. The Canadian Modern Language Review, 56(4), 555-584.
30. Turner, C. E., & Upshur, J. A. (2002). Rating scales derived from student samples: Effects of the scale maker and the student sample on scale content and student scores. TESOL Quarterly, 36(1), 49-70.
31. Upshur, J. A., & Turner, C. E. (1995). Constructing rating scales for second language tests. ELT Journal, 49(1), 3-12.
32. Upshur, J. A., & Turner, C. E. (1999). Systematic effects in the rating of second-language speaking ability: Test method and learner discourse. Language Testing, 16(1), 82-111.
33. Vaughan, C. (1991). Holistic assessment: What goes on in the rater's mind? In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts. Norwood, NJ: Ablex.
34. Watson Todd, R., Thienpermpool, P., & Keyuravong, S. (2004). Measuring the coherence of writing using topic-based analysis. Assessing Writing, 9, 85-104.
35. Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University Press.
36. White, E. M. (1985). Teaching and assessing writing. San Francisco: Jossey-Bass.
37. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.