Writing evaluation: What can analytic versus holistic essay scoring tell us?

System

Volumn 29, Issue 3, 2001, Pages 371-383

  Bacha, Nahla ^a  

^a LEBANESE AMERICAN UNIVERSITY   (Lebanon)

Author keywords

Analytic evaluation; English as a Foreign Second Language; Essay evaluation; Holistic evaluation; Writing evaluation

Indexed keywords

EID: 0042418714     PISSN: 0346251X     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0346-251X(01)00025-2     Document Type: Article

References (66)

1. Alderson C.J., Beretta A. Evaluating Second Language Education. 1992;Cambridge University Press, Cambridge.
2. Bachman L.F. Fundamental Considerations in Language Testing. 1990;Oxford University Press, Oxford.
3. Bachman L.F. What does language testing have to offer. TESOL Quarterly. 25(4):1991;671-672.
4. Bamberg B. Multiple-choice and holistic essay scores: what are they measuring? College Composition and Communication. 33(4):1982;404-406.
5. Braddock R.et al. Research in Written Composition. 1963;National Council of Teachers of English, Illinois.
6. Brossell G. Rhetorical specification in essay examination topics. College English. 45(2):1983;165-173.
7. Carlman N. Topic differences on writing tests: how much do they matter? English Quarterly. 19(1):1986;39-49.
8. Carlson, S. et al., 1985: Relationship of Admission Test Scores to Writing Performance of Native and Non-native Speakers of English (TOEFL Research Reports, 19). Educational Testing Service, Princeton, New Jersey.
9. Caudery T. The validity of timed essay tests in the assessment of writing skills. ELT Journal. 44(2):1990;122-131.
10. Charney D. The validity of using holistic scoring to evaluate writing. Research in the Teaching of English. 18:1984;65-81.
11. Cohen L., Manion L. Research Methods in Education. 1994;Routledge, New York.
12. Connor-Linton J. Looking behind the curtain: what do L2 composition ratings really mean? TESOL Quarterly. 29(4):1995;762-765.
13. Cooper C.R., Odell L. Evaluating Writing: Describing, Measuring, Judging. 1977;National Council of Teachers of English, Urbana, Illinois.
14. Crowhurst M., Piche G.L. Audience and mode of discourse effects on syntactic complexity in writing at two grade levels. Research in the Teaching of English. 13(2):1979;101-109.
15. Cumming A. Expertise in evaluating second language compositions. Language Testing. 7(1):1990;31-51.
16. Cushing-Weigle S. Effects of training on raters of ESL compositions. Language Testing. 11(2):1994;197-223.
17. Douglas D. Developments in language testing. Annual Review of Applied Linguistics. 15(5):1995;167-187.
18. Doushaq M.H. An investigation into stylistic errors of Arab students learning English for academic purposes. English for Specific Purposes. 5(1):1986;27-39.
19. Elbow P. Ranking, evaluating, and liking: sorting out three forms of judgments. Straub R. A Sourcebook for Responding to Student Writing. 1999;175-196 Hampton Press, Inc, New Jersey.
20. Gamaroff R. Rater reliability in language assessment: the bug of all bears. System. 28(1):2000;31-53.
21. Hamp-Lyons, L., 1986a. Testing second language writing in academic settings. Unpublished doctoral dissertation, University of Edinburgh, England.
22. Hamp-Lyons L. Writing in a foreign language and rhetorical transfer: influences on raters' evaluation. Meara P. Spoken Language. 1986b;72-84 Center for Information on Language Teaching, London.
23. Hamp-Lyons L. Second language writing: assessment issues. Kroll B. Second Language Writing: Research Insights for the Classroom. 1990;69-87 Cambridge University Press, Cambridge.
24. Hamp-Lyons L. Assessing Second Language Writing in Academic Contexts. 1991;Ablex, Norwood, NJ.
25. Hamp-Lyons L. Rating nonnative writing: the trouble with holistic scoring. TESOL Quarterly. 29(4):1995;759-762.
26. Hayward M. Evaluations of essay prompts by nonnative speakers of English. TESOL Quarterly. 24(4):1990;753-758.
27. Hillocks G. Jr. Research on Written Composition: New Directions for Teaching. 1986;National Conference on Research in English and ERIC Clearinghouse on Reading and Communication Skills, Urbana, Illinois.
28. Hoetker J. Essay examination topics and students' writing. College Composition and Communication. 33(4):1982;377-392.
29. Hoey M. Patterns of Lexis in Text. 1991;Oxford University Press, Oxford. (Winner of the Duke of Edinburgh English Language Prize, 1991).
30. Homburg T.J. Holistic evaluation of ESL compositions: can it be validated objectively? TESOL Quarterly. 18(1):1984;87-107.
31. Horowitz D.M. Essay examination prompts and the teaching of academic writing. English for Specific Purposes. 5(2):1986a;107-120.
32. Horowitz D.M. What professors actually require: academic tasks for the ESL classroom. TESOL Quarterly. 20(3):1986b;445-462.
33. Jacobs H.J.et al. Testing ESL Composition: A Practical Approach. 1981;Newbury House, Rowley, MA.
34. Kaczmarek C.M. Scoring and rating essay tasks. Oller J.W. Jr., Perkins K. Research in Language Testing. 1980;151-159 Newbury House Publishers Incorporated, MA.
35. Kegley P.H. The effect of mode-discourse on student writing performance: implications for policy. Educational Evaluation and Policy Analysis. 8(2):1986;147-154.
36. Kroll B. Second Language Writing: Research Insights for the Classroom. 1990a;Cambridge University Press, Cambridge.
37. Kroll B. What does time buy? ESL student performance on home versus class compositions. Kroll B. Second Language Writing Research Insights for the Composition. 1990b;140-154 Cambridge University Press, Cambridge.
38. Kunnan A.J. Validation in Language Assessment: Selected Papers from the 17th Language Testing Research Colloquium, Long Beach. 1998;Lawrence Erlbaum Associates, Publishers, New Jersey.
39. Lloyd-Jones R. Tests of writing ability. Tate G. Teaching Composition: 12 Bibliographical Essays. 1987;155-176 Christian University Press, Ft. Worth, Texas.
40. Mullen K.A. Evaluating writing proficiency in ESL. Oller J.W. Jr., Perkins K. Research in Language Testing. 1980;160-170 Newbury House Publishers Incorporated, MA.
41. Myers M. A Procedure for Writing Assessment and Holistic Scoring. 1980;National Council of Teachers of English, Urbana, Illinois.
42. Najimy N.C. Measure for Measure: A Guidebook for Evaluating Students' Expository Writing. 1981;National Council of Teachers of English, Urbana, Illinois.
43. Oller J.W. Jr., Perkins K. Research in Language Testing. 1980;Newbury House Publishers Incorporated, Massachusetts.
44. Pere-Woodley M.P. Writing in L1 and L2: analyzing and evaluating learners' texts. Language Teaching. 2:1991;69-83.
45. Perkins K. On the use of composition scoring techniques, objective measures and objective tests to evaluate ESL writing ability. TESOL Quarterly. 17(4):1983;651-671.
46. Pierce B.N. TOEFL test of written English (TWE) scoring guide. TESOL Quarterly. 25(1):1991;159-163.
47. Purves A.C., Degenhart Reflections on research and assessment in written composition. Research in the Teaching of English. 26(1):1992;108-122.
48. Quellmalz E.S.et al. Effects of discourse and response mode on the measurement of writing competence. Journal of Educational Measurement. 19(4):1982;241-258.
49. Raymond J.C. What we don't know about the evaluation of writing. College Composition and Communication. 33(4):1982;399-403.
50. Read J. Providing relevant content in an EAP writing test. English for Specific Purposes. 9:1990;109-121.
51. Reid J.M. Teaching ESL Writing. 1993;Prentice-Hall, New Jersey.
52. Ruth L., Murphy S. Designing writing tasks for the assessment of writing. 1988;Ablex, Norwood, NJ.
53. Shakir A. Coherence in EFL student-written texts: two perspectives. Foreign Language Annals. 24(5):1991;399-411.
54. Shohamy E. Performance assessment in language testing. Annual Review of Applied Linguistics. 15:1995;188-211.
55. Siegel A. Multiple tests: some practical considerations. TESOL Quarterly. 24(4):1990;773-775.
56. Smith W.et al. Some effects of varying the structure of a topic on college students' writing. Written Communication. 2(1):1985;73-88.
57. SPSS Base 7.5 for Windows' User's Guide, 1997. SPSS Inc, Michigan, Chicago.
58. Sweedler-Brown C.O. The effect of training on the appearance bias of holistic essay graders. Journal of Research and Development in Education. 26(1):1992;24-29.
59. Sweedler-Brown C.O. ESL essay evaluation: the influence of sentence-level and rhetorical features. Journal of Second Language Writing. 2(1):1993;3-17.
60. Silva T. Toward an understanding of the distinct nature of L2 writing: the ESL research and its implications. TESOL Quarterly. 27(4):1993;657-677.
61. Tedick D.J. ESL writing assessment: subject matter knowledge and its impact on performance. English for Specific Purposes. 9:1990;123-143.
62. Tedick, D.J., Mathison, M.A., 1995. Holistic scoring in ESL writing assessment: what does an analysis of rhetorical features reveal? In: Academic Writing in a Second Language: Essays on Research and Pedagogy. Ablex Publishing Corporation, Norwood, pp. 205-230.
63. Upshur J.A., Turner C.E. Constructing rating scales for second language tests. ELT Journal. 49(1):1995;3-12.
64. Weir, C., 1983. Identifying the language problems of overseas students in tertiary education in the UK. Unpublished doctoral dissertation, University of London, England.
65. Weir C. Communicative Language Testing. 1990;Prentice-Hall, London.
66. White E.M. Teaching and Assessing Writing: Recent Advances in Understanding, Evaluating, and Improving Student Performance. 1994;Jossey-Bass Publishers, San Francisco.