"I must impress the raters!" An investigation of Chinese test-takers' strategies to manage rater impressions

Assessing Writing

Volumn 25, Issue , 2015, Pages 21-36

  Xie, Qin ^a  

^a THE EDUCATION UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Essay tests; Holistic scoring; Test validity; Test taker perception; Test wiseness strategy; Washback

Indexed keywords

EID: 84934873442     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2015.05.001     Document Type: Article

References (46)

1. Anthony L. AntConc (Version 3.4.1) [Computer Software] 2013, Waseda University, Tokyo, Japan, Available from http://www.antlab.sci.waseda.ac.jp/.
2. Bacha N. Writing evaluation: What can analytic versus holistic essay scoring tell us?. System 2001, 29(3):371-383.
3. Barkaoui K. Explaining ESL essay holistic scores: A multilevel modeling approach. Language Testing 2010, 27(4):515-535.
4. Barkaoui K. Effects of marking method and rater experience on ESL essay scores and rater performance. Assessment in Education: Principles, Policy and Practice 2011, 18(3):279-293. 10.1080/0969594x.2010.526585.
5. Caudery T. The validity of timed essay tests in the assessment of writing skills. ELT Journal 1990, 44(2):122-131.
6. Charney D. The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English 1984, 18(1):65-81.
7. Cheng L.Y., Curtis A. English language assessment and the Chinese learner 2010, Routledge, New York.
8. Cohen A.D. The coming of age of research on test-taking strategies. Language Assessment Quarterly 2006, 3(4):307-331.
9. Crossley S.A., McNamara D.S. Cohesion coherence and expert evaluations of writing proficiency 2010.
10. Crossley S.A., McNamara D.S. Understanding expert ratings of essay quality: Coh-Metrix analyses of first and second language writing. International Journal of Continuing Engineering Education and Life-Long Learning 2011, 21(2-3):170-191. 10.1504/ijceell.2011.040197.
11. Cumming A. Assessing academic writing in foreign and second languages. Language Teaching 2009, 42(01):95-107.
12. Daller H., Phelan D. What's in a teacher's mind? Teacher ratings of EFL essays and different aspects of lexical richness. Modelling and assessing vocabulary knowledge 2007, 221-234. Cambridge University Press, Cambridge. H. Daller, J. Milton, J. Trefers-Daller (Eds.).
13. Doe C., Fox J. Exploring the testing process: Three test takers' observed and reported strategy use over time and testing contexts. Canadian Modern Language Review 2011, 67(1):29-54.
14. Elbow P. Writing with power: Techniques for mastering the writing process 1998, Oxford University Press., Oxford, New York. 3rd ed.
15. Engber C.A. The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing 1995, 4(2):139-155.
16. Ferris D.R. Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly 1994, 28(2):414-420.
17. Fritz E., Ruegg R. Rater sensitivity to lexical accuracy, sophistication and range when assessing writing. Assessing Writing 2013, 18(2):173-181.
18. Gamaroff R. Rater reliability in language assessment: The bug of all bears. System 2000, 28(1):31-53.
19. Gardiner T.C. Composition evaluation as an unfolding process: The impact of essay features on initial impressions and final assessments of writing quality 2004, PhD thesis, University of Georgia, Athens, Georgia.
20. Glenwright P. Language proficiency assessment for teachers: The effects of benchmarking on writing assessment in Hong Kong schools. Assessing Writing 2002, 8(2):84-109.
21. Grant L., Ginther A. Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing 2000, 9(2):123-145.
22. Hamp-Lyons L. Rating nonnative writing: The trouble with holistic scoring. TESOL Quarterly 1995, 29(4):759-762.
23. Huot B. Reliability, validity, and holistic scoring: What we know and what we need to know. College Composition and Communication 1990, 41(2):201-213.
24. Knoch U., Elder C. Validity and fairness implications of varying time conditions on a diagnostic test of academic English writing proficiency. System 2010, 38(1):63-74.
25. Lee H.K. The relationship between writers' perceptions and their performance on a field-specific writing test. Assessing Writing 2008, 13(2):93-110.
26. Lee Y.W., Gentile C., Kantor R. Toward automated multi-trait scoring of essays: Investigating links among holistic, analytic, and text feature scores. Applied Linguistics 2010, 31(3):391-417. 10.1093/applin/amp040.
27. Lim G.S. The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced raters. Language Testing 2011, 28(4):543-560. 10.1177/0265532211406422.
28. Lumley T. Assessment criteria in a large-scale writing test: What do they really mean to the raters?. Language Testing 2002, 19(3):246-276.
29. McKee G., Malvern D., Richards B. Measuring vocabulary diversity using dedicated software. Literary and Linguistic Computing 2000, 15(3):323-337.
30. McNamara D.S., Louwerse M.M., Cai Z., Graesser A. Coh-Metrix version 3.0 2012, Retrieved from http://cohmetrix.memphis.edu.
31. Messick S. Issues of effectiveness and equity in the coaching controversy: Implications for educational and testing practice. Educational Psychologist 1982, 17(2):67-91.
32. Millman J., Bishop C.H., Ebel R. An analysis of test-wiseness. Educational and Psychological Measurement 1965, 25(3):707-726.
33. O'Malley J.M., Chamot A.U. Learning strategies in second language acquisition 1990, Cambridge University Press, Cambridge, UK.
34. Oxford R.L. Language learning strategies: What every teacher should know 1990, Heinle & Heinle Publisher, Boston.
35. Plakans L. Comparing composing processes in writing-only and reading-to-write test tasks. Assessing Writing 2008, 13(2):111-129.
36. Qi L. Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high-stakes English test in China. Assessment in Education 2007, 14(1):51-74.
37. Ramineni C., Williamson D.M. Automated essay scoring: Psychometric guidelines and practices. Assessing Writing 2013, 18(1):25-39. 10.1016/j.asw.2012.10.004.
38. Schafer W.D., Gagne P., Lissitz R.W. Resistance to confounding style and content in scoring constructed-response items. Educational Measurement: Issues and Practice 2005, 24(2):22-28. 10.1111/j.1745-3992.2005.00008.x.
39. Weigle S.C. Assessing writing 2002, Cambridge University Press, Cambridge, London.
40. White E.M. Holistic scoring: Past triumphs, future challenges. Validating holistic scoring for writing assessment: Theoretical and empirical foundations 1993, 79-108. Hampton Press, Inc, Cresskill, New Jersey. M.M. Williamson, B. Huot (Eds.).
41. White E.M. An apologia for the timed impromptu essay test. College Composition and Communication 1995, 46(1):30-45. 10.2307/358868.
42. Williamson M.M. An introduction to holistic scoring: The social, historical, and theoretical context for writing assessment. Validating holistic scoring for writing assessment: Theoretical and empirical foundations 1993, 1-44. Hampton Press, Inc, Cresskill, New Jersey. M.M. Williamson, B. Huot (Eds.).
43. Xie Q. Does test preparation work? Implications for score validity. Language Assessment Quarterly (SSCI, A) 2013, 10(2):196-218. 10.1080/15434303.2012.721423.
44. Xie Q., Andrews S. Do test design and uses influence test preparation: Testing a model of washback with Structural Equation Modeling. Language Testing 2013, 30(1):49-70. 10.1177/0265532212442634.
45. Xu Y., Wu Z. Test-taking strategies for a high-stakes writing test: An exploratory study of 12 Chinese EFL learners. Assessing Writing 2012, 17(3):174-190.
46. Yang H.C., Plakans L. Second language writers' strategy use and performance on an integrated reading listening and writing task. TESOL Quarterly 2012, 46(1):80-103.