English language learners and automated scoring of essays: Critical considerations

Assessing Writing

Volumn 18, Issue 1, 2013, Pages 85-99

  Weigle, Sara Cushing ^a  

^a GEORGIA STATE UNIVERSITY   (United States)

Author keywords

Automated essay scoring; English language learners; Second language writing; Writing assessment

Indexed keywords

EID: 84872404793     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2012.10.006     Document Type: Article

References (75)

1. Alderson J.C. Bands and scores. Language testing in the 1990: The Communicative Legacy 1991, 71-86. MacMillan, London, UK. J.C. Alderson, B. North (Eds.).
2. Alderson J.C. The CEFR and the need for more research. The Modern Language Journal 2007, 91(4):659-663.
3. Alderson J., Charles, Figueras N., Kuiper H., Nold G. Analysing tests of reading and listening in relation to the Common European Framework of Reference: The Experience of the Dutch CEFR Construct Project. Language Assessment Quarterly 2006, 3(1.):3-30.
4. Arter J., Spandel V., Culham R., Pollard J. The impact of training students to be self-assessors of writing. Paper presented at the annual meeting of the American Educational Research Association 1994, April.
5. Attali Y., Burstein J. Automated essay scoring with e-rater v. 2.0. Journal of Technology, Learning, and Assessment 2006, 4(3):1-30.
6. Bachman L. Building and supporting a case for test use. Language Assessment Quarterly 2005, 19(4):453-476.
7. Bardovi-Harlig K. Tense and aspect in second language acquisition: Form, meaning, and use 2000, Blackwell, Oxford, UK.
8. Barkaoui K. Rating scale impact on EFL essay marking: A mixed-method study. Assessing Writing 2007, 12(2):86-107.
9. Barkaoui K. Effects of marking method and rater experience on ESL essay scores. Assessment in Education: Principles, Policy & Practice 2011, 18(3):277-291.
10. Biber D. Variation across speech and writing 1988, Cambridge University Press, Cambridge, UK.
11. Building a validity argument for the Test of English as a Foreign Language™ 2008, Routledge, New York, NY. C.A. Chapelle, M.K. Enright, J.M. Jamieson (Eds.).
12. Chenoweth N.A., Hayes J.R. Fluency in writing: Generating text in L1 and L2. Written Communication 2001, 18(1):80-98.
13. Cheville J. Automated scoring technologies and the rising influence of error. English Journal 2004, 93(4):47-52.
14. Chodorow M., Gamon M., Tetreault J. The utility of grammatical error detection systems for English language learners: Feedback and assessment. Language Testing 2010, 27(3):419-436.
15. Collier V. Age and rate of acquisition of second languages for academic purposes. TESOL Quarterly 1987, 21(4):617-641.
16. Collier V. How long? A synthesis of research on academic achievement in a second language. TESOL Quarterly 1989, 23:500-531.
17. Conference on College Composition and Communication. (2004, February 25). CCCC position statement on teaching, learning, and assessing writing in digital environments. Retrieved from http://www.ncte.org/cccc/resources/positions/digitalenvironments.
18. Condon W. Large-scale assessment, locally-developed measures, and automated scoring of essays: Fishing for red herrings?. Assessing Writing 2013, 18:100-108. 10.1016/j.asw.2012.10.002.
19. Connor U. New directions in contrastive rhetoric. TESOL Quarterly 2002, 36(4):493-510.
20. Coulter C., Smith M.L. English language learners in a comprehensive high school. Bilingual Research Journal: The Journal of the National Association for Bilingual Education 2006, 30(3.):309-335.
21. Council of Europe Common European framework of reference for languages: Learning, teaching, assessment 2001, Cambridge University Press, Cambridge, UK.
22. Crossley S.A., McNamara D.S. Computationally assessing lexical differences in L1 and L2 writing. Journal of Second Language Writing 2009, 17(2):119-135.
23. Crossley S.A., McNamara D.S. Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading 2012, 35(2):115-135.
24. Crossley S.A., Salsbury T., McNamara D.S. Predicting the proficiency level of language learners using lexical indices. Language Testing 2012, 29(2):240-260.
25. Crusan D. Assessment in the second language writing classroom 2010, University of Michigan Press, Ann Arbor, MI.
26. Cumming A. Writing expertise and second language proficiency. Language Learning 1989, 39(1):81-141.
27. Cumming A. Expertise in evaluating second language compositions. Language Testing 1990, 7(1):31-51.
28. Cummins J. Linguistic interdependence and the educational development of bilingual children. Review of Educational Research 1979, 49(2):222-251.
29. Deane P. On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing 2013, 18:7-24. 10.1016/j.asw.2012.10.002.
30. Duff P. Language, literacy, content, and (pop) culture: Challenges for ESL students in mainstream courses. Canadian Modern Language Review 2001, 58(1):103-132.
31. Durgunoĝlu A.Y., Hughes T. How prepared are the U.S. preservice teachers to teach English language learners?. International Journal of Teaching and Learning in Higher Education 2010, 22(1):32-41.
32. Eckes T. Rater types in writing performance assessments: A classification approach to rater variability. Language Testing 2008, 25(2):155-185.
33. ® to score essays written by English language learners: A complement to human judgment. Language Testing 2010, 27(3):317-334.
34. Ferris D.R. Teaching college writing to diverse student populations 2009, University of Michigan Press, Ann Arbor, MI.
35. Hakuta K. Educating language minority students and affirming their equal rights: Research and practical perspectives. Educational Researcher 2011, 40:163.
36. Hamp-Lyons Assessing second language writing in academic contexts 1991, Ablex, Norwood, NJ.
37. Herrington A., Moran C. What happens when machines read our students' writing?. College English 2001, 63(4):480-499.
38. Hinkel E. Teaching academic ESL writing 2004, Lawrence Erlbaum Associates, Mahwah, NJ.
39. Hirvela A. Writing to learn in content areas: Research insights. Learning-to-write and writing-to-learn in an additional language 2011, 37-59. John Benjamins Publishing Company, Philadelphia, PA. R.M. Manchón (Ed.).
40. Hornberger N.H. Continua of biliteracy: an ecological framework for educational policy, research, and practice in multilingual settings 2003, Multilingual Matters, Clevedon, UK; Buffalo, NY.
41. IIE. (2011). International student enrollment increased by 5 percent in 2010/11, led by strong increase in students from China. Retrieved from http://www.iie.org/en/Who-We-Are/News-and-Events/Press-Center/Press-Releases/2011/2011-11-14-Open-Doors-International-Students.
42. Kane M. An argument-based approach to validity. Psychological Bulletin 1992, 112:527-535.
43. Kane M., Crooks T., Cohen A. Validating Measures of Performance. Educational Measurement: Issues and Practice 1999, 18(2):5-17. 10.1111/j.1745-3992.1999.tb00010.x.
44. Kane M. Validating high-stakes testing programs. Educational Measurement: Issues and Practices 2002, 21(1):31-41.
45. Leki I., Cumming A.H., Silva T. A synthesis of research on second language writing in English 2008, Routledge, New York.
46. Li X. " Good writing" in cross-cultural context 1996, SUNY Press, New York, NY.
47. Little D. The common European framework of reference for languages: content, purpose, origin, reception and impact. Language Teaching 2006, 39(3):167.
48. Lumley T. Assessing Second Language Writing: The Rater's Perspective (Volume 3 of Language Testing and Evaluation) 2005, Peter Lang, New York, NY.
49. Manchón R.M. Situating the learning-to-write and writing-to-learn dimensions of L2 writing. Learning-to-write and writing-to-learn in an additional language 2011, 3-14. John Benjamins Publishing Company, Philadelphia, PA. R.M. Manchón (Ed.).
50. Manchón R.M. Writing to learn the language: Issues in theory and research. Learning-to-write and writing-to-learn in an additional language 2011, 61-82. John Benjamins Publishing Company, Philadelphia, PA. R.M. Manchón (Ed.).
51. Messick S. Test validity: A matter of consequence. Social Indicators Research 1998, 45(1-3):35-44.
52. Migration Policy Institute. (2010). States and districts with the highest number and share of English language learners. Retrieved from http://migrationinformation.org/ellinfo/FactSheet_ELL2.pdf.
53. O'Neill P., Adler-Kassner L., Fleischer C., Hall A. " Creating the Framework for Success in Postsecondary Writing." Symposium: On the Framework for Success in Postsecondary Writing. College English 2012, 74:520-524.
54. Adult language acquisition: Cross-linguistic perspectives 1993, Cambridge University Press, New York, NY. C. Perdue (Ed.).
55. Rinnert R., Kobayashi H. Differing perceptions of EFL writing among readers in Japan. The Modern Language Journal 2001, 85(2):189-209.
56. Rudner L., Garcia V., Welch C. An evaluation of IntelliMetric™ essay scoring system. The Journal of Technology, Learning, and Assessment 2006, 4(4). Retrieved from. http://escholarship.bc.edu/ojs/index.php/jtla/article/view/1651/1493.
57. Sasaki M., Hirose K. Explanatory variables for EFL students' expository writing. Language Learning 1996, 46(1):137-168.
58. Shermis, M. D., & Hamner, B. (in press) Contrasting state-of-the-art automated scoring of essays. In M. D. Shermis and J. Burstein (Eds.), Handbook of automated essay evaluation: Current applications and new directions New York, NY: Routledge.
59. Shi L. Native and nonnative speaking EFL teachers' evaluation of Chinese students' English writing. Language Testing 2001, 18:303-332.
60. Sukkarieh J.Z., Blackmore J. c-rater: Automatic content scoring for short constructed responses. Florida Artificial Intelligence Research Society Conference 2009, Retrieved from. http://aaai.org/ocs/index.php/FLAIRS/2009/paper/view/122.
61. Sukkarieh J.Z., Stoyanchev S. Automated model building in c-rater. Proceedings of TextInfer: The ACL/IJCNLP 2009 Workshop on Applied Textual Inference 2009, 61-69.
62. Ware P. Computer-generated feedback on student writing. TESOL Quarterly 2011, 45(4):769-774.
63. Weigle S.C. Assessing writing 2002, Cambridge University Press, Cambridge, UK.
64. Weigle S.C. Second language writing expertise. Expertise in language learning and teaching 2005, 128-149. Palgrave Macmillan, Hampshire, UK. K. Johnson (Ed.).
65. Weigle S.C. Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. Language Testing 2010, 27(3):335-353.
66. Weigle, S. C. (2011). Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability. TOEFL iBT Research Report TOEFL iBT-15. Princeton, NJ: Educational Testing Service.
67. Weigle, S. C. (in press). Considerations for teaching an ESL/EFL writing course. In: M. Celce-Murcia, D. Brinton, & M. A. Snow (Eds.), Teaching English as a second or foreign language (4th ed.). Boston, MA: Heinle Cengage.
68. Weir J. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research 2005, 19(1):231-240.
69. Whalen K., Ménard N. L1 and L2 writers' strategic and linguistic knowledge: A model of multiple-level discourse processing. Language Learning 1995, 45(3):381-418.
70. Williams J. Teaching writing in second and foreign language classrooms 2005, McGraw-Hill, New York, NY.
71. Williamson D., Xi X., Breyer F.J. A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practice 2012, 31(1):2-13.
72. WPA. (2012). WPA outcomes statement for first-year composition. Retrieved from http://wpacouncil.org/positions/outcomes.html/.
73. Xi X. Automated scoring and feedback systems: Where are we and where are we heading?. Language Testing 2010, 27(3):291-300.
74. Yang, W., Li, M., Ye, L., & Weigle, S. C. (2010, March). Rater and task effects on scores on TOEFL iBT independent writing tasks. In: Paper presented at the annual meeting American Association of Applied Linguistics (AAAL), Atlanta, GA.
75. Zheng Y., Cheng L. College English Test (CET) in China. Language Testing 2008, 25(3):408-417.