Validating automated essay scoring for online writing placement

Assessing Writing

Volumn 18, Issue 1, 2013, Pages 40-61

  Ramineni, Chaitanya ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

Automated essay scoring; Online placement; Writing assessment

Indexed keywords

EID: 84872410061     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2012.10.005     Document Type: Article

References (55)

1. Armstrong W. The association among student success in courses, placement test scores, student background data, and instructor grading practices. Community College Journal of Research and Practice 2000, 21:681-695.
2. Attali Y., Burstein J. Automated essay scoring with e-rater v.2. Journal of Technology, Learning, and Assessment 2006, 4(3). Retrieved from http://www.jtla.org.
3. Baker T., Jolly P. The " hard evidence": Documenting the effectiveness of a basic writing program. Journal of Basic Writing 1999, 18(1):27-39.
4. Composition in the twenty-first century: Crisis and change 1996, Southern Illinois University Press, Carbondale and Edwardsville, IL. L.Z. Bloom, D.A. Daiker, E.M. White (Eds.).
5. Breland, H., Kubota, M., Nickerson, K., Trapani, C. S., & Walker, M. (2004). New SAT writing prompt study: Analyses of group impact and reliability (Research Report 2004-1). New York, NY: College Board.
6. Burstein J., Chodorow M., Leacock C. Automated essay evaluation: The criterion online writing service. AI Magazine 2004, 25(3):27-35.
7. Complete College America. (2012). Remediation: Higher education's bridge to nowhere. Retrieved from http://www.completecollege.org/docs/CCA-Remediation-final.pdf.
8. Corso G. The role of the writing coordinator in a culture of placement by ACCUPLACER. Machine scoring of student essays: Truth and consequences 2006, 154-165. Utah State University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
9. Deane P. On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing 2013, 18:7-24. 10.1016/j.asw.2012.10.002.
10. Efron B., Tibshirani R. An introduction to the bootstrap 1993, Chapman and Hall, New York, NY.
11. Elliot N. On a scale: A social history of writing assessment in America 2005, Peter Lang, New York, NY.
12. Elliot N., Briller V., Joshi K. Portfolio assessment: Quantification and community. Journal of Writing Assessment 2007, 3:5-30.
13. Elliot N., Deess P., Rudniy A., Joshi K. Placement of students into first year writing courses. Research in the Teaching of English 2012, 46(3):285-313.
14. Fowles M. Writing assessment for admission to graduate and professional programs: Lessons learned and a note for the future. Writing assessment in the 21st century: Essays in honor of Edward M. White 2012, 135-146. Hampton Press, New York, NY. N. Elliot, L. Perelman (Eds.).
15. Gere A.R., Aull L., Green T., Porter A. Assessing the validity of directed self-placement at a large university. Assessing Writing 2010, 15:154-176.
16. Gleason B. Evaluating writing programs in real time: The politics of remediation. College Composition and Communication 2000, 51:560-588.
17. Haberman, S. J., & Sinharay, S. (2008). Sample-size requirements for automated essay scoring (ETS Research Report 08-32). Princeton, NJ: Educational Testing Service.
18. Hambleton R.K., Pitoniak M.M. Setting performance standards. Educational measurement 2005, 433-470. American Council on Education/Praeger, Westport, CT. 4th ed. R.L. Brennan (Ed.).
19. Hamp-Lyons L. Worrying about rating. Assessing Writing 2007, 12:1-9.
20. Haswell R.H. Multiple inquiry in the validation of writing tests. Assessing Writing 1998, 5:89-101.
21. Haswell, R. H. (2005). Post-secondary entrance writing placement. Retrieved from http://comppile.org/profresources/placement.htm.
22. Herrington A., Moran C. WritePlacer Plus in place: An exploratory case study. Machine scoring of student essays: Truth and consequences 2006, 114-129. Utah State University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
23. Hester V., O'Neill V., Neal M., Eddington A., Huot B. Adding portfolios to the placement process: A longitudinal perspective. Blurring boundaries: Developing writers, researchers, and teachers 2007, 61-90. Hampton Press, Creskill, NJ. P. O'Neill (Ed.).
24. Huot B. (Re) Articulating writing assessment in teaching and learning 2002, Utah State University Press, Logan, UT.
25. Inoue A.B., Poe M. Racial formations in two writing assessments: Revisiting White and Thomas's findings on the English Placement Test after 30 years. Writing assessment in the 21st century: Essays in honor of Edward M. White 2012, 341-359. Hampton Press, New York, NY. N. Elliot, L. Perelman (Eds.).
26. James C.L. Validating a computerized scoring system for assessing writing and placing students in composition courses. Assessing Writing 2006, 11:167-178.
27. James C.L. Electronic scoring of essays: Does topic matter?. Assessing Writing 2008, 13:80-92.
28. Jones E. ACCUPLACER's essay-scoring technology: When reliability does not equal validity. Machine scoring of student essays: Truth and consequences 2006, 93-113. Utah Sate University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
29. Klobucar A., Elliot N., Deess P., Rudniy A., Joshi K. Automated scoring in context: Rapid assessment for placed students. Assessing Writing 2013, 18:62-84. 10.1016/j.asw.2012.10.002.
30. Landauer T.K., Laham D., Foltz P.W. Automated scoring and annotation of essays with the Intelligent Essay Assessor. Automated essay scoring: A cross-disciplinary perspective 2003, 87-112. Erlbaum, Hillsdale, NJ. M.D. Shermis, J.C. Burstein (Eds.).
31. Maddox T. Piloting the COMPASS E-write software at Jackson State Community College. Machine scoring of student essays: Truth and consequences 2006, 147-153. Utah State University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
32. Mattern, K., & Packman, S. (2009). Predictive validity of ACCUPLACER® scores for course placement: A meta-analysis (ETS Research Report 2009-2). New York, NY: College Board. Retrieved from http://professionals.collegeboard.com/gateway.
33. Matzen R., Sorensen C. E-write as means for placement into three composition courses. Machine scoring of student essays: Truth and consequences 2006, 130-137. Utah State University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
34. McGee T. Taking a spin on the Intelligent Essay Assessor. Machine scoring of student essays: Truth and consequences 2006, 79-92. Utah State University Press, Logan, UT. P.F. Ericsson, R. Haswell (Eds.).
35. McGregor, E. N., & Attinasi, L. G. (1996). Developmental course-taking and subsequent academic performance at Pima Community College. Tucson, AZ: Office of Institutional Research at Pima Community College (ERIC Document Reproduction Service No. ED. 413018).
36. Neter J., Kutner M.H., Nachtsheim C.J., Wasserman W. Applied linear regression models 1996, Irwin, Homewood, IL.
37. O'Neill P., Moore C., Huot B. A guide to college writing assessment 2009, Utah State University Press, Logan, UT.
38. Page E.B. Project essay grade: PEG. Automated essay scoring: A cross-disciplinary perspective 2003, 43-54. Erlbaum, Hillsdale, NJ. M.D. Shermis, J.C. Burstein (Eds.).
39. Peckham I. Online placement in first-year writing. College Composition and Communication 2009, 60:517-540.
40. Peckham I. Online challenge versus offline ACT. College Composition and Communication 2010, 61:718-745.
41. Quinlan, T., Higgins, D., & Wolff, S. (2009). Evaluating the construct-coverage of the e-rater® scoring engine (ETS Research Report 09-01). Princeton, NJ: Educational Testing Service.
42. Ramineni C., Williamson D.M. Automated essay scoring: Psychometric guidelines and practices. Assessing Writing 2013, 18:25-39. 10.1016/j.asw.2012.10.002.
43. Reinheimer D.A. Validating placement: Local means, multiple measures. Assessing Writing 2007, 12:170-179.
44. Directed self-placement: Principles and practices 2003, Hampton Press, Cresskill, NJ. D.J. Royer, R. Gilles (Eds.).
45. Rudner L.M., Garcia V., Welch C. An evaluation of IntelliMetric™ essay scoring system. The Journal of Technology, Learning, and Assessment 2006, 4(4). Retrieved from http://www.jtla.org.
46. Shermis M.D., Mzumara H., Olson J., Harrington S. On-line grading of student essays: PEG goes on the World Wide Web. Assessment and Evaluation in Higher Education 2001, 26. Retrieved from http://www.cblt.soton.ac.uk/principles03/pdf/On-lineGrad_N_PEG.pdf.
47. Strickland K., Strickland J. Un-covering the curriculum: Whole language in secondary and postsecondary classrooms 1993, Boynton/Gook, Portsmouth, NH.
48. What is " college-level" writing? 2006, Vol. 1. National Council of Teachers of English, Urbana, IL.
49. What is " college-level" writing? 2010, Vol. 2. National Council of Teachers of English, Urbana, IL.
50. Wang J., Brown M.S. Automated essay scoring versus human scoring: A correlational study. Contemporary Issues in Technology and Teacher Education 2008, 8(4). Retrieved from http://www.citejournal.org/vol8/iss4/languagearts/article1.cfm.
51. Weigle, S. C. (2011). Validation of automated scores of TOEFL iBT® Tasks against nontest indicators of writing ability (ETS Research Report 11-24). Princeton, NJ: Educational Testing Service.
52. White E.M. Teaching and assessing writing: Recent advances in understanding, evaluating, and improving student performance 1994, Jossey-Bass, San Francisco, CA.
53. White E.M. The opening of the modern era of writing assessment: A narrative. College English 2001, 63:306-320.
54. Williamson D.M., Xi X., Breyer F.J. A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practices 2012, 31:2-13.
55. Willingham W.W., Pollack J.M., Lewis C. Grades and test scores: Accounting for observed differences. Journal of Educational Measurement 2002, 39:1-97.