COMPARING THE VALIDITY OF AUTOMATED AND HUMAN ESSAY SCORING

ETS Research Report Series

Volumn 2000, Issue 2, 2000, Pages i-23

  Powers, Donald E ^a   Burstein, Jill C ^a   Chodorow, Martin ^a   Fowles, Mary E ^a   Kukich, Karen ^a  

^a NONE

Author keywords

automated scoring; essay scoring; graduate record examinations (GRE ); validity; writing assessment; writing skills

Indexed keywords

EID: 85164492792     PISSN: None     EISSN: 23308516     Source Type: Journal    
DOI: 10.1002/j.2333-8504.2000.tb01833.x     Document Type: Article

References (21)

1. Burstein, J., Braden-Harder, L., Chodorow, M., Hua, S., Kaplan, B., Kukich, K., Lu, C., Nolan, J., Rock, D., & Wolff, S. (1998). Computer analysis of essay content for automated score prediction: A prototype automated scoring system for GMAT analytical writing assessment essays (ETS Research Report RR-98-15). Princeton, NJ: Educational Testing Service.
2. Burstein, J., & Chodorow, M. (1999, June). Automated essay scoring for nonnative English speakers. In Computer-Mediated Language Assessment and Evaluation of Natural Language Processing. Joint Symposium of the Association of Computational Linguistics and the International Association of Language Learning Technologies, College Park, Maryland.
3. Daigon, A. (1966). Computer grading of English composition. English Journal, 55, 46–52.
4. DeLoughry, T. J. (1995, October 20). Duke professor pushes concept of grading essays by computer. Chronicle of Higher Education, 42, A24–25.
5. Kaplan, R. M., Wolff, S., Burstein, J. C., Lu, C., Rock, D., & Kaplan, B. (1998). Scoring essays automatically using surface features (GRE Report No. 94-21 and ETS Research Report No. RR-98-30). Princeton, NJ: Educational Testing Service.
6. Labi, N. (1999, February 27). When computers do the grading. Time Magazine, 153, p. 57.
7. Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259–284.
8. Landauer, T. K., Laham, D., Rehder, B., & Schreiner, M. E. (1997). How well can passage meaning be derived without word order? A comparison of latent semantic analysis and humans. In M. G. Shafto & P. Langley (Eds.), Proceedings of the 19th Annual Conference of the Cognitive Science Society (pp. 412–417). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
9. Meng, X., Rosenthal, R., & Rubin, D. B. (1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111, 172–175.
10. Messick, S. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan.
11. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741–749.
12. Mitchell, J. S. (1998, May 27). Commentary: SATs don't get you in. Education Week on the Web [On-line serial]. Available: http://www.edweek.org/ew/current/3mitch.h17.
13. Murray, B. (1998, August). The latest techno tool: Essay-grading computers. APA Monitor, 29, p. 43.
14. Page, E. B. (1966). The imminence of grading essays by computer. Phi Delta Kappan, 48, 238–243.
15. Page, E. B. (1968). Analyzing student essays by computer. International Review of Education, 14, 210–225.
16. Page, E. B. & Petersen, N. S. (1995). The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan, 76, 561–565.
17. Petersen, N. S. (1997, March). Automated scoring of writing essays: Can such scores be valid? Paper presented at the annual meeting of the National Council on Education, Chicago, IL.
18. Powers, D. E., Fowles, M. E., & Welsh, C. K. (1999). Further validation of a writing assessment for graduate admissions (GRE Board Report No. 96-13R and ETS Research Report No. RR-99-18). Princeton, NJ: Educational Testing Service.
19. Schaeffer, G. A., Fowles, M. A., & Briel, J. B. (in press). Assessment of the psychometric characteristics of the GRE writing test (GRE Board Report No. 96-11). Princeton, NJ: Educational Testing Service.
20. Schwartz, A. E. (1998, April 26). Graded by machine. The Washington Post, p. C07.
21. Scott, J. (1999, January 31). Looking for the tidy mind, alas. The New York Times, Sec. 4, p. 2.