Automated essay evaluation and the teaching of writing

Handbook of Automated Essay Evaluation: Current Applications and New Directions

Volumn , Issue , 2013, Pages 16-35

  Elliot, Norbert ^a   Klobucar, Andrew ^a  

^a NEW JERSEY INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

POPULATION STATISTICS;

EID: 84894113648     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203122761     Document Type: Chapter

References (85)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association
2. Bazerman, C. (2008). Introduction. In C. Bazerman (Ed.), Handbook on research on writing: History, society, school, individual, text (pp. 1–4). New York, NY: Lawrence Erlbaum
3. Behm, N., Glau, G. R., Holdstein, D. H., Roen, D., & White, E. M. (2012). The WPA outcomes statement: A decade later. Anderson, SC: Parlor Press
4. Bejar, I. (2012). Rater cognition: Implications for validity. Educational Measurement: Issues and Practice, 31, 2–9
5. Bennett, R. E. (1993). On the meanings of constructed response. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 1–27). Hillsdale, NJ: Lawrence Erlbaum
6. Bennett, R.E. (2011). Automated scoring of constructed-response literacy and mathematics items. Washington, DC: Arabella Philanthropic Investment Advisors
7. Bennett, R. E., & Bejar, I. I. (1998). Validity and automated scoring: It’s not only the scoring. Educational Measurement: Issues and Practice, 17(4), 9–17
8. Borsboom, D. (2005). Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge, UK: Cambridge University Press
9. Burstein, J. (2012). Fostering best practice in writing assessment and instruction with e-rater®. In N. Elliot & L. Perelman (Eds.), Writing assessment in the 21st century: Essays in honor of Edward M. White (pp. 187–217). New York, NY: Hampton Press
10. Cambridge, D., Cambridge, B., & Yancey, K. B. (2009). Electronic portfolios 2.0: Emergent research on implementation and impact. Sterling, VA: Stylus Publishing
11. Camp, R. (1993). The place of portfolios in our changing views of writing assessment. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement: Issues in constructed response, performance testing, and portfolio assessment (pp. 183–212). Hillsdale, NJ: Lawrence Erlbaum
12. Carr, N. G. (2003). IT doesn’t matter. Harvard Business Review, 81, 41–49
13. Chandler-Olcott, K. (2002). Teacher research as a self-extending system for practitioners. Teacher Education Quarterly, 29, 23–38
14. Cheville, J. (2004). Automated scoring technologies and the rising influence of error. English Journal, 93, 47–52
15. Chomsky, N. (1987). On power and ideology: The Managua lectures. Boston, MA: South End Press
16. Christensen, C. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Boston, MA: Harvard Business School Press
17. Christensen, C., Horn, M. B., Caldera, L., & Soares, L. (2011). Disrupting college: How disruptive innovation can deliver quality and affordability to postsecondary education. Washington, DC: Center for American Progress
18. Christensen, C., Horn, M. B., & Johnson, C. W. (2008). Disrupting class: How disruptive innovation will change the way the world learns. New York, NY: McGraw Hill
19. Christensen, C., & Raynor, M. E. (2003). The innovator’s solution: Creating and sustaining successful growth. Boston, MA: Harvard Business School Press
20. Cochran-Smith, M., & Lytle, S. (1993). Inside/outside: Teacher research and knowledge. New York, NY: Teachers College Press
21. Cohen, A. M., & Kisker, C. B. (2010). The shaping of American higher education: Emergence and growth of the contemporary system. San Francisco, CA: Jossey Bass
22. Complete College America. (2012). Remediation: Higher education’s bridge to nowhere. Washington, DC: Complete College America
23. Conference on College Composition and Communication (2004). Position statement on teaching, learning, and assessing writing in digital environments. Retrieved from http://www.ncte.org/cccc/resources/positions/digitalenvironments
24. Conference on College Composition and Communication (2009). Writing assessment: A position statement. Retrieved from http://www.ncte.org/cccc/resources/positions/writingassessment
25. De Raad, B. (2000). The big five personality factors: The psycholexical approach to personality. Gottingen: Hogrefe & Huber
26. Deane, P., & Quinlan, T. (2010). What automated analyses of corpora can tell us about students’ writing skills. Journal of Writing Research, 2(2), 152–177
27. DeMillo, R. A. (2012). So you’ve got technology. So what? Chronicle of Higher Education. Retrieved from http://chronicle.com/article/So-Youve-Got-Technology-So/131663/
28. Derrida, J. (Ed.). (1976). Of grammatology (G. C. Spivak, Trans.). Baltimore, MD: Johns Hopkins University Press
29. Dillon, R. F., & Sternberg, R. J. (Eds.). (1986). Cognition and instruction. San Diego, CA: Academic Press
30. Elliot, N., Briller, V., & Joshi, K. (2007). Portfolio assessment: Quantification and community. Journal of Writing Assessment, 3, 5–30
31. Elliot, N., Deess, P., Rudniy, A., & Johsi, K. (2012). Placement of students into first-year writing courses. Research in the Teaching of English, 46, 285–313
32. Ellul, J. (1964). The technological society (J. Wilkinson, Trans.). New York, NY: Knopf
33. Embretson, S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179–197
34. Ericsson, P. F., & Haswell, R. (Eds.) (2006). Machine scoring of student essays: Truth and consequences. Logan, UT: Utah State University Press
35. Farley, T. (2009). Making the grade: My misadventures in the standardized testing industry. Sausalito, CA: PoliPoint Press
36. Flower, L. (1994). The construction of negotiated meaning: A social cognitive theory of writing. Carbondale and Edwardsville, IL: Southern Illinois University Press
37. Forgione, P. D. (2012). Coming together to raise achievement: New assessments for the Common Core State Standards. Retreived from http://www.k12center.org/rsc/pdf/Assessments_for_the_Common_Core_Standards.pdf
38. Graham, S., & Perin, S. (2007). A meta-analysis of writing instruction for adolescent students. Journal of Educational Psychology, 99, 445–476
39. Hamp-Lyons, L., & Condon, W. (2000). Assessing the portfolio: Principles for practice, theory and research. Creskill, NJ: Hampton Press
40. Harrington, S., Rhodes, K., Fischer, R. O., & Malenczyk, R. (Eds.). (2005). The outcomes book: Debate and consensus after the WPA outcomes statement. Logan, UT: Utah State University Press
41. Hayes, J. R. (2012). Modeling and remodeling writing. Written Communication, 29, 369–388
42. Herrington, A., & Moran, C. (2012). Writing to a machine is not writing at all. In N. Elliot & L. Perelman (Eds.), Writing assessment in the 21st century: Essays in honor of Edward M. White (pp. 219–232). New York, NY: Hampton Press
43. Hillocks, G. (1986). Research on written composition: New directions for teaching. Urbana, IL: National Council of Teachers of English
44. Hirschmann, N. J. (1992). Rethinking obligation: A feminist method for political theory. Ithaca, NY: Cornell University Press
45. Huot, B. (1996). Towards a new theory of writing assessment. College Composition and Communication, 47, 549–566
46. Kahenman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux
47. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed). (pp. 17–64). Westport, CT: American Council on Education/Praeger
48. Klobucar, A., Elliot, N., Deess, P., Rudniy, O., & Joshi, K. (2013). Automated scoring in context: Rapid assessment for placed students. Assessing Writing, 18, 62–84
49. Kobrin, J. L., Patterson, B. F., Shaw, E. J., Mattern, K. D., & Barbuti, S. M. (2008). Validity of the SAT® for predicting first-year college grade point average (College Board Report 2008–5). Retreived from http://professionals.collegeboard.com
50. Latour, B. (1999). Pandora’s hope: Essays on the reality of science studies. Cambridge, MA: Harvard University Press
51. Lissitz, R. W. (Ed.) (2009). The concept of validity: Revisions, new directions, and applications. Charlotte, NC: Information Age Publishing
52. Lynne, P. (2004). Coming to terms: A theory of writing assessment. Logan, UT: Utah State University Press
53. Messick, S. (1979). Potential uses of noncognitive measurement in education. Journal of Educational Psychology, 71, 281–292
54. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 134–103). New York, NY: American Council on Education and Macmillan
55. Mislevy, R. J. (2007). Validity by design. Educational Researcher, 36, 463–469
56. Mumford, L. (1934). Technics and civilization. New York, NY: Harcourt, Brace
57. National Center for Education Statistics (2012). The nation’s report card: Writing 2011 (NCES 2012–470). Washington, DC: Institute of Education Sciences, U.S. Department of Education. Retrieved from http://nces.ed.gov/nationsreportcard/pdf/main2011/2012470.pdf
58. O’Connor, M., & Paunonen, S. (2007). Big five personality predictors of post-secondary academic performance. Personality and Individual Differences, 43, 971–990
59. Palmer, O. (1961). Sense or nonsense? The objective testing of English composition. English Journal, 50, 314–320
60. Perl, S. (1979). The composing process of unskilled college writers. Research in the Teaching of English, 13, 317–336
61. Powers, D. E., Burstein, J., Chodorow, M., Fowles, M.E., & Kukich, K. (2001). Stumping erater: Challenging the validity of automated essay scoring. Princeton, NJ: Educational Testing Service
62. Ramineni, C. (2013). Validating automated essay scoring for online writing placement: A case study. Assessing Writing, 18, 40–61
63. Sabatini, J. P., Bennett, R. E., & Deane, P. (2011). Four years of cognitively based assessment of, for, and as learning (CBAL): Learning about through-course assessment (TCA). Princeton, NJ: Educational Testing Service
64. Scott-Clayton, J. (2012). Do high-stakes placement exams predict college success? (CCRC Working Paper 41). New York, NY: Community College Research
65. Shanahan, J. G., Qu, Y., & Wiebe, J. (Eds.). (2006). Computing attitude and affect in text: Theory and applications. Berlin, Germany: Springer
66. Shermis, M. D., & Hamner, B. (2012). Contrasting state-of-the-art automated scoring of essays: Analysis. Paper presented at the annual meeting of the National Council of Measurement in Education, Vancouver, BC, Canada. Retrieved from http://www.scoreright.org/NCME_2012_Paper3_29_12.pdf
67. Sireci, S. G., & Parker, P. (2006). Validity on trial: Psychometric and legal conceptualizations of validity. Educational Measurement: Issues and Practices, 25, 27–34
68. Slomp, D. H. (2012). Challenges in assessing the development of writing ability: Theories, constructs and methods. Assessing Writing, 17, 81–91
69. Smith, W. L. (1992). The importance of teacher knowledge in composition placement testing. In R. J. Hayes (Ed.), Reading empirical research studies: The rhetoric of research (pp. 289–316). Hillsdale, NJ: Erlbaum
70. Sternglass, M. (1997). Time to know them: A longitudinal study of writing and learning at the college level. Mahwah, NJ: Lawrence Erlbaum
71. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., & Stede, M. (2011). Lexicon-based method for sentiment analysis. Computational Linguistics, 37(2), 267–307
72. Toulmin, S. E. (1958). The uses of argument. Cambridge, UK: Cambridge University Press
73. Toulmin, S. E. (2001). Return to reason. Cambridge, MA: Harvard University Press
74. U. S. Department of Education. (2012). Race to the Top Assessment Program. May 25. Retrieved from http://www2.ed.gov/programs/racetothetop-assessment/index.html
75. Wardle, E., & Roozen, K. (2012). Addressing the complexity of writing development: Toward an ecological model of assessment. Assessing Writing, 17, 106–119
76. Warschauer, M., & Grimes, D. (2008). Automated writing in the classroom. Pedagogies: An International Journal, 3, 22–26
77. Wells, J. B., Christiansen, M. H, Race, D. S., Acheson, D. J., & MacDonald, M. (2009). Experience and sentence processing: Statistical learning and relative clause comprehension. Cognitive Psychology, 58, 250–271
78. White, E. M. (1985). Teaching and assessing writing: Recent advances in understanding, evaluating, and improving student performance. San Francisco, CA: Jossey Bass
79. White, E. M. (2001). The opening of the modern era of writing assessment: A narrative. College English, 63, 306–320
80. Whithaus, C. (2006). Always already: Automated essay scoring and grammar checkers in college writing courses. In P. E. Ericsson & R. Haswell (Eds.), Machine scoring of student essays: Truth and consequences (pp. 166–176). Logan, UT: Utah State University Press
81. Williamson, D. M., Xi, X., & Breyer, F. J. (2012). A framework for evaluation and use of automated scoring. Educational Measurement: Issues and Practices, 31, 2–13
82. Willingham, W. W., Pollack, J. M., & Lewis, C. (2002). Grades and test scores: Accounting for observed differences. Journal of Educational Measurement, 39, 1–97
83. Winner, L. (1980). Do artifacts have politics? Daedalus, 109, 121–136
84. Wiseman, C. S. (2012). Rater effects: Ego engagement in rater decision-making. Assessing Writing, 17, 150–173
85. Yancey, K. B. (2012). The rhetorical situation of writing assessment: Exigence, location, and the making of knowledge In N. Elliot & L. Perelman (Eds.), Writing assessment in the 21st century: Essays in honor of Edward M. White (pp. 475–492). New York, NY: Hampton Press