The association between SAT prompt characteristics, response features, and essay scores

Assessing Writing

Volumn 16, Issue 3, 2011, Pages 154-169

  Kobrin, Jennifer L ^a   Deng, Hui ^a   Shaw, Emily J ^a  

^a The College Board   (United States)

Author keywords

College admissions testing; Essay tests; SAT

Indexed keywords

EID: 79960055445     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2011.01.001     Document Type: Article

References (63)

1. Albertson B.R. Organization and development features of grade 8 and grade 10 writers: A descriptive study of Delaware Student Testing Program [DSTP] essays. Research in the Teaching of English 2007, 41(4):435-464.
2. Barkaoui K. Participants, texts, and processes in ESL/EFL essay tests: A narrative review of the literature. The Canadian Modern Language Review 2007, 64(1):99-134.
3. Baron, D. (2005, May 6). The College Board's new essay reverses decades of progress toward literacy. The Chronicle of Higher Education, pp. B14-B15.
4. Breland, H. M., Bonner, M. W., & Kubota, M. Y. (1995). Factors in performance on brief, impromptu essay examinations (College Board Research Rep. No. 95-4). New York: College Entrance Examination Board.
5. Breland, H. M., Camp, R., Jones, R. J., Morris, M. M., & Rock, D. R. (1987). Assessing writing skill (College Board Research Rep. No. 11). New York: College Entrance Examination Board.
6. Breland, H. M., Danos, D. O., Kahn, H. D., Kubota, M. Y., & Sudlow, M. W. (1991). A study of gender and performance on Advanced Placement history examinations (College Board Research Rep. No. 91-4). New York: College Entrance Examination Board.
7. Breland, H. M., & Jones, R. J. (1988). Remote scoring of essays (College Board Research Rep. 88-3). New York: College Entrance Examination Board.
8. Bridgeman B., Morgan R., Wang M.-M. Choice among essay topic: Impact on performance and validity. Journal of Educational Measurement 1997, 34:273-286.
9. Briggs D. Influence of handwriting on assessment. Educational Research 1970, 13:50-55.
10. Camp R. Changing the model for the direct assessment of writing. Assessing writing: A critical sourcebook 2009, 102-130. Bedford/St. Martins. B. Huot, P. O'Neill (Eds.).
11. Cioffi, F. L. (2005, May 20). Argumentation in a culture of discord. The Chronicle of Higher Education, p. B6.
12. College Board ScoreWrite: A guide to preparing for the new SAT essay 2004, College Entrance Examination Board, New York.
13. College Board. (2006). 2006 college-bound seniors: Total group profile report. New York: College Entrance Examination Board. Retrieved December 2007, from http://professionals.collegeboard.com/data-reports-research/sat/archived/2006.
14. Connor-Linton J. Crosscultural comparison of writing standards: American ESL and Japanese EFL. World Englishes 1995, 14:99-115.
15. Cumming A. Expertise in evaluating second language compositions. Language Testing 1990, 7:31-51.
16. Cumming A., Kantor R., Powers D. Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal 2002, 86:67-96.
17. Daly J.A., Dickson-Marman F. Contrast effects in evaluating essays. Journal of Educational Measurement 1982, 19:309-316.
18. Writing and learning in cross-national perspective 2002, Lawrence Erlbaum Associates. D. Foster, D.R. Russell (Eds.).
19. Hales L.W., Tokar E. The effect of the quality of preceding responses on the grades assigned to subsequent responses to an essay question. Journal of Educational Measurement 1975, 12:115-117.
20. Hamp-Lyons L. What is writing? What is 'scholastic aptitude'? What are the consequences? SAT I writing - a trip down memory lane. Assessing Writing 2005, 10:151-156.
21. Hamp-Lyons L. Second language writing: Assessment issues. Second language writing: Research insights for the classroom 1990, 69-87. Cambridge University Press. B. Kroll (Ed.).
22. Hardison C.M., Sackett P.R. Use of writing samples on standardized tests: Susceptibility to rule-based coaching and the resulting effects on score improvement. Applied Measurement in Education 2008, 21:227-252.
23. Hayward M. Choosing an essay test question: It's more than what you know. Teaching English in the Two-Year College 1989, 16:174-178.
24. Hinkel E. Second language writers' text: Linguistic and rhetorical features 2002, Lawrence Erlbaum Associates.
25. Hoetker J., Brossell G. The effects of systematic variations in essay topics on the writing performance of college freshmen. College Composition and Communication 1989, 40(4):414-421.
26. Hoover, E. (2005, April 22). And on the 11th day, the readers rested. The Chronicle of Higher Education, p. A39.
27. Hughes D.C., Keeling B., Tuck B. The influence of context position and scoring method on essay scoring. Journal of Educational Measurement 1980, 17:131-135.
28. Huot B. The literature of direct writing assessment: Major concerns and prevailing trends. Review of Educational Research 1990, 60(2):237-263.
29. Huot B., O'Neill P., Moore C. A usable past for writing assessment. College English 2010, 72(5):495-517.
30. Jennings M., Fox J., Graves B., Shohamy E. The test-takers' choice: An investigation of the effect of topic on language-test performance. Language Testing 1999, 16:426-456.
31. Kane M.T. Validation. Educational measurement 2006, 18-64. American Council on Education/Praeger, Westport, CT. 4th ed. R.L. Brennan (Ed.).
32. Katzman, J., Lutz, A., & Olson, E. (2004, March). Would Shakespeare get into Swarthmore? The Atlantic Monthly, pp. 97-100.
33. Kirkup, C., Schagen, I., Wheater, R., Morrison, J., & Whetton, C. (2007). Use of an aptitude test in university entrance - a validity study: Relationships between SAT® scores, attainment measures and background variables (Research Report RR846). NFER Trading Ltd.
34. Kobrin J.K., Deng H., Shaw E.J. Does quantity equal quality? The relationship between length of response and scores on the SAT essay. The Journal of Applied Testing Technology 2007, 8(1).
35. Kobrin, J. L., Patterson, B. F., Shaw, E. J., Mattern, K. D., & Barbuti, S. M. (2008). Validity of the SAT for predicting first-year college grade point average (College Board Research Report No. 2008-5). New York: College Entrance Examination Board.
36. Korn E.L., Graubard B.I. Analysis of health surveys 1999, John Wiley & Sons, Inc, New York.
37. Markham L.R. Influences of handwriting quality on teacher evaluation of written work. American Educational Research Journal 1976, 13:277-283.
38. Mattern, K., Camara, W., & Kobrin, J. L. (2007). SAT writing: An overview of research and psychometrics to date. (College Board Research Note No. RN-32). New York: College Entrance Examination Board.
39. Mattern, K. D., Patterson, B. F., Shaw, E. J., Kobrin, J. L., & Barbuti, S. M. (2008). Differential validity and prediction of the SAT. (College Board Research Rep. No. 2008-4). New York: College Entrance Examination Board.
40. Mazzeo, J., Schmitt, A. P., & Bleistein, C. A. (1993). Sex-related differences on constructed-response and multiple-choice sections of advanced placement examinations (College Board Research Report 92-7). New York: The College Board.
41. Moss P.A. Can there be validity without reliability?. Educational Researcher 1994, 23(2):5-12.
42. Noeth, R. J. & Kobrin, J. L. (2007). Writing changes in the nation's K-12 education system. (College Board Research Note No. RN-34). New York: College Entrance Examination Board.
43. Nunnally T.E. Breaking the five-paragraph-theme barrier. The English Journal 1991, 80:67-71.
44. Patterson, B. F., Mattern, K. D., Kobrin, J. L. (2009). Validity of the SAT for predicting FYGPA: 2007 SAT validity sample (College Board Statistical Report). New York: The College Board.
45. Penny J.A. Reading high stakes writing samples: My life as a reader. Assessing Writing 2003, 8:192-215.
46. Perelman, L. (2005, May 29). New SAT: Write long, badly, and prosper. Los Angeles Times, p. 5.
47. Pomplun, M., Wright, D., Oleka, N., & Sudlow, M. (1992). An analysis of English Composition Test essay prompts for differential difficulty (College Board Research Report 92-4). New York: The College Board.
48. Popp S.E.O., Ryan J.M., Thompson M.S. The critical role of anchor paper selection in writing assessment. Applied Measurement in Education 2009, 22:255-271.
49. Powers D.E., Fowles M.E., Farnum M., Ramsey P. Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word-processed essays. Journal of Educational Measurement 1994, 31:220-233.
50. Raudenbush S.W., Bryk A.S. Hierarchical linear models: Applications and data analysis methods 2002, Sage Publications, Inc. 2nd ed.
51. Raudenbush S., Bryk A., Congdon R. HLM for Windows (Version 6. 04) [Computer software] 2007, Scientific Software International, Lincolnwood, IL.
52. Roberts, J. K. (2007, April). Group dependency in the presence of small intraclass correlation coefficients: An argument in favor of not interpreting the ICC. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
53. Sacchetti, M. (2005, March 9). For SAT takers a new test; Essay section may be struggle for students. Boston Globe, p. B1.
54. Shi L. Native- and nonnative-speaking EFL teachers' evaluation of Chinese students' English writing. Language Testing 2001, 18:303-325.
55. Skutches, G. (2005, July 24). New SAT essay format is "teaching to the test," not teaching writing, critical thinking [Opinion]. The Morning Call, p. D1.
56. Sloan C.A., McGinnis I. The effect of handwriting on teachers' grading of high school essays. Journal of the Association for the Study of Perception 1982, 17:15-21.
57. Sweedler-Brown C.O. The effect of training on the appearance bias of holistic essay graders. Journal of Research and Development in Education 1992, 26:24-29.
58. Thomas P. The negative impact of testing writing skills. Educational Leadership 2004, 62:76-79.
59. Wesley K. The ill effects of the five paragraph theme. English Journal 2000, 90:57-60.
60. Winerip, M. (2005, May 4). SAT essay test rewards length and ignores errors. The New York Times. Retrieved from http://www.nytimes.com/.
61. Witte S.P., Trachsel M., Walters K. Literacy and the direct assessment of writing: A diachronic perspective. Writing assessment: Issues and strategies 1986, 13-34. Longman, Inc. K.L. Greenberg, H.S. Wiener, R.A. Donovan (Eds.).
62. Wolfe, E. W., & Feltovich, B. (1994, April). Learning to rate essays: A study of scorer cognition. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA (ERIC Document Reproduction Service No. ED368777).
63. Worden D.L. Finding process in product: Prewriting and revision in timed essay responses. Assessing Writing 2009, 14(3):157-177.