Cognitive differences in proficient and nonproficient essay scorers

Written Communication

Volumn 15, Issue 4, 1998, Pages 465-492

  Wolfe, Edward W ^a   Kao, Chi Wen ^a   Ranney, Michael ^a  

^a UNIVERSITY OF FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33749847596     PISSN: 07410883     EISSN: 15528472     Source Type: Journal    
DOI: 10.1177/0741088398015004002     Document Type: Article

References (33)

1. American College Testing Program. (1994). Local scaring guide: Wth Grade Writing Assessment. Iowa City, IA: Author.
2. Barritt, L., Stock, P. L., & Clark, F. (1986). Researching practice: Evaluating assessment essays. College Composition and Communications, 37,315-327.
3. Breland, H. M. (1983). The direct assessment of writing skill: A measurement review (College Board Report No. 83-6). New York: College Entrance Examination Board.
4. Breland, H. M., & Jones, R. J. (1984). Perceptions of writing skills. Written Communication, 1,101-119.
5. Brennan, R. L. (1992). Elements ofgeneralizability theory (2nd ed.). Iowa City, IA: ACT.
6. Chamey, D. (1984). The validity of using holistic scoring to evaluate writing: A critical overview. Research in the Teaching of English, 18,65-81.
7. Cohen, J. (1988). Statistical power analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8. Diederich, P., French, J. W., & Carlton, S. (1961). Factors in judgments of writing ability (RB61-15). Princeton, NJ: Educational Testing Service.
9. Engelhard, G. J. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31,93-112.
10. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: MIT Press.
11. Fowles, M. E. (1978). Basic skills assessment: Manual for scoring the writing sample. Analytic scoring/holistic scoring. Princeton, NJ: Educational Testing Service.
12. Frederiksen, J. R. (1992, April). Learning to "see:" Scoring video portfolios or "beyond the hunter-gatherer" in performance assessment. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
13. Frederiksen, J. R., Sipusic, M., Gamoran, M., & Wolfe, E. W. (1992). Video portfolio assessment: A study for the National Board for Professional Teaching Standards. Oakland, CA: Cognitive Science Research Center, Educational Testing Service.
14. Freedman, S. W., & Calfee, R. C. (1983). Holistic assessment of writing: Experimental design and cognitive theory. In P. Mosenthal, L. Tamor, & S. A. Walmsley (Eds.), Research on writing: Principles and methods (pp. 75-98). New York: Longman.
15. Glaser, R., & Chi, M.T.H. (1988). Overview. In M.T.H. Chi, R. Glaser, & M. J. Fair (Eds.), The nature of expertise (pp. xv-xxviii). Hillsdale, NJ: Lawrence Erlbaum.
16. Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6,65-70.
17. Huot, B. A. (1993). The influence of holistic scoring procedures on reading and rating student essays. In M. M. Williamson & B. A. Huot (Eds.), Validating holistic scoring for writing assessment (pp. 206-236). Cresskill, NJ: Hampton.
18. Kirk, R. E. (1995). Experimental design: Procedures for the betevioral sciences (3rd ed.). New York: Brroks/Cole.
19. Lane, S., Liu, M., Ankenmann, R. D., & Stone, C. A. (1996). Generalizability and validity of mathematics performance assessment. Journal of Educational Measurement, 33, 71-92.
20. Liebetrau, A. M. (1983). Measures of association. Newbury Park, CA: Sage.
21. Linacre, J. M. (1989). Many-facet Rasch measurement. Chicago, IL: MESA.
22. Lunz, M. E., Wright, B. D., & Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3,331-345.
23. Moss, P. A. (1994). Can there be validity without reliability? Educational Researdier, 23(2), 5-12.
24. Pula, J. J., & Huot, B. A. (1993). A model of background influences on holistic raters. In M. M. Williamson & B. A. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 237-265). Cresskill, NJ: Hampton.
25. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement, 30,215-232.
26. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86,420-428.
27. Vaughan, C. (1991). Holistic assessment: What goes on in the rater's mind? In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts (pp. 111-125). Norwood, NJ: Ablex.
28. Voss, J. F. & Post, T. A. (1988). On the solving of ill-structured problems. In M.T.H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 261-285). Hillsdale, NJ: Lawrence Erlbaum.
29. White, E. M. (1985). Teaching and assessing writing. San Francisco, CA: Jossey Bass.
30. Wolfe, E. W. (1995). A study of expertise in essay scoring. Unpublished doctoral dissertation, University of California, Berkeley.
31. Wolfe, E. W. (1997). The relationship between essay reading style and scoring proficiency in a psychometric scoring system. Assessing Writing, 4,83-106.
32. Wolfe, E. W., & Feltovich, B. (1994, April). Learning how to rate essays: A study of scorer cognition. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
33. Wolfe, E. W, Gitomer, D., & Carter, R. (1998, April). The influence of changes in assessment design on the psychometric qualities of scores. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.