Using FACETS to model rater training effects

Language Testing

Volumn 15, Issue 2, 1998, Pages 263-287

  Weigle, Sara Cushing ^a  

^a GEORGIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0002422895     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1177/026553229801500205     Document Type: Article

References (30)

1. Andrich, D. 1978a: A rating formulation for ordered response categories. Psychometrika 43, 561-573.
2. Andrich, D. 1978b: Scaling attitude items constructed and scored in the Likert tradition. Educational and Psychological Measurement 38, 665-680.
3. Barritt, L., Stock, P. and Clarke, F. 1986: Researching practice: evaluating assessment essays. College Composition and Communication 37, 315-327.
4. Charney, D. 1984: The validity of using holistic scoring to evaluate writing: a critical overview. Research in the Teaching of English 18, 65-81.
5. Connor-Linton, J. 1995: Looking behind the curtain: what do L2 composition ratings really mean? TESOL Quarterly 29, 762-765.
6. Cumming, A. 1990: Expertise in evaluating second language compositions. Language Testing 7, 31-51.
7. Diederich, P. B., French, J. W. and Carlton, S. T. 1961: Factors in judgments of writing ability (Research Bulletin 61-15). Princeton, NJ: Educational Testing Service (Eric Document Reproduction Nu. ED 002 172).
8. Ericsson, K. A. and Simon, H. 1984: Verbal reports as data. Cambridge, MA: MIT Press.
9. Freedman, S. W. 1981: Influences on evaluators of expository essays: beyond the text. Research in the Teaching of English 15, 245-255.
10. Gere, A. R. 1980: Written composition: towards a theory of evaluation. College English 42, 44-58.
11. Hamp-Lyons, L. 1990: Second language writing: assessment issues. In Kroll, B., editor, Second language writing: research insights for the classroom, Cambridge: Cambridge University Press, 69-87.
12. Huot, B. 1988: The validity of holistic scoring: a comparison of the talk-aloud protocols of expert and novice holistic raters. Unpublished doctoral dissertation, Indiana University of Pennsylvania.
13. Huot, B. 1990: Reliability, validity, and holistic scoring: what we know and what we need to know. College Composition and Communication 41, 201-213.
14. Linacre, J. M. 1989: Many-faceted Rasch measurement. Chicago, IL: MESA Press.
15. Linacre, J. M. and Wright, B. D. 1993: A user's guide to FACETS (Version 2.6). Chicago, IL: MESA Press.
16. Linacre, J. M. and Wright, B. D. 1994: A user's guide to FACETS (Version 2.7). Chicago, IL: Mesa Press.
17. Lunz, M. E. and Stahl, J. A. 1990: Judge consistency and severity across grading periods. Evaluation and the Health Professions 13, 425-444.
18. Lunz, M. E., Stahl, J. A. and Wright, B. D. 1991: The invariance of judge severity calibrations. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
19. Lunz, M. E., Wright, B. D. and Linacre, J. M. 1990: Measuring the impact of judge severity on examination scores. Applied Measurement in Education 3, 331-345.
20. Myford, C. M. 1991a: Assessment of acting ability. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
21. Myford, C. M. 1991b: Judging acting ability: the transition from novice to expert. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL
22. Ruth, L. and Murphy, S. 1988: Designing writing tasks for the assessment of writing. Norwood, NJ: Ablex Publishing Corp.
23. Shohamy, E., Gordon, C. and Kraemer, R. 1992: The effect of raters' background and training on the reliability of direct writing tests. Modern Language Journal 76, 27-33.
24. Stahl, J. A. and Lunz, M. E. 1991: Judge performance reports: media and message. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
25. Stock, P. L. and Robinson, J. L. 1987: Taking on testing. English Education 19, 93-121.
26. Tyndall, B. and Kenyon, D. 1996: Validation of a new holistic rating scale using Rasch multi-faceted analysis. In Cumming, A. and Berwick, R., editors, Validation in language testing, Clevedon, Avon: Multilingual Matters, 39-57.
27. Vaughan, C. 1992: Holistic assessment: what goes on in the rater's mind? In Hamp-Lyons, L., editor, Assessing second language writing in academic contexts, Norwood, NJ: Ablex Publishing Corp, 111-126.
28. Weigle, S.C. 1994a: Effects of training on raters of ESL compositions: quantitative and qualitative approaches. Unpublished doctoral dissertation, UCLA.
29. Weigle, S. C. 1994b: Effects of training on raters of ESL compositions. Language Testing 11, 197-223.
30. Weigle, S. C. and Lynch, B. K. 1996: Hypothesis testing in construct validation. In Cumming, A. and Berwick, R., editors, Validation in language testing, Clevedon, Avon.: Multilingual Matters, 58-71.