The stability of marker characteristics across tests of the same subject and across subjects

Journal of Applied Measurement

Volumn 7, Issue 2, 2006, Pages 192-205

  Lamprianou, Iasonas ^a,b  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b NONE   (Cyprus)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; EUROPE; EVALUATION; HISTORY; HUMAN; LANGUAGE; METHODOLOGY; STANDARD;

EUROPE; EVALUATION STUDIES; HISTORY; HUMANS; LANGUAGE; RESEARCH DESIGN;

EID: 33646351041     PISSN: 15297713     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. Banerji, M. (1999). Validation of scores/measures from a K-2 developmental assessment in mathematics. Educational and Psychological Measurement, 59(4), 694-715.
2. Barrett, S. (2001 ). The impact of training on rater variability. International Education Journal, 2, 49-58.
3. Bonk, W. J., and Ockey, G. J. (2003). A manyfacet Rasch analysis of the second language group oral discussion task. Language Testing, 20, 89-110.
4. Braun, H. I. (1988). Understanding score reliability: Experiments in calibrating essay readers. Journal of Educational Statistics, 13, 1-18.
5. Congdon, P. J., and McQueen, J. (2000). The permanence of rater severity in large-scale assessment programs. Journal of Educational Measurement, 37, 163-178.
6. Cushing, S. W. (1998). Using FACETS to model rater training effects. Language Testing, 15, 263-287.
7. Gushing, S. W. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing, 6(2), 145-178.
8. Engelhard, G. (1992). The measurement of writing competence with a many-faceted Rasch model. Applied Measurement in Education, 5(3), 171-191.
9. Engelhard, G. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31, 93-112.
10. Engelhard, G, Jr. (2002). Monitoring raters in performance assessments. In G. Tindal and T. M. Haladyna (Eds.), Large scale assessments for all students: Validity, technical adequacy, and implementation (pp. 261-288). Mahwah, NJ: Lawrence Erlbaum.
11. Engelhard, G, Jr., and Myford, C. M. (2003). Monitoring faculty consultant performance in the Advanced Placement English literature and composition program with a many-faceted Rasch model (College Board Research Report: 2003 - 1). New York: College Entrance Examination Board.
12. Freedman, S.W. (1981): Influences on evaluators of expository essays: beyond the text. Research in the Teaching of English, 15, 245-255.
13. Gamaroff, R. (2000). Rater reliability in language assessment: The bug of all bears. System, 28, 31-53.
14. Greatorex, J., and Bell, F. (2002, August). Does the gender of examiners influence their marking? A paper to accompany a poster presented at the EARLI Special Interest Group on Assessment and Evaluation, University of North Umbria, UK.
15. Hoskens, M., and Wilson, M. (2001). Real-time feedback on rater drift in constructed-response items: An example from the Golden State Examination. Journal of Educational Measurement, 38, 121-145.
16. Kingsbury, F. A. (1922). Analysing ratings and training raters. Journal of Personnel Research, 1, 377-388.
17. Klauer, K. C. (1995). The assessment of person fit. In G. H. Fischer and I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications (pp. 97-110). New York: Academic Press.
18. Lamprianou, I., and Boyle, B. (2004). Accuracy of measurement in the context of mathematics national curriculum tests in England for ethnic minority pupils and pupils who speak English as an additional language. Journal of Educational Measurement, 41, 239-260.
19. nd ed.). Chicago: MESA Press.
20. Linacre, J. M., and Wright, B. D. (1994). Chisquare fit statistics. Rasch Measurement Transactions, 8(2), 360-361.
21. Linacre, J. M., and Wright, B. D. (2002). Construction of measures from many-facet data. Journal of Applied Measurement, 3, 486-512.
22. Lumley, T., Lynch, B. K., and McNamara, T. F. (1994). A new approach to standard-setting in language assessment. Melbourne Papers in Language Testing, 3, 19-40.
23. Lumley, T., and McNamara, T. F. (1995). Reader characteristics and reader bias: Implications for training. Language Testing, 12, 54-71
24. Lunz, M. E., Wright, B. D., and Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3(4), 331-345.
25. Lunz, M. E., Stahl, J. A., and Wright, B. D. (1991, April). The invariance of judge severity calibrations. Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL.
26. Masters, G. N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 49, 269-272.
27. Murphy, K. R., Jako, R. A., and Anhalt, R. L. (1993). Nature and consequences of halo error: A critical analysis. Journal of Applied Psychology, 78(2), 218-225.
28. Myford, C. M., and Wolfe, E. W. (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4, 386-422.
29. Myford, C. M., and Wolfe, E. W. (2004). Detecting and measuring rater effects using manyfacet Rasch measurement: Part II. Journal of Applied Measurement, 5, 189-227.
30. Rae, G, and Hyland, P. (2001). Generalizability and classical test theory analyses of Koppitz's scoring system for human figure drawings. British Journal of Educational Psychology, 71, 369-382.
31. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. (Expanded edition, 1980. Chicago: University of Chicago Press.)
32. Saal, F. E., Downey, R. G., and Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88(2), 413-428.
33. Smith, R. M. (1991). The distributional properties of Rasch item fit statistics. Educational and Psychological Measurement, 51, 541-565.
34. Smith, R. M. (2000). Fit analysis in latent trait measurement models. Journal of Applied Measurement, 1, 199-218.
35. Stahl, J. A., and Lunz, M. E. (1991, April). Judge performance reports: Media and message. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
36. Thorndike, E. L. ( 1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29.
37. Weigle, S. (1998). Using FACETS to model rater training effects. Language Testing, 15, 263-287.
38. Wolfe, E. W. (2004). Identifying rater effects using latent trait models. Psychology Science, 46(1), 35-51.
39. Wright, B. D., and Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.
40. Wright, B. D., and Mok, M. (2000). Rasch models overview. Journal of Applied Measurement, 1, 83-106.
41. Wright, B. D., and Stone, M. H. (1979). Best test design. Chicago: MESA Press.