Diagnosing a Common Rater Halo Effect using the Polytomous Rasch Model

Journal of Applied Measurement

Volumn 12, Issue 3, 2011, Pages 194-211

  Marais, Ida ^a   Andrich, David ^a  

^a UNIVERSITY OF WESTERN AUSTRALIA   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

HALO EFFECT; MULTI-MODAL; POLYTOMOUS; UNIMODAL;

MEASUREMENTS;

ARTICLE; HUMAN; OBSERVER VARIATION; PSYCHOMETRY; RESEARCH; STATISTICAL MODEL;

HUMANS; MODELS, STATISTICAL; OBSERVER VARIATION; PSYCHOMETRICS; RESEARCH;

EID: 84856037817     PISSN: 15297713     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (37)

1. Andersen, E. B. (1977). Sufficient statistics and latent trait models. Psychometrika, 42,69-81.
2. Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 357-374.
3. Andrich, D. (1982). An index of person separation in latent trait theory, the traditional KR-20 index, and the Guttman scale response pattern. Education Research and Perspectives, 9(1), 95-104.
4. Andrich, D. (1985). A latent trait model for items with response dependencies: Implications for test construction and analysis. In S. E. Embretson (Ed.), Test design (pp. 245-275). New York: Academic Press.
5. Andrich, D. (1991). Essay. Review of Rolf Langeheine and Jurgen Rost, Latent Trait and Latent Class Analysis, New York, 1988. Psychometrika, 56, 155-168.
6. Andrich, D. (2004). Understanding resistance to the data-model relationship in Rasch's paradigm: A reflection for the next generation. Journal of Applied Measurement, 3, 325-357.
7. Andrich, D. (2005). Georg Rasch: Mathematician and statistician. In Kimberly Kempf-Leonard (Editor-in-Chief). Encyclopedia of social measurement, Vol. 3 (pp. 299- 306). Amsterdam: Academic Press.
8. Andrich, D. (2006). A report to the Curriculum Council of Western Australia regarding assessment for tertiary selection. http://www.curriculum.wa. edu.au/internet/Communications/ Publications/
9. Andrich, D., Sheridan, B., and Luo, G. (2010). RUMM2030 [Computer software]. Perth, WA, Australia: RUMM Laboratory.
10. Embretson, S. E., and Reise, S. P. (2000) Item response theory for psychologists. London: Erlbaum.
11. Engelhard, G., Jr. (1992). The measurement of writing ability with a many-faceted Rasch model. Applied Measurement in Education, 5(3), 171-191.
12. Engelhard, G., Jr. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31(2), 93-112.
13. Fisicaro, S. A., and Lance, C. E. (1990). Implications of three causal models for the measurement of halo error. Applied Psychological Measurement, 14,419-429.
14. Fisicaro, S. A., and Vance, R. J. (1994). Comments on the Measurement of Halo. Educational and Psychological Measurement, 54, 366-371.
15. Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., and Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.
16. nd ed.). New York: McGraw-Hill.
17. Humphry, S., and Heldsinger, S. (2009). Do rubrics help to inform and direct teaching practice? In M. O'Keefe, E. Webb, and K. A. Hoad (Eds.), ACER Research Conference series (pp. 57-62). Sydney, NSW: ACER.
18. Hambleton, R. K., Swaminathan, H., and Rogers, J. (1991) Fundamentals of IRT. Thousand Oaks, CA: Sage.
19. Johnson, D. M., and Vidulich, R. N. (1956). Experimental manipulation of the halo effect. The Journal of Applied Psychology, 40(2), 130-134.
20. Lazarsfeld, P. F., and Henry, N. W. (1968). Latent structure analysis. Boston: Houghton Mifflin.
21. Linacre, J. M. (1989). Many-faceted Rasch measurement. Chicago: MESA Press.
22. Linacre, J. M., and Wright, B. D. (2002). Construction of measures from many-facet data. Journal of Applied Measurement, 3,486-512.
23. Lunz, M. E., Wright, B. D., and Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3, 331-345.
24. Luo, G., and Andrich, D. (2004). Weighted likelihood estimation of person locations in an unfolding model for polytomous responses. In D. Banks, L. House, F. R. McMorris, P. Arabie, and W. Gaul (Eds.). Classification, clustering, and data mining applications. Proceedings of the meeting of the International Federation of Classification Societies (IFCS) (pp. 241-250). New York: Springer.
25. Magno, C. (2009) Demonstrating the difference between classical test theory and item response theory using derived test data. The International Journal of Educational and Psychological Assessment, 7(1), 1-11.
26. Marais, I., and Andrich, D. A. (2008a). Formalising dimension and response violations of local independence in the unidimensional Rasch model. Journal of Applied Measurement, 9, 200-215.
27. Marais, I., and Andrich, D. A. (2008b). Effects of varying magnitude and patterns of response dependence in the unidimensional Rasch model. Journal of Applied Measurement, 9, 105-124.
28. Myford, C. M, and Wolfe, E. W. (2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part II. In E. V. Smith, Jr., and R. M. Smith (Eds.), Introduction to Rasch measurement (pp. 518-574), Maple Grove, MN: JAM Press.
29. Newcomb, T. (1931). An experiment designed to test the validity of a rating technique. Journal of Educational Psychology, XXII, 279-289.
30. Rasch, G. (1961). On general laws and the meaning of measurement in psychology. In J. Neyman (Ed.). Proceedings of the fourth Berkeley symposium on mathematical statistics and probability. IV, (pp.321-334). Berkeley CA: University of California Press.
31. Smith, E. (2005). Effect of item redundancy on Rasch item and person estimates. Journal of Applied Measurement, 6, 147-163.
32. Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25-29.
33. van der Linden, W., and Hambleton, R. (Eds.) (1997). Handbook of modern item response theory. New York: Springer.
34. Warm, T. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.
35. Wright, B. D. (2003). Rack and stack: Time 1 vs. time 2. Rasch Measurement Transactions, 77(1), 905-906.
36. Wright, B. D., and Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.
37. Zenisky, A. L., Hambleton, R. K., and Sireci, S. G., (2002). Identification and evaluation of local item dependencies in the medical college admissions test. Journal of Educational Measurement, 39, 291-309.