Using classical and modern measurement theories to explore rater, domain, and gender influences on student writing ability

Journal of Applied Measurement

Volumn 10, Issue 3, 2009, Pages 225-246

  Gyagenda, Ismail S ^a   Engelhard Jr , George ^b  

^a MERCER UNIVERSITY   (United States)

^b EMORY UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; FEMALE; HUMAN; MALE; PSYCHOMETRY; SEX DIFFERENCE; STANDARD; STATISTICS; STUDENT; THEORETICAL MODEL; UNIVERSITY; WRITING;

ALGORITHMS; FEMALE; HUMANS; MALE; MODELS, THEORETICAL; PSYCHOMETRICS; SEX FACTORS; STUDENTS; UNIVERSITIES; WRITING;

EID: 77954682828     PISSN: 15297713     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (42)

1. Afflerbach, P. (1985). The statewide assessment of writing. Princeton, NJ: Educational Testing Service.
2. Andrich, D. (1988). Rasch models for measurement. Newbury Park, CA: Sage.
3. Applebee, A. N., Langer, J. A., Jenkins, L. B., Mullis, I., and Foertsch, M. A. (1990). Learning to write in our nation's schools: Instruction and achievement in 1988 at grade 4,8, and 12. Princeton, NJ: Educational Testing Service.
4. Applebee, A. N., Langer, J. A., and Mullis, I. (1985). Writing: Trends across the decade, 1974-1984. Princeton, NJ: Educational Testing Service.
5. Bond, L. (1995). Unintended consequences of performance assessments: Issues of bias and fairness. Educational Measurement: Issues and Practice, 14(3), 21-24.
6. Bridgeman, B., and McHale, F. (1996). Gender and ethnic group differences on the GMAT Analytical writing assessment. Princeton, NJ: Educational Testing Service. ERIC Reproduction Service. Document No. ED 400330.
7. Campbell, J. R., Voelkl, K., and Donahue, P. L. (1997). Report in brief. NAEP1996 trends in academic progress. Washington, DC: National Center for Education Statistics.
8. Doolittle, A., and Welch, C. (1989). Gender differences in performance on a College-Level achievement test. ERIC Reproduction Service. Document No. ED306 237.
9. Du, Y., Wright, B. D., and Brown, W L. (1996, April). Differential facet functioning detection in direct writing assessment. Paper presented at the annual meeting of the American Educational Research Association, New York. ERIC Reproduction Service. Document No. ED 400 293.
10. Du, Y., and Wright, B. D. (1997). Measuring student writing abilities in a large-scale writing assessment. In M. Wilson, G. Engelhard, and K. Draney (Eds.), Objective measurement: Theory into practice (pp. 1-24). Norwood, NJ: Abex.
11. Engelhard, G. (1992). The measurement of writing ability with a many-faceted Rasch model. Applied Measurement in Education, 5(3), 171-191.
12. Engelhard, G. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31(2), 93-112.
13. Engelhard, G. (1996). Evaluating rater accuracy in performance assessments. Journal of Educational Measurement, 33(1), 56-70.
14. Engelhard, G., Gordon, B., Walker, E. V., and Gabrielson, S. (1994). Writing tasks and gender: Influences on writing quality of black and white students. Journal of Educational Research, 87, 197-209.
15. Fan, X. ( 1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58 (3), 357-381.
16. Gabrielson, S., Gordon, B., and Engelhard, G. (1995). The effects of task choice on the quality of writing obtained in a statewide assessment. Applied Measurement in Education, 8(4), 273-290.
17. Georgia Department of Education. (1993). Georgia high school writing test: Assessment and instructional guide. Atlanta, GA: Author.
18. Green, K. E. ( 1991 ). Measurement theory. In K. E. Green, (Ed.), Educational testing: Issues and applications, (pp. 3-26). New York: Garland Publishing.
19. Hambleton, R. K., and Jones, R. W. (1993). An NCME instructional module on comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38-47.
20. Hedges, L. V., and Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269(5220), 41-45.
21. Huot, B. (1990). The literature of direct writing assessment: Major concerns and prevailing trends. Review of Educational Research, 60(2), 237-263.
22. Hyde, J. S., and Linn, M. C. (1988). Gender differences in verbal ability: A meta-analysis. Psychological Bulletin, 104, 53-69.
23. Linacre, J. M. and Wright, B. D. (1994). A user's guide to FACETS: Rasch measurement computer program. Chicago: MESA Press.
24. Linacre, J. M. (1996). Generalizability theory and many-faceted Rasch masurement. In G. Engelhard, Jr., and M. Wilson (Eds.), Objective measurement: Theory into practice, Vol. 3, (pp. 85-98). Norwood, NJ: Ablex.
25. Linn, R. L., Baker, E. L., and Dunbar, S. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), 15-21.
26. Lunz, M. E., Wright, B. D., and Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3, 331-345.
27. Maccoby E. E., and Jacklin, C. N. (1974). The psychology of sex differences. Stanford, CA: Stanford University Press.
28. MacMillan, P. D. (2000). Classical, generalizability, and multifaceted Rasch detection of interrater variability in large, sparse data sets. Journal of Experimental Education, 68 (2), 167-190.
29. Moon, T. R., and Hughes, K. R. (2002). Training and scoring issues involved in large-scale writing assessments. Educational Measurement: Issues and Practices, 21(2), 15-19.
30. Moore, A. D., and Young, S. (1997). Clarifying the blurred image: Estimating the inter-rater reliability of performance assessments. Paper presented at the annual meeting of the Rocky Mountain Educational Research Association, Jackson, WY. ERIC Reproduction Service. Document No. ED 414 319.
31. Penny, J., Johnson, R. L., and Gordon, B. (2000). Using rating augmentation to expand the scale of an analytic rubric. Journal of Experimental Education, 68 (2), 269-287.
32. Prater, D. L., and Mayo, N. B. (1984). Cognitive development level and syntactic maturity. Journal of Research and Development in Education, 17(3), 1-7.
33. Purves, A. C. ( 1992). Reflections on research and assessment in written composition. Research in the Teaching of English, 26(1), 108-122.
34. Randhawa, B. (1991). Gender differences in academic achievement: A closer look at mathematics. Alberta Journal of Educational Research, 37(3), 241-257.
35. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research. (Expanded edition, 1980. Chicago: University of Chicago Press.)
36. Ruth, L., and Murphy, S. (1988). Designing writing tasks for the assessment of writing. Norwood, NJ: Ablex.
37. Schick, R., DeMasi, M. E., and Green, M. S. (1992). Factors predicting writing performance. In A. C. Purves (Ed.), The IEA study of written composition II: Education and performance in fourteen countries (pp. 153-167). New York: Pergamon Press.
38. Verstralen, H., Bechger, T., and Maris, G. (2001). The combined use of classical test theory and item response theory. Measurement and Research Department Reports. Arnhem, The Netherlands: CITO (retrieved 06/26/2002 at http://www.cito.nl/)
39. Welch, C. J., and Miller, T.R. (1995). Assessing differential item functioning in direct writing assessments: Problems and an example. Journal of Educational Measurement, 32(2), 163-178.
40. Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97-116.
41. Wright, B. D., and Masters, G. N. (1982). Rating scale analysis. Chicago: MESA Press.
42. Zwick, R., Donoghue, J. R., and Grimes, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30(3), 233-251.