Evaluating the validity of educational rating data

Educational and Psychological Measurement

Volumn 59, Issue 1, 1999, Pages 25-37

  Harwell, Michael ^a  

^a UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0033249395     PISSN: 00131644     EISSN: None     Source Type: Journal    
DOI: 10.1177/0013164499591002     Document Type: Article

References (35)

1. American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1985). Standards for educational and psychological tests (3rd ed.). Washington, DC: APA.
2. Baxter, G. P., Shavelson, R. J., Goldman, S. R., & Pine, J. (1992). Evaluation of procedure-based scoring for hands-on science assessment. Journal of Educational Measurement, 29, 1-17.
3. Beaton, A. E. (1987). Implementing the new design: The NAEP 1983-84 technical report (No. 15-TR-20). Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.
4. Bennett, R. E., Rock, D. A., & Wang, M. (1991). Equivalence of free-response and multiple-choice items. Journal of Educational Measurement, 28, 77-92.
5. Braun, H. (1988). Understanding scorer reliability: Experiments in calibrating essay readers. Journal of Educational Statistics, 13, 1-18.
6. Breland, H. M., Danos, D. O., Kahn, H. D., Kubota, M. Y., & Bonner, M. W. (1994). Performance versus objectives testing and gender: An exploratory study of an advanced placement history examination. Journal of Educational Measurement, 31, 275-293.
7. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research on teaching. In N. L. Gage (Ed.), Handbook of Research on Teaching (pp. 171-246). Chicago: Rand McNally.
8. Cannella, G. S. (1992). Gender composition and conflict in dyadic sociocognitive interaction: Effects on spatial learning in young children. Journal of Experimental Education, 61, 29-41.
9. Carpenter, T. P., Fennema, E., Peterson, P. L., Chiang, C., & Loef, M. (1989). Using knowledge of children's mathematics thinking in classroom teaching: An experimental study. American Educational Research Journal, 26, 499-531.
10. Chase, C. I. (1968). The impact of some obvious variables on essay test scores. Journal of Educational Measurement, 3, 315-318.
11. Chase, C. I. (1979). Impact of achievement expectations and handwriting quality on scoring essay tests. Journal of Educational Measurement, 16, 39-42.
12. Chase, C. I. (1986). Essay test scoring: Interaction of relevant variables. Journal of Educational Measurement, 23, 33-41.
13. Coffman, W. E. (1971). Essay examinations. In R. L. Thorndike (Ed.), Educational measurement (2nd. ed.). Washington, DC: American Council on Education.
14. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimental: Design & analysis issues for field settings. Boston: Houghton Mifflin.
15. Englehard, G. (1994). Examining rater errors in the assessment of written composition with a many-faceted Rasch model. Journal of Educational Measurement, 31, 93-112.
16. Fuchs, L. S., Fuchs, D. Bentz, J., Phillips, N. B., & Hamlett, C. L. (1994). The nature of student interactions during peer tutoring with and without prior training and experience. American Educational Research Journal, 31, 75-103.
17. Gay, L. R., & Gallagher, P. D. (1976). The comparative effectiveness of tests versus written exercises. Journal of Educational Research, 70, 59-61.
18. Huot, B. (1990). The literature of direct writing assessment: Major concerns and prevailing trends. Review of Educational Research, 60, 237-263.
19. Julian, M. W., & Searcy, C. A. (1995, April). Regression procedures for the detection of rater effects in direct writing assessment. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
20. Keppel, G. (1991). Design and analysis: A researcher's handbook (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
21. King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as strategies for learning from lectures. American Educational Research Journal, 29, 303-323.
22. Knudson, R. E. (1991). Effects of instructional strategies, grade, and sex on students's persuasive writing. Journal of Experimental Education, 59, 141-152.
23. LeMahieu, P. G., Gitomer, D. H., & Eresh, J. T. (1995). Portfolios in large-scale assessment: Difficult but not impossible. Educational measurement: Issues and practice, 14, 11-28.
24. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental tests scores. Reading, MA: Addison-Wesley.
25. Martinez, M. E. (1991). Comparison of multiple-choice and constructed figural response items. Journal of Educational Measurement, 28, 131-145.
26. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). New York: Macmillan.
27. Moss, P. A., Cole, N. S., & Khampalikit, C. (1982). A comparison of procedures to assess written language skills at grades 4, 7, and 10. Journal of Educational Measurement, 19, 37-47.
28. Powers, D. E., Fowles, M. E., Farnum, M., & Ramsey, P. (1994). Will they think less of my handwritten essay if others word process theirs? Effects on essay scores of intermingling handwritten and word-processed essays. Journal of Educational Measurement, 31, 220-233.
29. Raymond, M. R., & Viswesvaran, C. (1993). Least squares models to correct for rater effects in performance assessment. Journal of Educational Measurement, 30, 253-268.
30. Ruiz-Primo, M. A., Baxter, G. P., & Shavelson, R. J. (1993). On the stability of performance assessments. Journal of Educational Measurement, 30, 41-53.
31. Saal, F. E., Downey, R. G., & Lahey, M. A. (1980). Rating the ratings: Assessing the psychometric quality of rating data. Psychological Bulletin, 88, 413-428.
32. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
33. Taylor, L. K., Cook, P. F., Green, E. E., & Rogers, J K. (1988). Better interviews: The effects of supervisor training on listening and collaborative skills. The Journal of Educational Research, 82, 89-95.
34. Webb, N. M., Rowley, G. L., & Shavelson, R. J. (1989). Using generalizability theory in counseling and development. Measurement and Evaluation in Counseling and Development, 21, 81-90.
35. Zoref, L., & Williams, P. (1980). A look at content bias in IQ tests. Journal of Educational Measurement, 17, 313-321.