Reliability in Classroom Assessment

SAGE Handbook of Research on Classroom Assessment

Volumn , Issue , 2013, Pages 107-124

  Parkes, Jay ^a  

^a UNIVERSITY OF NEW MEXICO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84894192988     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4135/9781452218649.n7     Document Type: Chapter

References (102)

1. Allchin, D. (1998). Values in science and in science education. In B. J.Fraser & K. G.Tobin (eds.), International handbook of science education (pp. 1083-1092). Dordrecht, Netherlands: Kluwer Academic Publishers.
2. American Federation of Teachers, National Council on Measurement in Education, & National Education Association. (1990). Standards for teacher competence in educational assessment of students. Retrieved from http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED323186
3. Attali, Y., & Burstein, J. (2006). Automated essay scoring with E-Rater V.2. The Journal of Technology, Learning, and Assessment. Retrieved from http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.173.2655
4. Baird, J.-A., Greatorex, J., & Bell, J. F. (2004). What makes marking reliable? Experiments with UK examinations. Assessment in Education: Principles, Policy and Practice, 11(3), 331-348.
5. Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practices, 18(1), 5-25.
6. Black, P. (2000). Research and the development of educational assessment. Oxford Review of Education, 26 (3/4), 407-419.
7. Borg, W. R., Worthen, B. R., & Valcarce, R. W. (1986). Teachers' perceptions of the importance of educational measurement. The Journal of Experimental Education, 55(1), 9-14.
8. Brennan, R. L. (1998). Misconceptions at the intersection of measurement theory and practice. Educational Measurement: Issues and Practice, 17(1), 5-9.
9. Brennan, R. L. (2001). An essay on the history and future of reliability from the perspective of replications. Journal of Educational Measurement, 38(4), 295-317.
10. Brennan, R. L. (2006). Educational measurement. Westport, CT: American Council on Education/Praeger.
11. Brennan, R. L., & Johnson, E. G. (1995). Generalizability of performance assessments. Educational Measurement: Issues and Practice, 14(4), 9-12, 27.
12. Brimi, H. M. (2011). Reliability of grading high school work in English. Practical Assessment, Research & Evaluation, 16(17). Retrieved from http://pareonline.net/getvn.asp?v=16&n=17
13. Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5-12.
14. Brookhart, S. M. (2007). Expanding views about formative classroom assessment: A review of the literature. In J. H.McMillan (ed.), Formative classroom assessment: Theory into practice (pp. 43-62). New York: Teachers College Press.
15. Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3-12.
16. Burt, C. (1945). The reliability of teachers' assessments of their pupils. British Journal of Educational Psychology, 15(2), 80-92.
17. Camburn, E., & Barnes, C. A. (2004). Assessing the validity of a language arts instruction log through triangulation. Elementary School Journal, 105(1), 49-74.
18. Campbell, C., & Collins, V. L. (2007). Identifying essential topics in general and special education introductory assessment textbooks. Educational Measurement: Issues and Practice, 26(1), 9-18.
19. Campbell, C., & Evans, J. A. (2000). Investigation of preservice teachers' classroom assessment practices during student teaching. The Journal of Educational Research, 93(6), 350-355.
20. Carter, K. (1984). Do teachers understand principles for writing tests? Journal of Teacher Education, 35(6), 57-60.
21. Chase, C. I. (1999). Contemporary assessment for educators. New York: Longman.
22. Cizek, G. J., Fitzgerald, S. M., & Rachor, R. E. (1995). Teachers' assessment practices: Preparation, isolation and the kitchen sink. Educational Assessment, 3(2), 159-179.
23. Culham, R. (2003). 6 + 1 traits of writing: The complete guide. New York: Scholastic.
24. Daniel, L. G., & King, D. A. (1998). Knowledge and use of testing and measurement literacy of elementary and secondary teachers. The Journal of Educational Research, 91(6), 331-344.
25. Delandshere, G., & Petrosky, A. R. (1998). Assessment of complex performances: Limitations of key measurement assumptions. Educational Researcher, 27(2), 14-24.
26. Diederich, P. B., French, J. W., & Carlton, S. T. (1961). Factors in judgments of writing ability. Princeton, NJ: Educational Testing Service.
27. Earl, L. M. (2003). Assessment as learning: Using classroom assessment to maximize learning. Thousand Oaks, CA: Corwin Press.
28. Eells, W. C. (1930). Reliability of repeated grading of essay type examinations. Journal of Educational Psychology, 21(1), 48-52.
29. Ennis, R. L. (2000). Test reliability: A practical exemplification of ordinary language philosophy. Philosophy of Education 1999, 242-248. Champaign, IL: Philosophy of Education Society.
30. Fleming, M., & Chambers, B. (1983). Teacher-made tests: Windows on the classroom. New Directions for Testing and Measurement, 19, 29-38.
31. Gamaroff, R. (2000). Rater reliability in language assessment: The bug of all bears. System, 28(1), 31-53.
32. Gao, X., Shavelson, R. J., & Baxter, G. P. (1994). Generalizability of large-scale performance assessments in science: Promises and problems. Applied Measurement in Education, 7(4), 323-342.
33. Gullickson, A. R. (1986). Teacher education and teacher-perceived needs in educational measurement and evaluation. Journal of Educational Measurement, 23(4), 347-354.
34. Gullickson, A. R., & Ellwein, M. C. (1985). Post hoc analysis of teacher-made tests: The goodness-of-fit between prescription and practice. Educational Measurement: Issues and Practice, 4(1), 15-18.
35. Gullickson, A. R., & The Joint Committee on Standards for Educational Evaluation. (2002). The student evaluation standards: How to improve evaluations of students. Thousand Oaks, CA: Corwin Press.
36. Haertel, E. (2006). Reliability. In R. L.Brennan (ed.), Educational measurement (Vol. 4, pp. 65-110). Westport, CT: American Council on Education/Praeger.
37. Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309-334.
38. Hammond, E. L. (1927). A study of the reliability of an objective examination in ninth-grade English. The School Review, 35(1), 45-51.
39. Harlen, W. (2005). Trusting teachers' judgment: Research evidence of the reliability and validity of teachers' assessment used for summative purposes. Research Papers in Education, 20(3), 245-270.
40. Haynie, W. J. (1992). Post hoc analysis of test items written by technology education teachers. Journal of Technology Education, 4(1), 26-38.
41. Impara, J. C., & Plake, B. S. (1995). Comparing counselors', school administrators' and teachers' knowledge in student assessment. Measurement and Evaluation in Counseling and Development, 28(2), 78-87.
42. Johnson, D. W., & Johnson, R. T. (2004). Assessing students in groups: Promoting group responsibility and individual accountability. Thousand Oaks, CA: Corwin Press.
43. Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144.
44. Jordan, S., & Mitchell, T. (2009). E-assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40(2), 371-385.
45. Kane, M. (1996). The precision of measurements. Applied Measurement in Education, 9(4), 355-379.
46. Kane, M. (2011). The errors of our ways. Journal of Educational Measurement, 48(1), 12-30.
47. Kelly, F. J. (1914). Teachers' marks: Their variability and standardization. New York: Teachers College Contributions to Education.
48. Koh, K. H. (2011). Improving teachers' assessment literacy through professional development. Teaching Education, 22(3), 255-276.
49. Koretz, D., Stecher, B., Klein, S., & McCaffrey, D. (1994). The Vermont portfolio assessment program: Findings and implications. Educational Measurement: Issues and Practice, 13(3), 5-16.
50. Li, H. (2003). The resolution of some paradoxes related to reliability and validity. Journal of Educational and Behavioral Statistics, 28(2), 89-95.
51. Linn, R. L. (1990). Essentials of student assessment: From accountability to instructional aid. Teachers College Record, 91(3), 424-436.
52. Lukin, L. E., Bandalos, D. L., Eckhout, T. J., & Mickelson, K. (2004). Facilitating the development of assessment literacy. Educational Measurement: Issues and Practice, 23(2), 26-32.
53. Mabry, L. (1995). Naturally occurring reliability. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
54. Marso, R. N., & Pigge, F. L. (1991). An analysis of teacher-made tests: Item types, cognitive demands, and item construction errors. Contemporary Educational Psychology, 16(3), 279-286.
55. McKenzie, J. W. (2005). Assessment requirements for school personnel in NCATE institutions. Pullman, WA: Washington State University.
56. McMillan, J. H. (2000). Fundamental assessment principles for teachers and school administrators. Practical Assessment, Research & Evaluation, 7(8). Retrieved from http://www.eric.ed.gov/ERICWebPortal/detail?accno=EJ638496
57. McMillan, J. H. (2003). Understanding and improving teachers' classroom assessment decision making: Implications for theory and practice. Educational Measurement: Issues and Practice, 22(4), 34-43.
58. McMillan, J. H. (2007). Formative classroom assessment. New York: Teachers College Press.
59. McMorris, R. F., & Boothroyd, R. A. (1993). Tests that teachers build: An analysis of classroom tests in science and mathematics. Applied Measurement in Education, 6(4), 321-342.
60. McWilliam, R. A., & Ware, W. B. (1994). The reliability of observations of young children's engagement: An application of generalizability theory. Journal of Early Intervention, 18(1), 34-47.
61. Mertler, C. A. (2003, October). Preservice versus inservice teachers' assessment literacy: Does classroom experience make a difference? Paper presented at the annual meeting of the MidWestern Educational Research Association, Columbus, OH.
62. Mertler, C. A. (2004). Secondary teachers' assessment literacy: Does classroom experience make a difference? American Secondary Education, 33(1), 49-64.
63. Messick, S. J. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.
64. Messick, S. J. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749.
65. Mislevy, R. J. (2004). Can there be reliability without "reliability?" Journal of Educational and Behavioral Statistics, 29(2), 241-244.
66. Moss, P. A. (1994). Can there be validity without reliability? Educational Researcher, 23(2), 5-12.
67. Moss, P. A. (2004). The meaning and consequences of "reliability." Journal of Educational and Behavioral Statistics, 29(2), 245-249.
68. Moss, P. A., Pullin, D., Gee, J. P., & Haertel, E. H. (2005). The idea of testing: Psychometric and sociocultural perspectives. Measurement: Interdisciplinary Research and Perspectives, 3(2), 63-83.
69. Munoz, D. (2009). Reliability as a context-dependent requirement for writing proficiency assessment. Language Studies Working Papers, 1, 46-54.
70. Nystrand, M., Cohen, A. S., & Dowling, N. M. (1993). Addressing reliability problems in the portfolio assessment of college writing. Educational Assessment, 1(1), 53-70.
71. O'Sullivan, R. G., & Chalnick, M. K. (1991). Measurement-related course work requirements for teacher certification and recertification. Educational Measurement: Issues and Practice, 10(1), 17-19, 23.
72. Onocha, C. O., & Okpala, P. N. (1991). Teachers' perceptions of the relevance of topics taught in educational measurement courses. Educational Research, 33(3), 228-232.
73. Parkes, J. (2000). The interaction of assessment format and examinees' perceptions of control. Educational Research, 42(2), 175-182.
74. Parkes, J. (2001). The role of transfer in task variability in performance assessments. Educational Assessment, 7(2), 143-164.
75. Parkes, J. (2007). Reliability as argument. Educational Measurement: Issues and Practice, 26(4), 2-10.
76. Pitts, J., Coles, C., Thomas, P., & Smith, F. (2002). Enhancing reliability in portfolio assessment: Discussions between assessors. Medical Teacher, 24(2), 197-201.
77. Reckase, M. D. (1995). Portfolio assessment: A theoretical estimate of score reliability. Educational Measurement: Isses and Practice, 14(1), 12-14, 31.
78. Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435-448.
79. Rudner, L. M., & Shafer, W. D. (2002). Reliability. In L. M.Rudner & W. D.Shafer (eds.), What teachers need to know about assessment (pp. 17-23). Washington, DC: National Education Association.
80. Rugg, H. O. (1918). Teachers' marks and the reconstruction of the marking system. The Elementary School Journal, 18(9), 701-719.
81. Sawilowsky, S. S. (2000a). Reliability: Rejoinder to Thompson and Vacha-Haase. Educational and Psychological Measurement, 60(2), 196-200.
82. Sawilowsky, S. S. (2000b). Psychometrics versus datametrics: Comment on Vacha-Haase's "reliability generalization" method and some EPM editorial policies. Educational and Psychological Measurement, 60(2), 157-173.
83. Shavelson, R. J., & Baxter, G. P. (1992). What we've learned about assessing hands-on science. Educational Leadership, 49(8), 20-25.
84. Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14.
85. Shepard, L. A. (2006). Classroom assessment. In R. L.Brennan (ed.), Educational measurement (4th ed.). Westport, CT: American Council on Education/Praeger.
86. Slomp, D. H., & Fuite, J. (2004). Following Phaedrus: Alternate choices in surmounting the reliability/validity dilemma. Assessing Writing, 9(3), 190-207.
87. Smith, J. K. (2003). Reconsidering reliability in classroom assessment and grading. Educational Measurement: Issues and Practice, 22(4), 26-33.
88. Starch, D., & Elliott, E. C. (1912). Reliability of grading high-school work in English. The School Review, 20(7), 442-457.
89. Starch, D., & Elliott, E. C. (1913a). Reliability of grading work in history. The School Review, 21(10), 676-681.
90. Starch, D., & Elliott, E. C. (1913b). Reliability of grading work in mathematics. The School Review, 21(4), 254-259.
91. Stiggins, R. J. (1991). Assessment literacy. Phi Delta Kappan, 72(7), 534-539.
92. Stiggins, R. J. (1999). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practice, 18(1), 23-27.
93. Stiggins, R. J. (2001). The unfulfilled promise of classroom assessment. Educational Measurement: Issues and Practice, 20(3), 5-15.
94. Stigler, S. M. (1986). The history of statistics: The measurement of uncertainty before 1900. Cambridge, MA: Belknap Press of Harvard University Press.
95. Stuhlmann, J., Daniel, C., Dellinger, A., Denny, R., & Powers, T. (1999). A generalizability study of the effects of training on teachers' abilities to rate children's writing using a rubric. Journal of Reading Psychology, 20(2), 107-127.
96. Suen, H. K., & French, J. L. (2003). A history of the development of psychological and educational testing. In C. R.Reynolds & R.Kamphaus (eds.), Handbook of psychological and educational assessment of children: Intelligence, aptitude, and achievement (Vol. 2, pp. 3-23). New York: Guilford Press.
97. Suen, H. K., Logan, C., Neisworth, J., & Bagnato, S. (1995). Parent/professional congruence: Is it necessary? Journal of Early Intervention, 19(3), 257-266.
98. Thompson, B., & Vacha-Haase, T. (2000).
99. Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60(2), 174-195.
100. Valentin, J. D. (2005, August). Ascertaining the reliability and content-related validity of mathematics tests constructed by teachers: A snapshot in the primary schools in Seychelles. Paper presented at the Third East Asia Regional Conference on Mathematics Education, Shanghai.
101. Way, W. D., Dolan, R. P., & Nichols, P. (2010). Psychometric challenges and opportunities in implementing formative assessment. In H. L.Andrade & G. J.Cizek (eds.), Handbook of formative assessment (pp. 297-315). New York: Routledge.
102. Wiggins, G. (1989). A true test: Toward more authentic and equitable assessment. Phi Delta Kappan, 70(9), 703-13.