Subjectivity of teacher judgments: Exploring student characteristics that influence teacher judgments of student ability

Teaching and Teacher Education

Volumn 65, Issue , 2017, Pages 48-60

  Meissel, Kane ^a   Meyer, Frauke ^a   Yao, Esther S ^a   Rubie Davies, Christine M ^a  

^a UNIVERSITY OF AUCKLAND   (New Zealand)

Author keywords

Social justice; Standardized testing; Student achievement; Teacher judgments

Indexed keywords

EID: 85015747732     PISSN: 0742051X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tate.2017.02.021     Document Type: Article

References (96)

1. Alvidrez, J., Weinstein, R.S., Early teacher perceptions and later student academic achievement. Journal of Educational Psychology 91:4 (1999), 731–746, 10.1037/0022-0663.91.4.731.
2. Angoff, W.H., Criterion-referencing, norm-referencing and the SAT. The College Board Review 92 (1974), 3–5 21.
3. Begeny, J.C., Eckert, T.L., Montarello, S.A., Storie, M.S., Teachers' perceptions of students' reading abilities: An examination of the relationship between teachers' judgments and students' performance across a continuum of rating methods. School Psychology Quarterly, 23(1), 2008, 43.
4. Begeny, J.C., Krouse, H.E., Brown, K.G., Mann, C.M., Teacher judgments of students' reading abilities across a continuum of rating methods and achievement measures. School Psychology Review 40:1 (2011), 23–38.
5. Benner, A.D., Mistry, R.S., Congruence of mother and teacher educational expectations and low-income youth's academic competence. Journal of Educational Psychology 99:1 (2007), 140–153, 10.1037/0022-0663.99.1.140.
6. Bennett, R.E., Gottesman, R.L., Rock, D.A., Cerullo, F., Influence of behavior perceptions and gender on teachers' judgments of students' academic skill. Journal of Educational Psychology 85:2 (1993), 347–356, 10.1037/0022-0663.85.2.347.
7. Beswick, J.F., Willms, J.D., Sloat, E.A., A comparative study of teacher ratings of emergent literacy skills and student performance on a standardized measure. Education 126 (2005), 116–137.
8. Bishop, R., Berryman, M., Culture speaks: Cultural relationships & classroom learning. 2006, Huia, Wellington, NZ.
9. Blank, A., Houkamau, C., Kingi, H., Unconscious bias and education: A comparative study of Māori and African American students. 2016 Retrieved from http://apo.org.au/resource/unconscious-bias-and-education-comparative-study-maori-and-african-american-students.
10. de Boer, H., Bosker, R.J., van der Werf, M.P.C., Sustainability of teacher expectation bias effects on long-term student performance. Journal of Educational Psychology 102:1 (2010), 168–179, 10.1037/a0017289.
11. Brown, G., asTTle - a national testing system for formative assessment: How the national testing policy ended up helping schools and teachers. Lai, M., Kushner, S., (eds.) A national developmental and negotiated approach to school and curriculum evaluation, 2013, Emerald Group Publishing, Bingley, UK, 39–56.
12. Brown, G.T.L., Teachers' self-reported assessment practices and conceptions: Using structural equation modelling to examine measurement and structural models. Teo, T., Kline, M.S., (eds.) Structural equation modelling in educational research: Concepts and applications, 2009, Sense Publishers, Rotterdam, NL, 243–266.
13. Chamberlain, M., Blueprint for national standards. 2010 Retrieved from New Zealand Education Gazette http://www.edgazette.govt.nz/Articles/Article.aspx?ArticleId=8187.
14. Chang, D.F., Sue, S., The effects of race and problem type on teachers' assessments of student behavior. Journal of Consulting and Clinical Psychology 71:2 (2003), 235–242, 10.1037/0022-006X.71.2.235.
15. Chubbuck, S.M., Zembylas, M., Social justice and teacher education: Context, theory, and practice. Loughran, John, Lynn_Hamilton, Mary, (eds.) International handbook of teacher education, Vol. 2, 2016, Springer, Singapore, 463–501.
16. Clark, C.M., Peterson, P.L., Teachers' thought processes. Wittrock, M.C., (eds.) Third handbook of research on teaching, 1986, Macmillan, New York, NY, 255–296.
17. Cohen, J., Statistical power analysis for the behavioral sciences. 1988, Laurence Erlbaum, Hillsdale, NJ.
18. Courtney, B., National standards: A parent's perspective. New Zealand Journal of Teachers' Work 7 (2010), 8–14.
19. Darr, C., McDowall, S., Ferral, H., Twist, J., Watson, V., Progressive achievement Test: Reading - teacher manual. 2008, New Zealand Council for Educational Research, Wellington, NZ.
20. Darr, C., Neill, A., Stephanou, A., Ferral, H., Progressive achievement Test: Mathematics – teacher manual. 2006, New Zealand Council for Educational Research, Wellington, NZ.
21. Doherty, J., Conolly, M., How accurately can primary school teachers predict the scores of their pupils in standardized tests of attainment? A study of some non-cognitive factors that influence specific judgements. Educational Studies 11:1 (1985), 41–60, 10.1080/0305569850110105.
22. Dompnier, B., Pansu, P., Bressoux, P., An integrative model of scholastic judgments: Pupils' characteristics, class context, halo effect and internal attributions. European Journal of Psychology of Education 21:2 (2006), 119–133, 10.1007/BF03173572.
23. Dorman, J.P., The effect of clustering on statistical tests: An illustration using classroom environment data. Educational Psychology 28:5 (2008), 583–595.
24. Eames, D., National standards policy: How parents mark it. 2010 Retrieved from nzherald.co.nz http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10624503.
25. Education Counts, Teacher census 2004. 2005 Retrieved 24/11/2016, from www.educationcounts.govt.nz/publications/schooling/teacher_census.
26. Erwin, J.O., Worrell, F.C., Assessment practices and the underrepresentation of minority students in gifted and talented education. Journal of Psychoeducational Assessment 30:1 (2012), 74–87.
27. Fairfax New Zealand Limited, How New Zealand schools rate. 2012 Retrieved from stuff.co.nz http://www.stuff.co.nz/national/education/7715044/How-New-Zealand-schools-rate.
28. Fairfax New Zealand Limited, School report. 2016 Retrieved from stuff.co.nz http://www.stuff.co.nz/national/education/school-report.
29. Feinberg, A.B., Shapiro, E.S., Accuracy of teacher judgments in predicting oral reading fluency. School Psychology Quarterly 18:1 (2003), 52–65.
30. Fothergill, J., & Taylor-Jorgensen, T. (n.d.). Assessment in New Zealand primary schools. Retrieved June 2016, from https://sites.google.com/site/ttjjmf/home.
31. Francis, B., Archer, L., Hodgen, J., Pepper, D., Taylor, B., Travers, M.-C., Exploring the relative lack of impact of research on ‘ability grouping’ in England: A discourse analytic account. Cambridge Journal of Education, 2016, 1–17, 10.1080/0305764X.2015.1093095.
32. Gerritsen, J., School groups wary of national standards computer system. 2016 Radio New Zealand. Retrieved from www.radionz.co.nz/news/national/295621/school-groups-wary-of-national-standards-computer-system.
33. Gill, J., Bayesian methods: A social and behavioral sciences approach. 2002, Chapman and Hall, Boca Raton, Fla.
34. Glock, S., Krolak-Schwerdt, S., Pit-ten Cate, I.M., Are school placement recommendations accurate? The effect of students' ethnicity on teachers' judgments and recognition memory. European Journal of Psychology of Education 30:2 (2015), 169–188, 10.1007/s10212-014-0237-2.
35. Haitana, T.N., Testing Tamariki: How suitable is the PPVT-III?. Master's Dissertation, 2007, University of Waikato, Hamilton, NZ.
36. Hanushek, E.A., Link, S., Woessmann, L., Does school autonomy make sense everywhere? Panel estimates from PISA. Journal of Development Economics 104 (2013), 212–232.
37. Harlen, W., Trusting teachers' judgement: Research evidence of the reliability and validity of teachers' assessment used for summative purposes. Research Papers in Education 20:3 (2005), 245–270, 10.1080/02671520500193744.
38. Hattie, J.A.C., Narrow the gap, fix the tail, or close the curves: The power of words. Rubie, C., Rawlinson, C., (eds.) Challenging thinking about teaching and learning, 2008, Nova Science.
39. Hattie, J.A., Brown, G.T.L., Standard setting for asTTle reading: A comparison of methods. asTTle Tech. Rep. #21, 2003, University of Auckland/Ministry of Education.
40. Hattie, J.A.C., Brown, G.T.L., Keegan, P., Irving, S.E., MacKay, A.J., Sutherland, T., et al. Validation evidence of asTTle reading assessment Results: Norms and criteria. AsTTle Tech. Rep. 22, 2003, University of Auckland/Ministry of Education.
41. Hecht, S.A., Greenfield, D.B., Explaining the predictive accuracy of teacher judgments of their students' reading achievement: The role of gender, classroom behavior, and emergent literacy skills in a longitudinal sample of children exposed to poverty. Reading and Writing 15:7–8 (2002), 789–809, 10.1023/A:1020985701556.
42. Helwig, R., Anderson, L., Tindal, G., Influence of elementary student gender on teachers' perceptions of mathematics achievement. The Journal of Educational Research 95:2 (2001), 93–102, 10.1080/00220670109596577.
43. Hoge, R.D., Butcher, R., Analysis of teacher judgments of pupil achievement levels. Journal of Educational Psychology 76:5 (1984), 777–781, 10.1037/0022-0663.76.5.777.
44. Hoge, R.D., Coladarci, T., Teacher-based judgments of academic achievement: A review of literature. Review of Educational Research 59:3 (1989), 297–313, 10.3102/00346543059003297.
45. Hopkins, K.D., George, C.A., Williams, D.D., The concurrent validity of standardized achievement tests by content area using teachers' ratings as criteria. Journal of Educational Measurement 22 (1985), 177–182.
46. Hurwitz, J.T., Elliott, S.N., Braden, J.P., The influence of test familiarity and student disability status upon teachers' judgments of students' test performance. School Psychology Quarterly, 22(2), 2007, 115.
47. Jussim, L., Harber, K.D., Teacher expectations and self-fulfilling prophecies: Knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review 9:2 (2005), 131–155, 10.1207/s15327957pspr0902_3.
48. Kaiser, J., Retelsdorf, J., Südkamp, A., Möller, J., Achievement and engagement: How student characteristics influence teacher judgments. Learning and Instruction 28 (2013), 73–84, 10.1016/j.learninstruc.2013.06.001.
49. Keegan, P., Brown, G.T., Hattie, J.A., 4 A psychometric view of sociocultural factors in test validity: The development of standardised test materials for Māori medium schools in New Zealand/Aotearoa. Phillipson, S., Ku, K.Y.L., Phillipson, S.N., (eds.) Constructing achievement: A sociocultural perspective, 2014, Routledge, London, 42–54.
50. Kim, H.Y., Statistical notes for clinical researchers: Assessing normal distribution using skewness and kurtosis. Restorative Dentistry & Endodontics 38:1 (2013), 52–54, 10.5395/rde.2013.38.1.52.
51. Kim, K.H., Zabelina, D., Cultural bias in assessment: Can creativity assessment help?. The International Journal of Critical Pedagogy, 6(2), 2015.
52. Kline, T.J., Psychological testing: A practical approach to design and evaluation. 2005, Sage, Thousand Oaks, CA.
53. Kreft, I.G.G., de Leeuw, J., Aiken, L.S., The effect of different forms of centering in Hierarchical Linear Models. Multivariate Behavioral Research 30 (1995), 1–21.
54. Lai, C.K., Marini, M., Lehr, S.A., Cerruti, C., Shin, J.L., Joy-Gaba, J.A.,. Nosek, B.A., Reducing implicit racial preferences: I. A comparative investigation of 17 interventions. Journal of Experimental Psychology: General 143:4 (2014), 1765–1785, 10.1037/a0036260.
55. Lembke, E.S., Foegen, A., Whittaker, T.A., Hampton, D., Establishing technically adequate measures of progress in early numeracy. Assessment for Effective Intervention 33:4 (2008), 206–214, 10.1177/1534508407313479.
56. Li, H., Pfeiffer, S.I., Petscher, Y., Kumtepe, A.T., Mo, G., Validation of the gifted rating scales–school form in China. Gifted Child Quarterly 52:2 (2008), 160–169, 10.1177/0016986208315802.
57. Martínez, J.F., Stecher, B., Borko, H., Classroom assessment practices, teacher judgments, and student achievement in mathematics: Evidence from the ECLS. Educational Assessment 14:2 (2009), 78–102, 10.1080/10627190903039429.
58. May, S. (n.d.). Assessment: What are the cultural issues in relation to Pasifika, Asian, ESOL, immigrant and refugee learners? University of Waikato. Retrieved June 2016, from http://assessment.tki.org.nz/Media/Files/May-S.-Assessment-what-are-the-cultural-issues-in-relation-to-Pasifika-Asian-ESOL-immigrant-and-refugee-learners-University-of-Waikato.
59. McGrady, P.B., Reynolds, J.R., Racial mismatch in the classroom beyond Black-white differences. Sociology of Education 86:1 (2013), 3–17.
60. McKown, C., Weinstein, R.S., Teacher expectations, classroom context, and the achievement gap. Journal of School Psychology 46:3 (2008), 235–261.
61. Meisels, S.J., Bickel, D.D., Nicholson, J., Xue, Y., Atkins-Burnett, S., Trusting teachers' judgments: A validity study of a curriculum-embedded performance assessment in kindergarten to grade 3. American Educational Research Journal 38:1 (2001), 73–95, 10.3102/00028312038001073.
62. Methe, S.A., Hintze, J.M., Floyd, R.G., Validation and decision accuracy of early numeracy skill indicators. School Psychology Review 37:3 (2008), 359–373.
63. Ministry of Education, National standards. 2010 Retrieved from TKI: Te Kete Ipurangi http://nzcurriculum.tki.org.nz/National-Standards.
64. Ministry of Education, Overall teacher judgment. 2011 Retrieved from www.nzcurriculum.tki.org.nz/National-Standards/Key-information/Fact-sheets/Overall-teacher-judgment/ http://www.nzcurriculum.tki.org.nz/National-Standards/Key-information/Fact-sheets/Overall-teacher-judgment/.
65. Ministry of Education, National Standards: Key information. 2012 Retrieved from http://www.nzcurriculum.tki.org.nz/National-Standards/.
66. Ministry of Education, & NZCER, e-asTTle technical manual. 2012, Ministry of Education, Wellington, NZ.
67. Ministry of Education, Find a school. 2016 Retrieved from Education Counts https://www.educationcounts.govt.nz/find-school.
68. Nieto, S., Placing equity front and center some thoughts on transforming teacher education for a new century. Journal of Teacher Education 51:3 (2000), 180–187.
69. Ogle, L., Sen, A., Pahlke, E., Jocelyn, L., Kastberg, D., Roey, S., Williams, T., International comparisons in fourth grade reading literacy: Findings from the Progress in International Reading Literacy Study (PIRLS) of 2001. (NCES 2003-073), 2003, U.S. Government Printing Office, Washington, DC.
70. Organisation for Economic Co-operation and Development (OECD), PISA 2003 Technical Report., 2005, OECD Publications, Paris, France.
71. Organisation for Economic Co-operation and Development (OECD), PISA 2012 results: Excellence through equity: Giving every student the chance to succeed, Vol. 2, 2013, OECD Publications, Paris, France http://dx.doi.org/10.1787/ 9789264201132-en.
72. ÖZerk, K., Whitehead, D., The impact of national standards assessment in New Zealand, and national testing protocols in Norway on indigenous schooling. International Electronic Journal of Elementary Education, 4(3), 2012, 545.
73. Parsons, S., Hallam, S., The impact of streaming on attainment at age seven: Evidence from the Millennium Cohort Study. Oxford Review of Education 40:5 (2014), 567–589, 10.1080/03054985.2014.959911.
74. Peterson, E.R., Rubie-Davies, C.M., Osborne, D., Sibley, C., Implicit and explicit teacher expectations: Relations with ethnicity and achievement. Learning and Instruction 42 (2016), 123–140, 10.1016/j.learninstruc.2016.01.010.
75. Raphael, T.E., Au, K.H., Goldman, S.R., Whole school instructional improvement through the standards-based change process. Hoffman, J., Goodman, Y., (eds.) Changing literacies for changing times, 2009, Routledge, London, 198–229.
76. Raudenbush, S.W., Bryk, A.S., Hierarchical linear models: Applications and data analysis methods. 2nd ed., 2002, Sage Publications, Thousand Oaks.
77. Ready, D.D., Wright, D.L., Accuracy and inaccuracy in teachers' perceptions of young children's cognitive abilities: The role of child background and classroom context. American Educational Research Journal 48:2 (2011), 335–360, 10.3102/0002831210374874.
78. Rubie-Davies, C.M., Teacher expectations and perceptions of student attributes: Is there a relationship?. British Journal of Educational Psychology 80:1 (2010), 121–135.
79. Rubie-Davies, C., Becoming a high expectation teacher: Raising the bar. 2014, Routledge, New York, NY.
80. Rubie-Davies, C.M., Hattie, J., Hamilton, R., Expecting the best for students: Teacher expectations and academic outcomes. British Journal of Educational Psychology 76:3 (2006), 429–444.
81. Rubie-Davies, C.M., Peterson, E.R., Flint, A., Garrett, L., McDonald, L.M., Watson, P., O'Neill, H., Ethnicity and teacher expectations in New Zealand. Procedia-Social and Behavioral Sciences 69 (2012), 256–261.
82. Rubie-Davies, C.M., Peterson, E.R., Sibley, C.G., Rosenthal, R., A teacher expectation intervention: Modelling the practices of high expectation teachers. Contemporary Educational Psychology 40 (2015), 72–85.
83. Rubie-Davies, C.M., Weinstein, R.S., Huang, F.L., Gregory, A., Cowan, P.A., Cowan, C.P., Successive teacher expectation effects across the early school years. Journal of Applied Developmental Psychology 35:3 (2014), 181–191.
84. Satherley, P., Student outcome overview 2001–2005: Research findings on student achievement in reading, writing and mathematics in New Zealand schools. 2006, Ministry of Education, Research Division, Wellington, NZ.
85. Schmidt, W., Burroughs, N., Zoido, P., Houang, R., The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher 44:7 (2015), 371–386, 10.3102/0013189X15603982.
86. Sharpley, C.F., Edgar, E., Teachers' ratings vs standardized tests: An empirical investigation of agreement between two indices of achievement. Psychology in the Schools 23:1 (1986), 106–111, 10.1002/1520-6807(198601)23:1<106::AID-PITS2310230117>3.0.CO;2-C.
87. Smith, L.A., Anderson, V., Blanch, K., Five beginning teachers' reflections on enacting New Zealand's national standards. Teaching and Teacher Education 54 (2016), 107–116.
88. Statistics New Zealand, Time series data for student numbers. 2016 Retrieved 23/11/2016, from Education Counts www.educationcounts.govt.nz/statistics/schooling/student-numbers/6028.
89. Südkamp, A., Kaiser, J., Möller, J., Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology 104:3 (2012), 743–762, 10.1037/a0027627.
90. Südkamp, A., Kaiser, J., Möller, J., Teachers' judgments of students' academic achievement. Krolak-Schwerdt, S., Glock, S., Böhmer, M., (eds.) Teachers' professional development: Assessment, training, and learning, 2014, Sense Publishers, Rotterdam, NL, 5–25.
91. Thrupp, M., National standards for student achievement: Is New Zealand's idiosyncratic approach any better. Australian Journal of Language and Literacy 36:2 (2013), 99–110.
92. Wiliam, D., Bartholomew, H., It's not which school but which set you're in that matters: The influence of ability grouping practices on student progress in mathematics. British Educational Research Journal 30:2 (2004), 279–293, 10.1080/0141192042000195245.
93. Wilson, A., Madjar, I., McNaughton, S., Opportunity to learn about disciplinary literacy in senior secondary English classrooms in New Zealand. The Curriculum Journal 27:2 (2016), 204–228, 10.1080/09585176.2015.1134339.
94. Woltman, H., Feldstain, A., MacKay, J.C., Rocchi, M., An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology 8:1 (2012), 52–69.
95. Wyatt-Smith, C., Klenowski, V., Gunn, S., The centrality of teachers' judgment practice in assessment: A study of standards in moderation. Assessment in Education: Principles, Policy & Practice 17 (2010), 59–75.
96. Wylie, C., Cosslett, G., Burgon, J., New Zealand principals: Autonomy at a cost. A decade of research on school principals, 2016, Springer International Publishing, 269–290.