Accuracy of teachers' judgments of students' cognitive abilities: A meta-analysis

Educational Research Review

Volumn 19, Issue , 2016, Pages 85-103

  Machts, Nils ^a   Kaiser, Johanna ^a   Schmidt, Fabian T C ^b   Möller, Jens ^a  

^a UNIVERSITY OF KIEL   (Germany)

^b LEIBNIZ INSTITUTE FOR SCIENCE AND MATHEMATICS EDUCATION   (Germany)

Author keywords

Cognitive abilities; Judgment accuracy; Meta analysis; Teacher judgment

Indexed keywords

EID: 84975483550     PISSN: 1747938X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.edurev.2016.06.003     Document Type: Review

References (94)

1. *Alvidrez J., Weinstein R.S. Early teacher perceptions and later student academic achievement. Journal of Educational Psychology 1999, 91:731-746. 10.1037/0022-0663.91.4.731.
2. Anders Y., McElvany N., Baumert J. The evaluation of learning-relevant student characteristics at the transition from elementary school to high school: How differentiated are teachers' judgments?. Der Übergang von der Grundschule in die weiterführende Schule. Leistungsgerechtigkeit und regionale, soziale und ethnisch-kulturelle Disparitäten 2010, 314-330. BMBF, Bonn & Berlin, Germany. K. Maaz, J. Baumert, C. Gresch, N. McElvany (Eds.).
3. Baddeley A.D. Working memory 1986, Oxford University Press, Oxford, United Kingdom.
4. *Baudson T.G., Preckel F. Development and validation of the German test for (highly) intelligent kids-T(H)INK. European Journal of Psychological Assessment 2013, 29:171-181. 10.1027/1015-5759/a000142.
5. Bennett R.E., Gottesman R.L., Rock D.A., Cerullo F. Influence of behavior perceptions and gender on teachers' judgments of students' academic skill. Journal of Educational Psychology 1993, 85:347-356. 10.1037/0022-0663.85.2.347.
6. *Blei S. The accuracy of teacher ratings in predicting cognitive ability and achievement: A longitudinal study 2010, (doctoral dissertation). Retrieved from:. http://diginole.lib.fsu.edu/etd/5468/[05/22/13].
7. *Bonvin P., Genoud P.A. Teacher perceptions of student characteristics. Paper presented at european conference on educational research (ECER), Geneva, Switzerland 2006.
8. Borenstein M. Effect sizes for continuous data. The handbook of research synthesis and meta-analysis 2009, 279-293. Russel Sage Foundation, New York, NY. H. Cooper, L.V. Hedges, J.C. Valentine (Eds.).
9. *Brandau H., Daghofer F., Hollerer L., Kaschnitz W., Kellner K., Kirchmair G., et al. The relationship between creativity, teacher ratings on behavior, age, and gender in pupils from seven to ten years. The Journal of Creative Behavior 2007, 41:91-113. 10.1002/j.2162-6057.2007.tb01283.x.
10. Cardelle-Elawar M. Effects of teaching metacognitive skills to students with low mathematics ability. Teaching and Teacher Education 1992, 8(2):109-121. 10.1016/0742-051X(92)90002-K.
11. *Carr M., Kurtz-Costes B.E. Is being smart everything? The influence of student achievement on teachers' perceptions. British Journal of Educational Psychology 1994, 64:263-276.
12. Carroll J.B. Human cognitive abilities: A survey of factor-analytic studies 1993, University Press, Cambridge, United Kingdom.
13. Carroll J.B. Psychometrics, intelligence, and public perception. Intelligence 1997, 24:25-52. 10.1016/S0160-2896(97)90012-X.
14. Cattell R.B., Weiss R., Osterland J. Grundintelligenztest Skala 1: CFT 1 1997, Westermann, Braunschweig, Germany.
15. *Chamorro-Premuzic T., Arteche A., Furnham A., Trickot N. Assessing pupils' intelligence through self, parental, and teacher estimates. Educational Psychology 2009, 29:83-97. 10.1080/01443410802520662.
16. Cheung M.W.-L. Meta-analysis: A structural equation modeling approach 2015, John Wiley & Sons.
17. Cohen J. Statistical power analysis for the behavioral sciences 1988, Academic Press, New York.
18. Cooper H.M. Integrating research: A guide for literature reviews 1998, Sage, Newbury Park, CA. 3rd ed.
19. Cropley A.J. Creativity in education & learning: A guide for teachers and educators 2001, Routledge, London, United Kingdom.
20. Cropley A.J., Cropley D. Fostering creativity: A diagnostic approach for higher education and organizations 2009, Hampton Press, Cresskill, NJ.
21. Deary I.J., Strand S., Smith P., Fernandes C. Intelligence and educational achievement. Intelligence 2007, 35:13-21. 10.1016/j.intell.2006.02.001.
22. *Desoete A. Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning 2008, 3:189-206. 10.1007/s11409-008-9026-0.
23. *Domino G. The D-48-application in Mexican-American children of a culture fair test. School Psychology International 2001, 22:253-257. 10.1177/0143034301223002.
24. Dünnebier K., Gräsel C., Krolak-Schwerdt S. Biases in teachers' assessments of student performance: An experimental study of anchoring effects. German Journal of Educational Psychology 2009, 23(3):187-195. 10.1024/1010-0652.23.34.187.
25. Dweck C. Mindset: The new psychology of success 2006, Random House, New York.
26. *Eaves R.C., Campbell-Whatley G., Dunn C., Reilly A., Tate-Braxton C. Comparison of the Slosson full-range intelligence test and teacher judgments as predictors of students' academic achievement. Journal of Psychoeducational Assessment 1994, 12:381-392. 10.1177/073428299401200408.
27. Federer M., Stüber S., Margraf J., Schneider S., Herrle J. Self- and external assessment of child anxiety. Zeitschrift für Differentielle und Diagnostische Psychologie 2001, 22:194-205. 10.1024//0170-1789.22.3.194.
28. Fischbach A., Baudson T.G., Preckel F., Martin R., Brunner M. Do teacher judgments of student intelligence predict life outcomes?. Learning and Individual Differences 2013, 27:109-119. 10.1016/j.lindif.2013.07.004.
29. Fischbach A., Schuchardt K., Brandenburg J., Klesczewski J., Balke-Melcher C., Schmidt C., et al. Learning weaknesses and learning disabilities: On the importance of diagnostic criteria. Lernen und Lernstörungen 2013, 2:65-76.
30. Follman J. Teachers' estimates of pupils' IQs and pupils' tested IQs. Psychological Reports 1991, 69:350.
31. Freund P.A., Kasten N. How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability. Psychological Bulletin 2012, 138:296-321. 10.1037/a0026556.
32. *García-Ros R., Talaya I., Pérez-González F. The process of identifying gifted children in elementary education: Teachers' evaluations of creativity. School Psychology International 2012, 33:661-672. 10.1177/0143034311421434.
33. Givvin K.B., Stipek D.J., Salmon J.M., MacGyvers V.L. In the eyes of the beholder: Students' and teachers' judgments of students' motivation. Teaching and Teacher Education 2001, 17:321-331. 10.1016/S0742-051X(00)00060-3.
34. Gliem J.A., Gliem R.R. Calculating, interpreting, and reporting Cronbach's alpha reliability coefficient for Likert-type scales. Paper presented at midwest research-to-practice conference in adult, continuing, and community education, Columbus, OH 2003.
35. Gottfredson L.S. Why g matters: The complexity of everyday life. Intelligence 1997, 24:79-132. 10.1016/S0160-2896(97)90014-3.
36. *Gralewski J., Karwowski M. Polite girls and creative boys? Students' gender moderates accuracy of teachers' ratings of creativity. The Journal of Creative Behavior 2013, 47:290-304. 10.1002/jocb.36.
37. Guilford J.P. Creativity. American Psychologist 1950, 5:444-454.
38. *Hany E.A. Do teachers identify gifted children better than tests do?. Psychologie in Erziehung und Unterricht 1991, 38:37-50.
39. Hany E.A. Modeling teachers' judgment of giftedness: A methodological inquiry of biased judgment. High Ability Studies 1997, 8:159-178. 10.1080/1359813970080203.
40. Hattie J., Timperley H. The power of feedback. Review of Educational Research 2007, 77(1):81-112. 10.3102/003465430298487.
41. Helmke A., Schrader F.-W. Interactional effects of instructional quality and teacher judgment accuracy on achievement. Teaching and Teacher Education 1987, 3:91-98. 10.1016/0742-051X(87)90010-2.
42. Hoge R.D., Coladarci T. Teacher-based judgments of academic achievement: A review of literature. Review of Educational Research 1989, 59:297-313. 10.2307/1170184.
43. Hurwitz J.T., Elliott S.N., Braden J.P. The influence of test familiarity and student disability status upon teachers' judgments of students' test performance. School Psychology Quarterly 2007, 22:115-144. 10.1037/1045-3830.22.2.115.
44. Impara J.C., Plake B.S. Teachers' ability to estimate item difficulty: A test of assumptions in the Angoff standard setting method. Journal of Educational Measurement 1998, 35:69-81. 10.1111/j.1745-3984.1998.tb00528.x.
45. Jarosewich T., Pfeiffer S.I., Morris J. Identifying gifted students using teacher rating scales: A review of existing instruments. Journal of Psychoeducational Assessment 2002, 20:322-336.
46. Jussim L., Harber K.D. Teacher expectations and self-fulfilling prophecies: Knowns and unknowns, resolved and unresolved controversies. Personality and Social Psychology Review 2005, 9:131-155. 10.1207/s15327957pspr0902_3.
47. Kaiser J., Helm F., Retelsdorf J., Südkamp A., Möller J. On the relation of intelligence and judgment accuracy in the process of assessing student achievement in the simulated classroom. German Journal of Educational Psychology 2012, 26:251-261. 10.1024/1010-0652/a000076.
48. Kaiser J., Möller J., Helm F., Kunter M. Das Schülerinventar: Welche Schülermerkmale die Leistungsurteile von Lehrkräften beeinflussen. Zeitschrift für Erziehungswissenschaft 2015, 1-24.
49. Kaiser J., Retelsdorf J., Südkamp A., Möller J. Achievement and engagement: How student characteristics influence teacher judgments. Learning and Instruction 2013, 28:73-84. 10.1016/j.learninstruc.2013.06.001.
50. Karing C. Diagnostic competence of elementary and secondary school teachers in the domains of competence and interests. German Journal of Educational Psychology 2009, 23:197-209. 10.1024/1010-0652.23.34.197.
51. *Klockgether K. Die Qualität von LehrerInneneinschätzungen bei der Identifikation von hoher kognitiver Leistungsfähigkeit bei Kindern 2011, (Master's Thesis), Retrieved from:. http://othes.univie.ac.at/14854/[05/23/13].
52. *Koch H., Kastner-Koller U., Deimann P., Kossmeier C., Koitz C., Steiner M. The development of kindergarten children as evaluated by their kindergarten teachers and mothers. Psychological Test and Assessment Modeling 2011, 53:241-257.
53. Kousoulas F., Mega G. Students' divergent thinking and teachers' ratings of creativity: Does gender play a role?. The Journal of Creative Behavior 2009, 43:209-222. 10.1002/j.2162-6057.2009.tb01315.x.
54. *Kuo C.C., Maker J., Su F.L., Hu C. Identifying young gifted children and cultivating problem solving abilities and multiple intelligences. Learning and Individual Differences 2010, 20:365-379. 10.1016/j.lindif.2010.05.005.
55. *Levinson E.M., Folino L. Correlations of scores on the gifted evaluation scale with those on WISC-III and Kaufman Brief intelligence test for students referred for gifted evaluation. Psychological Reports 1994, 74:419-424. 10.2466/pr0.1994.74.2.419.
56. Lohman D.F., Hagen E.P., Thorndike R.L. Cognitive abilities test 2001, NFER-Nelson, Windsor, United Kingdom.
57. *Lorenz w Diagnostische Kompetenz von Grundschullehrkräften: strukturelle Aspekte und Bedingungen 2011, University of Bamberg Press, Bamberg, Germany.
58. Lorenz C., Artelt C. Domain specificity and stability of diagnostic competence among primary school teachers in the school subjects of German and mathematics. German Journal of Educational Psychology 2009, 23:211-222. 10.1024/1010-0652.23.34.211.
59. Lowenstein L.F. Teachers' effectiveness in identifying gifted children. Gifted Education International 1982, 1:33-35. 10.1177/026142948200100121.
60. Marsh H.W., Craven R.G. Self-other agreement on multiple dimensions of preadolescent self-concept: Inferences by teachers, mothers, and fathers. Journal of Educational Psychology 1991, 83:393. 10.1037/0022-0663.83.3.393.
61. McElvany N., Schroeder S., Hachfeld A., Baumert J., Richter T., Schnotz W., et al. Teachers' diagnostic skills to assess student abilities and task difficulty of learning materials incorporating instructional pictures. German Journal of Educational Psychology 2009, 23:223-235. 10.1024/1010-0652.23.34.223.
62. McIntosh D., Dixon F. Use of intelligence tests in the identification of giftedness. Contemporary intellectual theories, tests and issues 2005, 504-520. Guilford, New York, NY. 2nd ed. D. Flanagan, P. Harrison (Eds.).
63. *Miller S.A., Davis T.L. Beliefs about children: A comparative study of mothers, teachers, peers, and self. Child Development 1992, 63:1251-1265. 10.1111/j.1467-8624.1992.tb01693.x.
64. Möller J., Köller O. Attributionen und Schulleistung. [Attribution and school achievement]. Emotionen, Kognitionen und Schulleistung 1996, 115-136. Psychologie Verlags Union, Weinheim, Germany.
65. Möller J., Pohlmann B., Köller O., Marsh H.W. A meta-analytic path analysis of the internal/external frame of reference model of academic achievement and academic self-concept. Review of Educational Research 2009, 79:1129-1167. 10.3102/0034654309337522.
66. Mueller C.M., Dweck C.S. Praise for intelligence can undermine children's motivation and performance. Journal of Personality and Social Psychology 1998, 75(1):33.
67. *Neber H. Teacher identification of students for gifted programs: Nominations to a summer school for highly-gifted students. Psychology Science 2004, 46:348-362.
68. Neisser U., Boodoo G., Bouchard T.J., Boykin A.W., Brody N., Ceci S.J., et al. Intelligence: Knowns and unknowns. American psychologist 1996, 51(2):77.
69. *Newman T.M., Sparrow S.S., Pfeiffer S.I. Use of the WISC-IV in assessment and intervention planning for children who are gifted. WISC-IV clinical assessment and intervention 2008, 217-272. Elseviere, San Diego, CA. A. Prifitera, D.H. Saklofske, L.G. Weiss (Eds.).
70. Pfeiffer S.I., Jarosewich T. The Gifted Rating Scales-School Form: An analysis of the standardization sample based on age, gender, race, and diagnostic efficiency. Gifted Child Quarterly 2007, 51:39-50.
71. Praetorius A.K., Greb K., Lipowsky F., Gollwitzer M. Teachers as diagnosticians: What role does student performance play in the assessment of mathematical self-concepts?. Journal for Educational Research Online 2010, 2:121-144.
72. *Pretzlik U., Olsson J., Nabuco M.E., Cruz I. Teachers' implicit views of intelligence predict pupils' self-perception as learners. Cognitive Development 2003, 18(4):579-599. 10.1016/j.cogdev.2003.09.008.
73. *Protter-Lerner L. Psychometric evaluation of the nonverbal assessment of problem solving scale as used with children with autism 2004, (doctoral dissertation). Retrieved from Dissertations and Theses database. (UMI No. AAI3124962).
74. *Ricovero J. The relationships among teacher ratings, intelligence, and academic achievement in predicting "schoolhouse giftedness" 2000, (doctoral dissertation). Retrieved from Dissertations and Theses database. (UMI No. AAI9965263).
75. Rindermann H. Increase of intelligence in childhood and youth. Psychologie in Erziehung und Unterricht 2011, 58(3):210-224. 10.2378/peu2011.art29d.
76. Rost D.H. Intelligenz: Fakten und Mythen. [Intelligence: Facts and myths] 2009, Beltz, Weinheim, Germany.
77. Rost D.H., Hanses P. Not achieving-Not gifted? About the identification of gifted children by teacher-ratings. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie 1997, 29:167-177.
78. Runco M.A. Education for creative potential. Scandinavian Journal of Educational Research 2003, 47(3):317-324. 10.1080/00313830308598.
79. Runco M.A. Creativity: Theories and Themes: Research. Development and practice 2007, Elsevier, Amsterdam.
80. Runco M.A. Commentary: Divergent thinking is not synonymous with creativity. Psychology of Aesthetics, Creativity, and the Arts 2008, 2:93-96.
81. Simonton D.K. Expertise, competence, and creative ability: The perplexing complexities. The Psychology of abilities, competencies, and expertise 2003, 213-239. Cambridge University Press, New York, NY. R.J. Sternberg, E.L. Grigorenko (Eds.).
82. *Sommer U., Fink A., Neubauer A.C. Detection of high ability children by teachers and parents: Psychometric quality of new rating checklists for the assessment of intellectual, creative and social ability. Psychology Science 2008, 50:189-205.
83. *Spinath B. Accuracy of teachers; judgment of pupils' characteristics and the construct of diagnostic competence. German Journal of Educational Psychology 2005, 19:85-95. 10.1024/1010-0652.19.12.85.
84. Südkamp A., Kaiser J., Möller J. Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology 2012, 104:743-762. 10.1037/a0027627.
85. Thorndike R.L., Hagen E.P., Sattler J.M. Stanford-Binet intelligence scale 1986, Riverside Publishing Company, Chicago, IL.
86. *Tridico R.S. The predictive value of screening data in the identification of intellectually gifted students 2001, (doctoral dissertation). Retrieved from Dissertations and Theses database. (UMI No. AAI9998873).
87. *Urhahne D. Teachers' judgments of elementary students' ability, creativity and task commitment. Talent Development and Excellence 2011, 3:229-237.
88. Urhahne D., Zhou J., Stobbe M., Chao S.H., Zhu M., Shi J. Motivational and affective characteristics of underestimated students. German Journal of Educational Psychology 2010, 24:275-288. 10.1024/1010-0652/a000021.
89. Wahlsten D. The malleability of intelligence is not constrained by heritability. Intelligence, genes, and Success: Scientists respond to the bell curve 1997, 71-87. Springer New York, New York, NY. B. Devlin, S.E. Fienberg, D.P. Resnick, K. Roeder (Eds.).
90. *Wild K.P. Teachers' diagnostics of giftedness. Lebensumweltanalyse hochbegabter Kinder. Das Marburger Hochbegabtenprojekt 1993, 236-261. Hogrefe, Göttingen, Germany. D.H. Rost (Ed.).
91. Winne P.H., Nesbit J.C. The psychology of academic achievement. Annual Review of Psychology 2010, 61:653-678.
92. *Winzen J.M. Kreativitätsmessung in der Berufseignungsdiagnostik - Entwicklung und Validierung eines Verfahrens zur Kreativitätsmessung unter Berücksichtigung kognitiver Fähigkeiten und Aspekten der kreativen Persönlichkeit 2011, (doctoral dissertation). [Measurement of creativity in occupational aptitude assessment - development and validation of an instrument for creativity measurement under consideration of cognitive abilities and aspects of creative personality]. Received on personal request.
93. *Zavertnik J.L. Upper elementary teachers' predictions of their students' oral reading levels and levels of reading-related CHC abilities 2007, (doctoral dissertation). Retrieved from Dissertations and Theses database. (UMI No. AAI3268230).
94. *Ziegler A., Stoeger H. Identification of underachievement with standardized tests, student, parental and teacher assessments. An empirical study on the agreement among various diagnostic sources. Gifted and Talented International 2003, 18:87-94.