Identifying determinants of teachers' judgment (in)accuracy regarding students' school-related motivations using a Bayesian cross-classified multi-level model

Learning and Instruction

Volumn 52, Issue , 2017, Pages 148-160

  Praetorius, Anna Katharina ^a   Koch, Tobias ^b   Scheunpflug, Annette ^c   Zeinz, Horst ^d   Dresel, Markus ^e  

^a GERMAN INSTITUTE FOR INTERNATIONAL EDUCATIONAL RESEARCH   (Germany)

^b LEUPHANA UNIVERSITY OF LÜNEBURG   (Germany)

^c UNIVERSITY OF BAMBERG   (Germany)

^d UNIVERSITY OF MÜNSTER   (Germany)

^e UNIVERSITY OF AUGSBURG   (Germany)

Author keywords

Cross classification; Judgment accuracy; Motivation; Teacher judgments; Variance sources

Indexed keywords

EID: 85021633860     PISSN: 09594752     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.learninstruc.2017.06.003     Document Type: Article

References (86)

1. Al-Hoorie, A.H., Unconscious motivation. Part II: Implicit attitudes and L2 achievement. Studies in Second Language Learning and Teaching 6:4 (2016), 619–649, 10.14746/ssllt.2016.6.4.4.
2. Alvidrez, J., Weinstein, R.S., Early teacher perceptions and later student academic achievement. Journal of Educational Psychology 91 (1999), 731–746, 10.1037/0022-0663.91.4.731.
3. Ames, C., Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology 84 (1992), 261–271, 10.1037/0022-0663.84.3.261.
4. Behrmann, L., Souvignier, E., The relation between teachers' sensitivity, their instructional activities, and their students' achievement gains in reading. Zeitschrift für Pädagogische Psychologie 27 (2013), 283–293, 10.1024/1010-0652/a000112.
5. Bell, C.A., Qi, Y., Croft, A.J., Leusner, D., McCaffrey, D.F., Gitomer, D.H., et al. Improving observational score quality. Kane, T.J., Kerr, K.A., Pianta, R.C., (eds.) Designing teacher evaluation systems, 2015, John Wiley & Sons, Inc, San Francisco, 10.1002/9781119210856.ch3.
6. Berhenke, A., Miller, A.L., Brown, E., Seifer, R., Dickstein, S., Observed emotional and behavioral indicators of motivation predict school readiness in Head Start graduates. Early Childhood Research Quarterly 26:4 (2011), 430–441, 10.1016/j.ecresq.2011.04.001.
7. Böhmer, I., Hörstermann, T., Gräsel, C., Krolak-Schwerdt, S., Glock, S., An analysis of information search in the process of making school tracking decisions: Which judgment rule do teachers apply? (Trans.) Eine Analyse der Informationssuche bei der Erstellung der Übergangsempfehlung: Welcher Urteilsregel folgen Lehrkräfte? Journal for Educational Research Online 7 (2015), 59–81.
8. Bong, M., Between- and within-domain relations of academic motivation among middle and high school students: Self-efficacy, task value, and achievement goals. Journal of Educational Psychology 93 (2001), 23–34, 10.1037/0022-0663.93.1.23.
9. Brunner, M., Keller, U., Dierendonck, C., Reichert, M., Ugen, S., Fischbach, A., et al. The structure of academic self-concepts revisited: The nested Marsh/Shavelson model. Journal of Educational Psychology 102 (2010), 964–981, 10.1037/a0019644.
10. Brunswik, E., Perception and the representative design of psychological experiments. 1956, University of California Press, Berkeley, CA.
11. Chaplin, W.F., The next generation of moderator research in personality psychology. Journal of Personality 59:2 (1991), 143–178, 10.1111/j.1467-6494.1991.tb00772.x.
12. Clausen, M., Unterrichtsqualität – eine Frage der Perspektive? (Trans.) Instructional quality – a matter of perspective?, 2002, Waxmann, Münster, Germany.
13. Corno, L.Y.N., On teaching adaptively. Educational Psychologist 43:3 (2008), 161–173, 10.1080/00461520802178466.
14. Council of Chief State School Officers (CCSSO) Interstate Teacher Assessment and Support Consortium. TASC model core teaching standards and learning progressions for teachers 1.0, 2013, CCSSO Interstate Teacher Assessment and Support Consortium, Washington, DC.
15. Curtis, S.M., mcmcplots: Create plots from MCMC output. R package version 0.4.2, 2015 https://CRAN.R-project.org/package=mcmcplots.
16. Deci, E.L., Ryan, R.M., The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry 11 (2000), 227–268, 10.1207/s15327965pli1104_01.
17. Decuir-Gunby, J.T., Williams-Johnson, M.R., The influence of culture on emotions: Implications for education. Pekrun, R., Linnenbrink, L.A., (eds.) International handbook of emotions in education, 2014, Routledge, New York, NY, 539–557.
18. Dickhäuser, O., Reinhard, M.-A., Factors underlying expectancies of success and achievement: The influential roles of need for cognition and general or specific self-concepts. Journal of Personality and Social Psychology 90 (2006), 490–500, 10.1037/0022-3514.90.3.490.
19. Dickhäuser, O., Stiensmeier-Pelster, J., Wahrgenommene Lehrereinschätzungen und das Fähigkeitsselbstkonzept von Jungen und Mädchen in der Grundschule (Trans.) Perceived teacher judgments and the academic self-concept of boys and girls in primary schools Psychologie in Erziehung und Unterricht 50:2 (2003), 182–190.
20. Dunn, T.J., Baguley, T., Brunsden, V., From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology 105:3 (2014), 399–412, 10.1111/bjop.12046.
21. Feinberg, A.B., Shapiro, E.S., Accuracy of teacher judgments in predicting oral reading fluency. School Psychology Quarterly 18 (2003), 52–65, 10.1521/scpq.18.1.52.20876.
22. Fielding, A., Goldstein, H., Cross-classified and multiple membership structures in multilevel models: An introduction and review (Research Report RR791)., 2006, University, Department for Education and Skills, Birmingham.
23. Fortier, M.S., Vallerand, R.J., Guay, F., Academic motivation and school performance: Toward a structural model. Contemporary Educational Psychology 20:3 (1995), 257–274.
24. Funder, D.C., On the accuracy of personality judgment: A realistic approach. Psychological Review 102:4 (1995), 652–670.
25. Funder, D.C., Accurate personality judgment. Current Directions in Psychological Science 21:3 (2012), 177–182.
26. Funder, D.C., Colvin, C.R., Congruence of others' and self-judgments of personality. Hogan, R., Johnson, J.A., Briggs, S.R., (eds.) Handbook of personality psychology, 1997, Academic Press, San Diego, CA, 617–647.
27. Gelman, A., Hill, J., Data analysis using regression and multilevel hierarchical models. Vol. 1, 2007, Cambridge University Press, New York, NY.
28. Givvin, K.B., Stipek, D.J., Salmon, J.M., MacGyvers, V.L., In the eyes of the beholder: Students' and teachers' judgments of students' motivation. Teaching and Teacher Education 17 (2001), 321–331, 10.1016/S0742-051X(00)00060-3.
29. Glock, S., Krolak-Schwerdt, S., Klapproth, F., Böhmer, M., Beyond judgment bias: How students' ethnicity and academic profile consistency influence teachers' tracking judgments. Social Psychology in Education 16 (2013), 555–573, 10.1007/s11218-013-9227-5.
30. Gottfried, A.E., Academic intrinsic motivation in elementary and junior high school students. Journal of Educational Psychology 77:6 (1985), 631–645.
31. Guay, F., Marsh, H.W., Boivin, M., Academic self-concept and achievement: Developmental perspective on their causal ordering. Journal of Educational Psychology 95 (2003), 124–136, 10.1037/0022-0663.95.1.124.
32. Helmke, A., Schrader, F.-W., Interactional effects of instructional quality and teacher judgment accuracy on achievement. Teaching and Teacher Education 3 (1987), 91–98.
33. Herppich, S., Praetorius, A.K., Hetmanek, A., Glogger-Frey, I., Ufer, S., Leutner, D., Südkamp, A., Ein Arbeitsmodell für die empirische Erforschung der diagnostischen Kompetenz von Lehrkräften. Südkamp, A., Praetorius, A.-K., (eds.) Diagnostische Kompetenz von Lehrkräften: Theoretische und methodische Weiterentwicklungen [Judgmental competences of teachers: Theoretical and methodological developments], 2017, Waxmann, Münster, 75–95.
34. Hinnant, J.B., O'Brien, M., Ghazarian, S.R., The longitudinal relations of teacher expectations to achievement in the early school year. Journal of Educational Psychology 101 (2009), 662–670, 10.1037/a0014306.
35. Hopkins, K.D., George, C.A., Williams, D.D., The concurrent validity of standardized achievement tests by content area using teachers' ratings as criteria. Journal of Educational Measurement 22 (1985), 177–182, 10.1111/j.1745-3984.1985.tb01056.x.
36. Hulleman, C.S., Harackiewicz, J.M., Promoting interest and performance in high school science classes. Science 326 (2009), 1410–1412, 10.1126/science.1177067.
37. Hurwitz, J.T., Elliott, S.N., Braden, J.P., The influence of test familiarity and student disability status upon teachers' judgments of students' test performance. School Psychology Quarterly 22 (2007), 115–144, 10.1037/1045-3830.22.2.115.
38. Itskowitz, R., Navon, R., Strauss, H., Teachers' accuracy in evaluating students' self-image: Effect of perceived closeness. Journal of Educational Psychology 80 (1988), 337–341, 10.1037/0022-0663.80.3.337.
39. James, W., The self. Baumeister, R.F., (eds.) The self in social psychology, 1892/1999, Psychology Press, Philadelphia, PA, 69–77 Original: James, W. [1892/1948]. Psychology. Cleveland, OH: World Publishing.
40. Kaiser, J., Helm, F., Retelsdorf, J., Südkamp, A., Möller, J., Zum Zusammenhang von Intelligenz und Urteilsgenauigkeit bei der Beurteilung von Schülerleistungen im Simulierten Klassenraum (Trans.) On the relation of intelligence and judgement accuracy in the process of assessing student achievement in the simulated classroom Zeitschrift für Pädagogische Psychologie 26 (2012), 251–261, 10.1024/1010-0652/a000076.
41. Kaiser, J., Südkamp, A., Möller, J., The effects of student characteristics on teachers' judgment accuracy: Disentangling ethnicity, minority status, and achievement. Journal of Educational Psychology, 2017, 10.1037/edu0000156 Advance online publication.
42. Karing, C., Diagnostische Kompetenz von Grundschul- und Gymnasiallehrkräften im Leistungsbereich und im Bereich Interessen (Trans.) Diagnostic competence of elementary and secondary school teachers in the domains of competence and interests Zeitschrift für Pädagogische Psychologie 23 (2009), 197–209, 10.1024/1010-0652.23.34.197.
43. Kenny, D.T., Chekaluk, E., Early reading performance: A comparison of teacher-based and test-based assessment. Journal of Learning Disabilities 26 (1993), 227–236, 10.1177/002221949302600403.
44. Koch, T., Schultze, M., Jeon, M., Nussbeck, F., Praetorius, A.-K., Eid, M., A cross-classified CFA-MTMM model for structurally different and non-independent interchangeable methods. Multivariate Behavioral Research 51 (2016), 67–85, 10.1080/00273171.2015.1101367.
45. Kunter, M., Schümer, G., Artelt, C., Baumert, J., Klieme, E., Neubrand, M., et al. PISA 2000: Dokumentation der Erhebungsinstrumente (Trans.) PISA 2000: Documentation of the survey instruments, 2003, Max-Planck-Institut für Bildungsforschung, Berlin, Germany.
46. Lorenz, C., Diagnostische Kompetenz von Grundschullehrkräften. Strukturelle Aspekte und Bedingungen (Trans.) Diagnostic competence of primary school teachers. Structural aspects and conditions, 2011, University of Bamberg Press, Bamberg, Germany.
47. Machts, N., Kaiser, J., Schmidt, F.T., Möller, J., Accuracy of teachers' judgments of students' cognitive abilities: A meta-analysis. Educational Research Review 19 (2016), 85–103, 10.1016/j.edurev.2016.06.003.
48. Maguin, E., Loeber, R., How well do ratings of academic performance by mothers and their sons correspond to grades, achievement test scores, and teachers' ratings?. Journal of Behavioral Education 6 (1996), 405–425, 10.1007/BF02110514.
49. Marksteiner, T., Reinhard, M.-A., Dickhäuser, O., Sporer, S.L., How do teachers perceive cheating students? Beliefs about cues to deception and detection accuracy in the educational field. European Journal of Psychology of Education 27 (2012), 329–350, 10.1007/s10212-011-0074-5.
50. Marsh, H., A multidimensional, hierarchical model of self-concept: Theoretical and empirical justification. Educational Psychology Review 2 (1990), 77–172.
51. Marsh, H.W., Craven, R.G., Self-other agreement on multiple dimensions of preadolescent self-concept: Inferences by teachers, mothers, and fathers. Journal of Educational Psychology 83:3 (1991), 393–404.
52. Martinéz, J.F., Stecher, B., Borko, H., Classroom assessment practices, teacher judgments, and student achievement in mathematics: Evidence from the ECLS. Educational Assessment 14 (2009), 78–102, 10.1080/10627190903039429.
53. National Council on Measurement in Education, Washington, DC.& American Federation of Teachers, Washington, DC. & National Education Association, Washington, DC, Standards for teacher competence in educational assessment of students. [Washington, D.C.]: Distributed by ERIC Clearinghouse, 1990 http://www.eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED323186.
54. O'Mara, A.J., Marsh, H.W., Craven, R.G., Debus, R.L., Do self-concept interventions make a difference? A synergistic blend of construct validation and meta-analysis. Educational Psychologist 41:3 (2006), 181–206, 10.1207/s15326985ep4103_4.
55. Paulhus, D.L., Vazire, S., The self-report method. Robins, R.W., Fraley, R.C., Krueger, R.F., (eds.) Handbook of research methods in personality psychology, 2007, Guilford, New York, 224–239.
56. Pekrun, R., Elliot, A.J., Maier, M.A., Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational Psychology 101 (2009), 115–135, 10.10137/a0013383.
57. Pintrich, P.R., A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology 95:4 (2003), 667–686, 10.1037/0022-0663.95.4.667.
58. Plummer, M., rjags: Bayesian graphical models using MCMC. 2016 R package version 4-6. Retrieved from https://CRAN.R-project.org/package=rjags.
59. Plummer, M., Best, N., Cowles, K., Vines, K., CODA: Convergence diagnosis and output analysis for MCMC. R News 6:1 (2006), 7–11.
60. Pohlmann, S., The transition at the end of primary school - The formation of tracking recommendations from the teachers' perspective (Trans.) Der Übergang am Ende der Grundschulzeit – Zur Formation der Übergangsempfehlung aus der Sicht der Lehrkräfte, 2009, Waxmann, Münster, Germany.
61. Praetorius, A.-K., Berner, V.-D., Zeinz, H., Scheunpflug, A., Dresel, M., Judgment confidence and judgment accuracy of teachers in judging academic self-concepts of students. Journal of Educational Research 106 (2013), 64–76, 10.1080/00220671.2012.667010.
62. Praetorius, A.-K., Drexler, K., Rösch, L., Christophel, E., Heyne, N., Scheunpflug, A., Dresel, M., Judging students' self-concepts within 30 seconds? An application of the zero-acquaintance approach to research on teachers' judgment accuracy. Learning and Individual Differences 37 (2015), 231–236, 10.1016/j.lindif.2014.11.015.
63. Praetorius, A.-K., Greb, K., Dickhäuser, O., Lipowsky, F., Wie gut schätzen Lehrer die Fähigkeitsselbstkonzepte ihrer Schüler ein? Zur diagnostischen Kompetenz von Lehrkräften (Trans.) How teacher rate their students: on teachers' diagnostics competence regarding the academic self-concept Psychologie in Erziehung und Unterricht 58 (2011), 81–91, 10.2378/peu2010.art30d.
64. R Development Core Team, R: A language and environment for statistical computing. R foundation for statistical computing. 2008 Retrieved from http://www.R-project.org.
65. Reindl, M., Berner, V.-D., Scheunpflug, A., Zeinz, H., Dresel, M., Effect of negative peer climate on the development of autonomous motivation in mathematics. Learning and Individual Differences 38 (2015), 68–75, 10.1016/j.lindif.2015.01.017.
66. Retelsdorf, J., Köller, O., Möller, J., Reading achievement and reading self-concept – testing the reciprocal effects model. Learning and Instruction 29 (2014), 21–30, 10.1016/j.learninstruc.2013.07.004.
67. Reyes, M.R., Brackett, M.A., Rivers, S.E., White, M., Salovey, P., Classroom emotional climate, student engagement, and academic achievement. Journal of Educational Psychology 104:3 (2012), 1–13, 10.1037/a0027268.
68. Ross, J.A., The antecedents and consequences of teacher efficacy. Brophy, J., (eds.) Advances in research on teaching, 1998, JAI Press, Greenwich, CT, 385–400.
69. Schaffner, E., Schiefele, U., Ulferts, H., Reading amount as a mediator of the effects of intrinsic and extrinsic reading motivation on reading comprehension. Reading Research Quarterly 48:4 (2013), 369–385, 10.1002/rrq.52.
70. Schönbrodt, F.D., Perugini, M., At what sample size do correlations stabilize?. Journal of Research in Personality 47 (2013), 609–612, 10.1016/j.jrp.2013.05.009.
71. Schöne, C., Dickhäuser, O., Spinath, B., Stiensmeier-Pelster, J., SESSKO – Skalen zur Erfassung des schulischen Selbstkonzepts (Trans.) Scales to assess the academic self-concept, 2002, Hogrefe, Göttingen, Germany.
72. Schwarzer, R., Jerusalem, M., Skalen zur Erfassung von Lehrer- und Schüler-merkmalen. Dokumentation der psychometrischen Verfahren im Rahmen der wissenschaftlichen Begleitung des Modellversuchs Selbstwirksame Schulen. 1999, Freie Universität Berlin, Berlin.
73. Shavelson, R.J., Hubner, J.J., Stanton, G.C., Self-concept: Validation of construct interpretations. Review of Educational Research 46:3 (1976), 407–441.
74. Skinner, E.A., Belmont, M.J., Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology 85:4 (1993), 571–581.
75. Sparfeldt, J.R., Schilling, S.R., Rost, D.H., Müller, C., Bezugsnormorientierte Selbstkonzepte? Zur Eignung der SESSKO (Trans.) Reference-oriented self-concepts? Regarding the suitability of SESSKO Zeitschrift für Differentielle und Diagnostische Psychologie 24:4 (2003), 325–335.
76. Spiegelhalter, D.J., Best, N.G., Carlin, B.P., Van Der Linde, A., Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64 (2002), 583–639, 10.1111/1467-9868.00353.
77. Spinath, B., Akkuratheit der Einschätzung von Schülermerkmalen durch Lehrer und das Konstrukt der diagnostischen Kompetenz (Trans.) Accuracy of teacher judgments of student characteristics and the construct of diagnostic competence Zeitschrift für Pädagogische Psychologie/German Journal of Educational Psychology 19 (2005), 85–95, 10.1024/1010-0652.19.12.85.
78. Südkamp, A., Kaiser, J., Möller, J., Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology 104 (2012), 743–762, 10.1037/a0027627.
79. Tiedemann, J., Teachers' gender stereotypes as determinants of teacher perceptions in elementary school mathematics. Educational Studies in Mathematics 50 (2002), 49–62, 10.1023/A:1020518104346.
80. Tomlinson, C.A., Brighton, C., Hertberg, H., Callahan, C.M., Moon, T.R., Brimijoin, K., Reynolds, T., Differentiating instruction in response to student readiness, interest, and learning profile in academically diverse classrooms: A review of literature. Journal for the Education of the Gifted 27:2–3 (2003), 119–145.
81. Urhahne, D., Chao, S.-H., Florineth, M.L., Luttenberger, S., Paechter, M., Academic self-concept, learning motivation, and test anxiety of the underestimated student. British Journal of Educational Psychology 81 (2011), 161–177, 10.1348/000709910X504500.
82. Urhahne, D., Zhu, M., Accuracy of teachers' judgments of students' subjective well-being. Learning and Individual Differences 43 (2015), 226–232, 10.1016/j.lindif.2015.08.007.
83. Weinert, F.E., Schrader, F.W., Diagnose des Lehrers als Diagnostiker (Trans.) Diagnosis of the teachers as a diagnostician Petillon, H., Wagner, J., Wolf, B., (eds.) Schülergerechte Diagnose. Theoretische und empirische Beitriäge zur Pädagogischen Diagnostik [Diagnosis which does justice to the pupil: Theoretical and empirical contributions to diagnostics in teaching], 1986, Beltz, Weinheim, 1l–29.
84. Wigfield, A., Eccles, J.S., Expectancy–value theory of achievement motivation. Contemporary Educational Psychology 25:1 (2000), 68–81, 10.1006/ceps.1999.1015.
85. Wild, K.-P., Rost, D., Klassengröße und Genauigkeit von Schülerbeurteilungen (Trans.) Class size and accuracy of judgments of students Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie 27:1 (1995), 78–90.
86. Wiley, A.L., Tankersley, M., Simms, A., Teachers' causal attributions for student problem behavior: Implications for school-based behavioral interventions and research. Cook, B.G., Tankersley, M., Landrum, T.J., (eds.) Classroom behavior, contexts, and interventions, 2012, Emerald, Bingley, UK, 279–300.