How does information consistency influence prospective teachers’ decisions about task difficulty assignments? A within-subject experiment to explain data-based decision-making in heterogeneous classes

Learning and Instruction

Volumn 74, Issue , 2021, Pages

  Kärner, Tobias ^a   Warwas, Julia ^a   Krannich, Maike ^b   Weichsler, Niklas ^c  

^a UNIVERSITY OF HOHENHEIM   (Germany)

^b UNIVERSITY OF ZURICH   (Switzerland)

^c Teachers' College Weingarten   (Germany)

Author keywords

Information consistency; Information processing; Mixed methods approach; Proximal and distal performance indicators; Task difficulty assignment

Indexed keywords

EID: 85100371587     PISSN: 09594752     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.learninstruc.2020.101440     Document Type: Article

References (92)

1. Allenby, G.M., Lenk, P.J., Reassessing brand loyalty, price sensitivity, and merchandising effects on consumer brand choice. Journal of Business & Economic Statistics 13:3 (1995), 281–289, 10.1080/07350015.1995.10524602.
2. Allen, M.H., Matthews, C.E., Parsons, S.A., A second-grade teacher's adaptive teaching during an integrated science-literacy unit. Teaching and Teacher Education 35 (2013), 114–125, 10.1016/j.tate.2013.06.002.
3. Anders, Y., Kunter, M., Brunner, M., Krauss, S., Baumert, J., Diagnostische Fähigkeiten von Mathematiklehrkräften und ihre Auswirkungen auf die Leistungen ihrer Schülerinnen und Schüler [Mathematics teachers' diagnostic skills and their impact on students' achievements]. Psychologie in Erziehung und Unterricht 57:3 (2010), 175–193, 10.2378/peu2010.art13d.
4. Anderson, T.W., Hsiao, C., Formulation and estimation of dynamic models using panel data. Journal of Econometrics 18:1 (1982), 47–82, 10.1016/0304-4076(82)90095-1.
5. Arellano, M., Bond, S., Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations. The Review of Economic Studies 58:2 (1991), 277–297, 10.2307/2297968.
6. Asseburg, R., Frey, A., Too hard, too easy, or just right? The relationship between effort or boredom and ability-difficulty fit. Psychological Test and Assessment Modeling 55:1 (2013), 92–104.
7. Atılgan, H., Using Generalizability theory to assess the score reliability of the Special Ability Selection Examinations for music education programmes in higher education. International Journal of Research and Method in Education 31:1 (2008), 63–76, 10.1080/17437270801919925.
8. Auwarter, A.E., Aruguete, M.S., Effects of student gender and socioeconomic status on teacher perceptions. The Journal of Educational Research 101:4 (2008), 242–246, 10.3200/JOER.101.4.243-246.
9. Back, M., Nestler, S., Accuracy of judging personality. Hall, J.A., Schmid Mast, M., West, T.V., (eds.) The social psychology of perceiving others accurately, 2016, Cambridge University Press, Cambridge, 98–124.
10. Beck, K., Ensuring content validity of psychological and educational tests – the role of experts. Frontline Learning Research 8:6 (2020), 1–37, 10.14786/flr.v8i6.517.
11. Berliner, D.C., Implications of studies on expertise in pedagogy for teacher education and evaluation. Pfleiderer, J., (eds.) New directions for teacher assessment, 1988, Educational Testing Service, Princeton, 39–68.
12. Berliner, D.C., Learning about and learning from expert teachers. International Journal of Educational Research 35 (2001), 463–482, 10.1016/S0883-0355(02)00004-6.
13. Berliner, D.C., Describing the behavior and documenting the accomplishments of expert teachers. Bulletin of Science, Technology & Society 24:3 (2004), 200–212, 10.1177/0270467604265535.
14. Böckenholt, U., Analyzing multiple emotions over time by autoregressive negative multinomial regression models. Journal of the American Statistical Association 94:447 (1999), 757–765, 10.1080/01621459.1999.10474178.
15. Böhmer, I., Gräsel, C., Hörstermann, T., Krolak-Schwerdt, S., Die Informationssuche bei der Erstellung der Übergangsempfehlung – die Rolle von Fallkonsistenz und Expertise [Information search in decisions on school tracking recommendations – the influence of case consistency and expertise]. Unterrichtswissenschaft 40:2 (2012), 140–155.
16. Böhmer, I., Gräsel, C., Krolak-Schwerdt, S., Hörstermann, T., Glock, S., Teachers' school tracking decisions. Leutner, D., Fleischer, J., Grünkorn, J., Klieme, E., (eds.) Competence assessment in education. Research, models and instruments, 2017, Springer, Cham, 131–147.
17. Böhmer, I., Hörstermann, T., Gräsel, C., Krolak-Schwerdt, S., Glock, S., Eine Analyse der Informationssuche bei der Erstellung der Übergangsempfehlung: Welcher Urteilsregel folgen Lehrkräfte? [An analysis of information search in the process of making school tracking decisions: Which judgment rule do teachers apply?]. Journal for Educational Research Online 7:2 (2015), 59–81.
18. Boldt, A., Schiffer, A., Waszak, F., Yeung, N., Confidence predictions affect performance confidence and neural preparation in perceptual decision making. Scientific Reports, 9, 2019, 4031, 10.1038/s41598-019-40681-9.
19. Brennan, R.L., Performance Assessments from the perspective of generalizability theory. Applied Psychological Measurement 24:4 (2000), 339–353, 10.1177/01466210022031796.
20. Brühwiler, C., Adaptive Lehrkompetenz und schulisches Lernen. Effekte handlungssteuernder Kognitionen von Lehrpersonen auf Unterrichtsprozesse und Lernergebnisse der Schülerinnen und Schüler [Adaptive teaching competence and learning at school: Effects of behavioral cognitions of teachers on instruction and learning results of students]. 2014 Münster: Waxmann.
21. Byrnes, J.P., Cognitive development and learning in instructional contexts. 1996, Allyn & Bacon, Boston, MA.
22. Charness, G., Gneezy, U., Kuhn, M.A., Experimental methods: Between-subject and within-subject design. Journal of Economic Behavior & Organization 81:1 (2012), 1–8, 10.1016/j.learninstruc.2009.11.004.
23. Conrad, N., Patry, M.W., Conscientiousness and academic performance: A mediational analysis. International Journal for the Scholarship of Teaching & Learning 6:1 (2012), 1–14, 10.20429/IJSOTL.2012.060108.
24. Corno, L., On teaching adaptively. Educational Psychologist 43:3 (2008), 161–173, 10.1080/00461520802178466.
25. Corno, L., Snow, R.E., Adapting teaching to individual differences among learners. Wittrock, M.C., (eds.) Handbook of research on teaching, 3rd ed., 1986, MacMillan, New York, NY, 605–629.
26. Croskerry, P., A universal model of diagnostic reasoning. Academic Medicine 84:8 (2009), 1022–1028, 10.1097/ACM.0b013e3181ace703.
27. Csikszentmihalyi, M., Flow: The psychology of optimal experience. 1990, Harper Perennial, New York, NY.
28. Datnow, A., Hubbard, L., Teacher capacity for and beliefs about data-driven decision making: A literature review of international research. Journal of Educational Change 17:1 (2016), 7–28, 10.1007/s10833-015-9264-2.
29. De Boer, H., Bosker, R.J., Van der Werf, M.P.C., Sustainability of teacher expectation bias effects on long-term student performance. Journal of Educational Psychology 102:1 (2010), 168–179, 10.1037/a0017289.
30. Dusek, J.B., Joseph, G., The bases of teacher expectancies: A meta-analysis. Journal of Educational Psychology 75:3 (1983), 327–346, 10.1037/0022-0663.75.3.327.
31. Epstein, S., Pacini, R., Denes-Raj, V., Heier, H., Individual differences in intuitive-experiential and analytical-rational thinking styles. Journal of Personality and Social Psychology 71:2 (1996), 390–405, 10.1037/0022-3514.71.2.390.
32. Erber, R., Fiske, S.T., Outcome dependency and attention to inconsistent information. Journal of Personality and Social Psychology 47:4 (1984), 709–726, 10.1037/0022-3514.47.4.709.
33. Evans, J.S.B.T., Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology 59:1 (2008), 255–278, 10.1146/annurev.psych.59.103006.093629.
34. Eysink, T.H.S., Hulsbeek, M., Gijlers, H., Supporting primary school teachers in differentiating in the regular classroom. Teaching and Teacher Education 66 (2017), 107–116, 10.1016/j.tate.2017.04.002.
35. Faber, J.M., Luyten, H., Visscher, A.J., The effects of a digital formative assessment tool on mathematics achievement and student motivation: Results of a randomized experiment. Computers & Education 106 (2017), 83–96, 10.1016/j.compedu.2016.12.001.
36. Faul, F., Erdfelder, E., Lang, A., Buchner, A., G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods 39:2 (2007), 175–191, 10.3758/BF03193146.
37. Ferreira, M.B., Garcia-Marques, L., Sherman, S.J., Sherman, J.W., Automatic and controlled components of judgment and decision making. Journal of Personality and Social Psychology 91:5 (2006), 797–813, 10.1037/0022-3514.91.5.797.
38. Fiske, S.T., The continuum model and the stereotype content model. Van Lange, P.A.M., Kruglanski, A.W., Higgins, E.T., (eds.) Handbook of theories of social psychology, Vol. 1, 2012, SAGE Publications, Los Angeles, 267–288.
39. Fiske, S.T., Neuberg, S.L., A continuum of impression formation from category-based to individuating processes: Influences of information and motivation on attention and interpretation. Zanna, M.P., (eds.) Advances in experimental social psychology, 1990, Academic Press, New York, NY, 1–74.
40. Fiske, S.T., Neuberg, S.L., Beattie, A.E., Milberg, S.J., Category-based and attribute-based reactions to others: Some informational conditions of stereotyping and individuating processes. Journal of Experimental Social Psychology 23:5 (1987), 399–427, 10.1016/0022-1031(87)90038-2.
41. Ghani, J.A., Flow in human-computer interactions: Test of a model. Carey, J.M., (eds.) Human factors in information systems: Emerging theoretical bases, 1995, Ablex Publishing Corporation, Norwood, NJ, 291–312.
42. Glock, S., Krolak-Schwerdt, S., Klapproth, F., Böhmer, M., Beyond judgment bias: How students' ethnicity and academic profile consistency influence teachers' tracking judgments. Social Psychology of Education 16:4 (2013), 555–573, 10.1007/s11218-013-9227-5.
43. Hagerty, M.R., Aaker, D.A., A normative model of consumer information processing. Marketing Science 3:3 (1984), 227–246, 10.1287/mksc.3.3.227.
44. Hand, S., Rice, L., Greenlee, E., Exploring teachers' and students' gender role bias and students' confidence in STEM fields. Social Psychology of Education 20:4 (2017), 929–945, 10.1007/s11218-017-9408-8.
45. Harteis, C., Billett, S., Intuitive expertise: Theories and empirical evidence. Educational Research Review 9 (2013), 145–157, 10.1016/j.edurev.2013.02.001.
46. Hermkes, R., Mach, H., Minnameier, G., Interaction-based coding of scaffolding processes. Learning and Instruction 54 (2018), 147–155, 10.1016/j.learninstruc.2017.09.003.
47. Herrin, J., Cltest: Stata modules for performing cluster-adjusted chi-square and t-tests [statistical software components S424901. 2002, Boston College Department of Economics] Retrieved from https://ideas.repec.org/c/boc/bocode/s424901.html.
48. Hoge, R.D., Coladarci, T., Teacher based judgments of academic achievement: A review of the literature. Review of Educational Research 59 (1989), 297–313, 10.2307/1170184.
49. Houtveen, T., Van de Grift, W., Inclusion and adaptive instruction in elementary education. Journal of Education for Students Placed at Risk 6:4 (2001), 389–409, 10.1207/S15327671ESPR0604_5.
50. Hoyle, R.H., Statistical strategies for small sample research. 1999, SAGE Publications, Thousand Oaks, CA.
51. Judson, R.A., Owen, A., Estimating dynamic panel data models: A guide for macroeconomists. Economics Letters 65:1 (1999), 9–15, 10.1016/S0165-1765(99)00130-5.
52. Kalyuga, S., Expertise reversal effect and its implications for learner-tailored instruction. Educational Psychology Review 19:4 (2007), 509–539, 10.1007/s10648-007-9054-3.
53. Kärner, T., Warwas, J., Schumann, S., A Learning Analytics Approach to Address Heterogeneity in the Classroom: The Teachers’ Diagnostic Support System. Technology, Knowledge and Learning, 2020, 10.1007/s10758-020-09448-4.
54. Klassen, R.M., Tze, V.M.C., Teachers' self-efficacy, personality, and teaching effectiveness: A meta-analysis. Educational Research Review 12 (2014), 59–76, 10.1016/j.edurev.2014.06.001.
55. Konstantopoulos, S., The mean is not enough: Using quantile regression to examine trends in Asian-white differences across the entire achievement distribution. Teachers College Record 111:5 (2009), 1274–1295.
56. Krannich, M., Goetz, T., Lipnevich, A.A., Bieg, M., Roos, A.-L., Becker, E.S., et al. Being over- or underchallenged in class: Effects on students' career aspirations via academic self-concept and boredom. Learning and Individual Differences 69 (2019), 206–218, 10.1016/j.lindif.2018.10.004.
57. Kulik, J.A., Kulik, C.-L.C., Meta-analytic findings on grouping programs. Gifted Child Quarterly 36:2 (1992), 73–77, 10.1177/001698629203600204.
58. Landis, J.R., Koch, G.G., The measurement of observer agreement for categorical data. Biometrics 33:1 (1977), 159–174, 10.2307/2529310.
59. Lindberg, S.M., Hyde, J.S., Petersen, J.L., New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin 136:6 (2010), 1123–1135, 10.1037/a0021276.
60. Machts, N., Kaiser, J., Schmidt, F.T., Möller, J., Accuracy of teachers' judgments of students' cognitive abilities: A meta-analysis. Educational Research Review 19 (2016), 85–103, 10.1016/j.edurev.2016.06.003.
61. Marsh, H.W., Academic self-concept: Theory, measurement, and research. Suls, J., (eds.) Psychological perspectives on the self, 2014, Psychology Press, New York, NY, 71–110.
62. McIlroy, D., Poole, K., Ursavas, Ö.F., Moriarty, A., Distal and proximal associates of academic performance at secondary level: A mediation model of personality and self-efficacy. Learning and Individual Differences 38 (2015), 1–9, 10.1016/j.lindif.2015.01.004.
63. Murray, H.G., Low-inference teaching behaviors and college teaching effectiveness: Recent developments and controversies. Perry, R.P., Smart, J.C., (eds.) The scholarship of teaching and learning in higher education: An evidence-based perspective, 2007, Springer, Dordrecht, 145–200, 10.1007/1-4020-5742-3_6.
64. Mushquash, C., O Connor, B.P., SPSS and SAS programs for generalizability theory analyses. Behavior Research Methods 38:3 (2006), 542–547, 10.3758/BF03192810.
65. Newson, R., Confidence intervals for rank statistics: Somers' D and extensions. STATA Journal 6:3 (2006), 309–334, 10.1177/1536867X0600600302.
66. Peterson, D.K., Pitz, G.F., Confidence, uncertainty, and the use of information. Journal of Experimental Psychology 14:1 (1988), 85–92, 10.1037/0278-7393.14.1.85.
67. Pit-Ten Cate, I.M., Hörstermann, T., Krolak-Schwerdt, S., Gräsel, C., Böhmer, I., Glock, S., Teachers’ information processing and judgement accuracy: Effects of information consistency and accountability. European Journal of Psychology of Education 35 (2020), 675–702, 10.1007/s10212-019-00436-6.
68. Power, D.J., Decision support systems: Concepts and resources for managers. 2002, Quorum Books, Westport, CT.
69. Praetorius, A.-K., Berner, V.-D., Zeinz, H., Scheunpflug, A., Dresel, M., Judgment confidence and judgment accuracy of teachers in judging academic self-concepts of students. The Journal Of Educational Research 106:1 (2013), 64–76, 10.1080/00220671.2012.667010.
70. Rausch, A., Dispositional predictors of problem solving in the field of office work. Vocations and Learning 10:2 (2017), 177–199, 10.1007/s12186-016-9165-4.
71. Reis, S.M., McCoach, D.B., Little, C.M., Muller, L.M., Kaniskan, R.B., The effects of differentiated instruction and enrichment pedagogy on reading achievement in five elementary schools. American Educational Research Journal 48:2 (2011), 462–501, 10.3102/0002831210382891.
72. Rieu, A., Loib, K., Leuder, T., Herppich, St, Diagnostisches Urteilen als informationsverarbeitender Prozess – Wie nutzen Lehrkräfte ihr Wissen bei der Identifizierung und Gewichtung von Anforderungen in Aufgaben? [Judging task difficulty: Effects of PCK and time pressure on cognitive processes]. Unterrichtswissenschaft., 2020, 10.1007/s42010-020-00071-x Advance online publication.
73. Santangelo, T., Tomlinson, C.A., Teacher educators' perceptions and use of differentiated instruction practices: An exploratory investigation. Action in Teacher Education 34:4 (2012), 309–327, 10.1080/01626620.2012.717032.
74. Schunk, D.H., Pajares, F., Self-efficacy theory. Wenzel, K.R., Miele, D.B., (eds.) Educational psychology handbook series. Handbook of motivation at school, 2009, Routledge, New York. NY, 35–53.
75. Screencast O Matic, Screencast-O-Matic® [Computer software]. 2019 Retrieved from https://screencast-o-matic.com/.
76. Sembill, D., Wirtschaftslehreunterricht: Einige Forschungsergebnisse zum Zusammenhang von Verbalurteilen und Handeln am Beispiel der Unterrichtseinheit „Kaufvertrag" [Economics education: Research results on relationships between teacher judgements and teachers' didactical actions in the teaching unit ‘purchase contract’]. Zeitschrift für Berufs- und Wirtschaftspädagogik 83 (1987), 213–232.
77. Sembill, D., Wolf, K.D., Wuttke, E., Schumacher, L., Self-organized learning in vocational education – foundation, implementation, and evaluation. Beck, K., (eds.) Teaching-learning processes in vocational education. Foundations of modern training programmes, 2002, Lang, Frankfurt, 267–295.
78. Shavelson, R.J., Webb, N.M., Rowley, G.L., Generalizability theory. American Psychologist 44:6 (1989), 922–932, 10.1037/0003-066X.44.6.922.
79. Sherman, J.W., Conrey, F.R., Groom, C.J., Encoding flexibility revisited: Stereotype-inconsistent information under cognitive load. Social Cognition 22:2 (2004), 214–232, 10.1521/soco.22.2.214.35464.
80. Slavin, R.E., Ability grouping and student achievement in elementary schools: A best‐evidence synthesis. Review of Educational Research 57:3 (1987), 293–336, 10.2307/1170460.
81. Srull, T.K., Wyer, R.S., Person memory and judgment. Psychological Review 96:1 (1989), 58–83, 10.1037/0033-295X.96.1.58.
82. Stajkovic, A.D., Bandura, A., Locke, E.A., Lee, D., Sergent, K., Test of three conceptual models of influence of the big five personality traits and self-efficacy on academic performance: A meta-analytic path-analysis. Personality and Individual Differences 120 (2018), 238–245, 10.1016/j.paid.2017.08.014.
83. Stata 16, Xtabond — Arellano–Bond linear dynamic panel-data estimation. Retrieved from https://www.stata.com/manuals13/xtxtabond.pdf, 2020.
84. Südkamp, A., Kaiser, J., Möller, J., Accuracy of teachers' judgments of students' academic achievement: A meta-analysis. Journal of Educational Psychology 104:3 (2012), 743–762, 10.1037/a0027627.
85. Südkamp, A., Praetorius, A.-K., Spinath, B., Teachers' judgment accuracy concerning consistent and inconsistent student profiles. Teaching and Teacher Education 76 (2018), 204–213, 10.1016/j.tate.2017.09.016.
86. Sweller, J., Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction 4:4 (1994), 295–312, 10.1016/0959-4752(94)90003-5.
87. Sweller, J., Ayres, P., Kalyuga, S., Cognitive load theory. 2011, Springer, New York, NY.
88. Sweller, J., Van Merriënboer, J.J.G., Paas, F., Cognitive architecture and instructional design: 20 years later. Educational Psychology Review 31 (2019), 261–292, 10.1007/s10648-019-09465-5.
89. Tanilon, J., Performance samples on academic tasks: Improving prediction of academic performance. 2011, Doctoral dissertation, University of Leiden Retrieved from https://openaccess.leidenuniv.nl/bitstream/handle/1887/17890/Doctoral_thesis_Jenny_Tanilon%5b1%5d.pdf?sequence=16.
90. Urhahne, D., Chao, S.-H., Florineth, M.L., Luttenberger, S., Paechter, M., Academic self- concept, learning motivation, and test anxiety of the underestimated student. British Journal of Educational Psychology 81:1 (2011), 161–177, 10.1348/000709910X504500.
91. Vanlommel, K., Van Gasse, R., Vanhoof, J., Van Petegem, P., Teachers' high-stakes decision making. How teaching approaches affect rational and intuitive data collection. Teaching and Teacher Education 71:1 (2018), 108–119, 10.1016/j.tate.2017.12.011.
92. Vygotsky, L.S., Mind in society. The development of higher psychological processes. 1978, Harvard University Press, Cambridge, MA.