Students' Idiosyncratic perceptions of teaching quality in mathematics: A result of rater tendency alone or an expression of dyadic effects between students and teachers?

Journal of Educational Psychology

Volumn 110, Issue 5, 2018, Pages 709-725

  Göllner, Richard ^a   Wagner, Wolfgang ^a   Eccles, Jacquelynne S ^b   Trautwein, Ulrich ^a  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Dyadic student teacher effects; Rating tendencies; Students' idiosyncratic perceptions; Teaching quality

Indexed keywords

EID: 85040745786     PISSN: 00220663     EISSN: 19392176     Source Type: Journal    
DOI: 10.1037/edu0000236     Document Type: Article

References (72)

1. Baumert, J., Lehmann, R., Lehrke, M., Schmitz, B., Clausen, M., Hosenfeld, I., Neubrand, J. (1997). TIMSS-Mathematisch-naturwissenschaftlicher Unterricht im internationalen Vergleich [TIMSS-Mathematics and science instruction in international comparison]. Opladen, Germany: Leske & Buderich
2. Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186-203. http://dx.doi.org/10.1207/s15328007sem1302_2
3. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley. http://dx.doi.org/10.1002/9781118619179
4. Brophy, J., & Good, T. (1996). Teacher behavior and student achievement. In M. C. Wittrock (Ed.), Third handbook of research on teaching (pp. 328-375). Washington, DC: American Educational Research Association
5. Cable, D. M., & DeRue, D. S. (2002). The convergent and discriminant validity of subjective fit perceptions. Journal of Applied Psychology, 87, 875-884. http://dx.doi.org/10.1037/0021-9010.87.5.875
6. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504. http://dx.doi.org/10.1080/10705510701301834
7. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. http://dx.doi.org/10.1207/S15328007SEM0902_5
8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum
9. Congdon, P. J., & McQueen, J. (2000). The stability of rater severity in large-scale assessment programs. Journal of Educational Measurement, 37, 163-178. http://dx.doi.org/10.1111/j.1745-3984.2000.tb01081.x
10. Corey, S. M., & Beery, G. S. (1938). Effect of teacher popularity upon attitude toward school subjects. Journal of Educational Psychology, 29, 665-670. http://dx.doi.org/10.1037/h0054066
11. Corno, L. (1995). The principles of adaptive teaching. In A. C. Ornstein (Ed.), Teaching: Theory into practice (pp. 98-115). Boston, MA: Allyn & Bacon
12. Corno, L. (2008). On teaching adaptively. Educational Psychologist, 43, 161-173. http://dx.doi.org/10.1080/00461520802178466
13. Creemers, B. P. M., Kyriakides, L., & Antoniou, P. (2013). Establishing theoretical frameworks to describe teacher effectiveness. In B. P. M. Creemers, L. Kyriakides, & P. Antoniou (Eds.), Teacher professional development for improving quality of teaching (pp. 101-135). Dordrecht, the Netherlands: Springer. http://dx.doi.org/10.1007/978-94-007-5207-8_7
14. Den Brok, P., Brekelmans, M., & Wubbels, T. (2006). Multilevel issues in research using students' perceptions of learning environments: The case of the questionnaire on teacher interaction. Learning Environments Research, 9, 199-213. http://dx.doi.org/10.1007/s10984-006-9013-9
15. Den Brok, P., Fisher, D., Rickards, T., & Bull, E. (2006). Californian science students' perceptions of their classroom learning environments. Educational Research and Evaluation, 12, 3-25. http://dx.doi.org/10.1080/13803610500392053
16. Eccles, J. S., & Midgley, C. (1989). Stage-environment fit: Developmentally appropriate classrooms for early adolescents. In R. Ames & C. Ames (Eds.), Research on motivation in education (Vol. 3, pp. 139-181). New York, NY: Academic Press
17. Eccles, J. S., Midgley, C., Wigfield, A., Buchanan, C. M., Reuman, D., Flanagan, C., & Iver, D. M. (1993). Development during adolescence. The impact of stage-environment fit on young adolescents' experiences in schools and in families. American Psychologist, 48, 90-101. http://dx.doi.org/10.1037/0003-066X.48.2.90
18. Eccles, J. S., & Roeser, R. W. (2009). Schools, academic motivation, and stage-environment fit. In R. M. Lerner & L. Steinberg (Eds.), Handbook of adolescent psychology: Individual bases of adolescent development (3rd ed., Vol. 1, pp. 404-434). Hoboken, NJ: Wiley
19. Edwards, J. R., Caplan, R. D., & Harrison, R. V. (1998). Personenvironment fit theory: Conceptual foundations, empirical evidence, and directions for future research. In C. L. Cooper (Ed.), Theories of organizational stress (pp. 28-67). Oxford, UK: Oxford University Press
20. Edwards, J. R., & Shipp, A. J. (2007). The relationship between personenvironment fit and outcomes: An integrative theoretical framework. In C. Ostroff & T. A. Judge (Eds.), Perspectives on organizational fit (pp. 209-258). New York, NY: Erlbaum
21. Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8, 430-457. http://dx.doi.org/10.1207/S15328007SEM0803_5
22. Fauth, B., Decristan, J., Rieser, S., Klieme, E., & Büttner, G. (2014). Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes. Learning and Instruction, 29, 1-9. http://dx.doi.org/10.1016/j.learninstruc.2013.07.001
23. Feldlaufer, H., Midgley, C., & Eccles, J. S. (1988). Student, teacher, and observer perceptions of the classroom environment before and after the transition to junior high school. The Journal of Early Adolescence, 8, 133-156. http://dx.doi.org/10.1177/0272431688082003
24. Ferguson, R. F., & Danielson, C. (2014). How framework for teaching and Tripod 7Cs evidence distinguish key components of effective teaching. In T. Kane, K. Kerr, & R. Pianta (Eds.), Designing teacher evaluation systems: New guidance from the Measures of Effective Teaching project (pp. 98-143). San Francisco, CA: Jossey-Bass
25. Ferrer, E., Balluerka, N., & Widaman, K. F. (2008). Factorial invariance and the specification of second-order latent growth models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4, 22-36. http://dx.doi.org/10.1027/1614-2241.4.1.22
26. Fisicaro, S. A., & Lance, C. E. (1990). Implications of three causal models for the measurement of halo error. Applied Psychological Measurement, 14, 419-429. http://dx.doi.org/10.1177/014662169001400407
27. Fraser, B. J. (1991). Two decades of classroom environment research. In B. J. Fraser & H. J. Walberg (Eds.), Educational environments: Evaluation, antecedents and consequences (pp. 3-29). Oxford, UK: Pergamon Press
28. Freedman, S. W. (1979). How characteristics of student essays influence teachers' evaluations. Journal of Educational Psychology, 71, 328-338. http://dx.doi.org/10.1037/0022-0663.71.3.328
29. Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141, 2-18. http://dx.doi.org/10.1037/a0024338
30. Greenwald, A. G., & Gillmore, G. M. (1997). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209-1217. http://dx.doi.org/10.1037/0003-066X.52.11.1209
31. Griffin, B. (2004). Grading leniency, grade discrepancy, and student ratings of instruction. Contemporary Educational Psychology, 29, 410-425. http://dx.doi.org/10.1016/j.cedpsych.2003.11.001
32. Guilford, J. P. (1954). Psychometric methods. New York, NY: McGraw-Hill
33. Hamre, B. K., & Pianta, R. C. (2010). Classroom environments and developmental processes: Conceptualization and measurement. In J. L. Meece & J. S. Eccles (Eds.), Handbook of research on schools, schooling and human development (pp. 25-41). New York, NY: Routledge
34. Hattie, J. (2009). Visible learning: A synthesis of meta-analyses relating to achievement. London, UK: Routledge
35. Hoyt, W. T. (2000). Rater bias in psychological research: When is it a problem and what can we do about it? Psychological Methods, 5, 64-86. http://dx.doi.org/10.1037/1082-989X.5.1.64
36. Hu, L., & Bentler, P. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling. Concepts, issues, and applications (pp. 76-99). London, UK: Sage
37. Jak, S., Oort, F. J., & Dolan, C. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling, 20, 265-282. http://dx.doi.org/10.1080/10705511.2013.769392
38. Kish, L. (1965). Survey sampling. New York, NY: Wiley
39. Kornrich, S., & Furstenberg, F. (2013). Investing in children: Changes in parental spending on children, 1972-2007. Demography, 50, 1-23. http://dx.doi.org/10.1007/s13524-012-0146-4
40. Kunter, M., Baumert, J., & Köller, O. (2007). Effective classroom management and the development of subject-related interest. Learning and Instruction, 17, 494-509. http://dx.doi.org/10.1016/j.learninstruc.2007.09.002
41. Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105, 805-820. http://dx.doi.org/10.1037/a0032583
42. Kunter, M., Tsai, Y.-M., Klusmann, U., Brunner, M., Krauss, S., & Baumert, J. (2008). Students' and mathematics teachers' perceptions of teacher enthusiasm and instruction. Learning and Instruction, 18, 468-482. http://dx.doi.org/10.1016/j.learninstruc.2008.06.008
43. Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational Research Journal, 32, 465-491. http://dx.doi.org/10.3102/00028312032003465
44. Lam, A. C., Schenke, K., Conley, A. M., Ruzek, E. A., & Karabenick, S. A. (2015). Student perceptions of classroom achievement goal structure: Is it appropriate to aggregate? Journal of Educational Psychology, 107, 1102-1115. http://dx.doi.org/10.1037/edu0000028
45. Lance, C. E., LaPointe, J. A., & Fisicaro, S. A. (1994). Tests of three causal models of halo rater error. Organizational Behavior and Human Decision Processes, 57, 83-96. http://dx.doi.org/10.1006/obhd.1994.1005
46. Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34, 120-131. http://dx.doi.org/10.1016/j.cedpsych.2008.12.001
47. Murphy, K. R., & Anhalt, R. L. (1992). Is halo error a property of the rater, ratees, or the specific behaviors observed? Journal of Applied Psychology, 77, 494-500. http://dx.doi.org/10.1037/0021-9010.77.4.494
48. Muthén, B. (1989). Latent variable modeling in heterogeneous populations. Psychometrika, 54, 557-585. http://dx.doi.org/10.1007/BF02296397
49. Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. Sociological Methodology, 25, 267-316. http://dx.doi.org/10.2307/271070
50. Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus user's guide (7th ed.). Los Angeles, CA: Author
51. Organisation for Economic Co-operation and Development. (2004). Learning for tomorrow's world: First results from PISA 2003. Paris, France: Author
52. Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101, 115-135. http://dx.doi.org/10.1037/a0013383
53. Pekrun, R., Goetz, T., & Frenzel, A. C. (2005). Academic emotions questionnaire-mathematics (AEC-M) user's manual. Munich, Germany: University of Munich
54. Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom Assessment Scoring System (CLASS: PreK-3). Baltimore, MD: Brookes
55. Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., Neubrand, M., Schiefele, U. (2007): PISA 2003 (Version 1, Code book and dataset). Berlin, Germany: Institute for Educational Quality Improvement. http://dx.doi.org/10.5159/IQB_PISA_2003_v1
56. Prenzel, M., Baumert, J., Blum, W., Lehmann, R., Leutner, D., Neubrand, M., Schiefele, U. (2013): PISA-I-Plus 2003, 2004 (Version 1, Code book and dataset). Berlin, Germany: Institute for Educational Quality Improvement. http://dx.doi.org/10.5159/IQB_PISA_I_Plus_v1
57. Purves, A. C. (1987). The evolution of the IEA: A memoir. Comparative Education Review, 31, 10-28. http://dx.doi.org/10.1086/446653
58. Quin, L. (2017). Longitudinal and contextual associations between teacher-student relationships and student engagement: A systematic review. Review of Educational Research, 87, 345-387
59. Rhemtulla, M., Brosseau-Liard, P. é., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373. http://dx.doi.org/10.1037/a0029315
60. Scheerens, J., & Bosker, R. J. (1997). The foundations of educational effectiveness. Oxford, UK: Elsevier
61. Scherer, R., & Gustafsson, J. E. (2015). Student assessment of teaching as a source of information about aspects of teaching quality in multiple subject domains: An application of multilevel bifactor structural equation modeling. Frontiers in Psychology, 6, 1550. http://dx.doi.org/10.3389/fpsyg.2015.01550
62. Scullen, S. E., Mount, M. K., & Goff, M. (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85, 956-970. http://dx.doi.org/10.1037/0021-9010.85.6.956
63. Snijders, T., & Bosker, R. (2012). Multilevel analysis (2nd ed.). Thousand Oaks, CA: Sage
64. Snow, R. E. (1980). Aptitude, learner control, and adaptive instruction. Educational Psychologist, 15, 151-158. http://dx.doi.org/10.1080/00461528009529223
65. Steyer, R., & Schmitt, M. (1990). The effects of aggregation across and within occasions on consistency, specificity, and reliability. Methodika, 4, 58-94
66. Steyer, R., Schmitt, M., & Eid, M. (1999). Latent state-trait theory and research in personality and individual differences. European Journal of Personality, 13, 389-408. http://dx.doi.org/10.1002/(SICI)1099-0984(199909/10)13:5<389::AID-PER361>3.0.CO;2-A
67. Turner, J. C., & Meyer, D. K. (2000). Studying and understanding the instructional contexts of classrooms: Using our past to forge our future. Educational Psychologist, 35, 69-85. http://dx.doi.org/10.1207/S15326985EP3502_2
68. Villanova, P., Bernardin, J., Dahmus, S. A., & Sims, R. L. (1993). Rater leniency and performance appraisal discomfort. Educational and Psychological Measurement, 53, 789-799. http://dx.doi.org/10.1177/0013164493053003023
69. Wagner, W., Göllner, R., Helmke, A., Trautwein, U., & Lüdtke, O. (2013). Construct validity of student perceptions of instructional quality is high, but not perfect: Dimensionality and generalizability of domainindependent assessments. Learning and Instruction, 28, 1-11. http://dx.doi.org/10.1016/j.learninstruc.2013.03.003
70. Wagner, W., Göllner, R., Werth, S., Voss, T., Schmitz, B., & Trautwein, U. (2016). Student and teacher ratings of instructional quality: Consistency of ratings over time, agreement, and predictive power. Journal of Educational Psychology, 108, 705-721. http://dx.doi.org/10.1037/edu0000075
71. Wallace, T. L., Kelcey, B., & Ruzek, E. (2016). What can student perception surveys tell us about teaching? Empirically testing the underlying structure of the Tripod Student Perception Survey. American Educational Research Journal, 53, 1834-1868
72. Wang, M. C., Haertel, G. D., & Walberg, H. J. (1993). Toward a knowledge base for school learning. Review of Educational Research, 63, 249-294. http://dx.doi.org/10.3102/00346543063003249