Attending to General and Mathematics-Specific Dimensions of Teaching: Exploring Factors Across Two Observation Instruments

Educational Assessment

Volumn 22, Issue 2, 2017, Pages 71-94

  Blazar, David ^a   Braslow, David ^a   Charalambous, Charalambos Y ^b   Hill, Heather C ^a  

^a HARVARD GRADUATE SCHOOL OF EDUCATION   (United States)

^b UNIVERSITY OF CYPRUS   (Cyprus)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85018163778     PISSN: 10627197     EISSN: None     Source Type: Journal    
DOI: 10.1080/10627197.2017.1309274     Document Type: Article

References (88)

1. Allen, J. P., Pianta, R. C., Gregory, A., Mikami, A. Y., & Lun, J., (2011). An interaction-based approach to enhancing secondary school instruction and student achievement. Science, 333, 1034–1037.
2. Ball, D. L., (1988). Research on teaching mathematics:Making subject matter knowledge part of the equation. East Lansing, MI:National Center for Research on Teacher Education.
3. Ball, D. L., Thames, M. H., & Phelps, G., (2008). Content knowledge for teaching what makes it special? Journal of Teacher Education, 59, 389–407.
4. Bell, C. A., Gitomer, D. H., McCaffrey, D. F., Hamre, B. K., & Pianta, R. C., (2012). An argument approach to observation protocol validity. Educational Assessment, 17(2–3), 62–87.
5. Blazar, D., (2015). Effective teaching in elementary mathematics:Identifying classroom practices that support student achievement. Economics of Education Review, 48, 16–29.
6. Blazar, D., & Kraft, M. A., (2015). Exploring mechanisms of effective teacher coaching:A tale of two cohorts from a randomized experiment. Educational Evaluation and Policy Analysis, 37, 542–566.
7. Blazar, D., & Kraft, M. A., (2017). Teacher and teaching effects on students’ attitudes and behaviors. Educational Evaluation and Policy Analysis, 39, 146–170.
8. Blazar, D., Litke, E., & Barmore, J., (2016). What does it mean to be ranked a “high” or “low” value-added teacher? Observing differences in instructional quality across districts. American Educational Research Journal, 53, 324–359.
9. Brophy, J., (1986). Teaching and learning mathematics:Where research should be going? Journal for Research in Mathematics Education, 17, 323–346.
10. Brophy, J. E., & Good, T. L., (1986). Teacher behavior and student achievement. In M. C., Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 328–375). New York, NY:Macmillan.
11. Campbell, D. T., & Fiske, D. W., (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, 81–105.
12. Campbell, P. F., & Malkus, N. N., (2011). The impact of elementary mathematics coaches on student achievement. The Elementary School Journal, 111, 430–454.
13. Center on Great Teachers and Leaders. (2013). Databases on state teacher and principal policies. Retrieved from http://resource.tqsource.org/stateevaldb
14. Charalambous, C. Y., & Kyriakides, E., (2017). Working at the nexus of generic and content-specific teaching practices:An exploratory study based on TIMSS secondary analyses. Elementary School Journal, 117(3), 423–454.
15. Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J.-P., & Zhang, Z., (2012). Modeling general and specific variance in multifaceted constructs:A comparison of the bifactor model to other approaches. Journal of Personality, 80, 219–251.
16. Chetty, R., Friedman, J. N., & Rockoff, J. E., (2014). Measuring the impacts of teachers II:Teacher value-added and student outcomes in adulthood. American Economic Review, 104, 2633–2679.
17. Cohen, D. K., (2011). Teaching and its predicaments. Cambridge, MA:Harvard University Press.
18. Cohen, D. K., & Ball, D. L., (1999). Instruction, capacity, and improvement (CPRE Research Report Series RR–43). Philadelphia, PA:Consortium for Policy Research in Education, University of Pennsylvania Graduate School of Education.
19. Crocker, L., & Algina, J., (2008). Introduction to classical and modern test theory. Mason, OH:Cengage Learning.
20. Danielson Group. (2013a). The Framework for Teaching Evaluation Instrument. Princeton, NJ:Author.
21. Danielson Group. (2013b). General questions about the framework. Retrieved from https://www.danielsongroup.org/questions-about-the-framework-for-teaching/
22. Danielson, C., & McGreal, T. L., (2000). Teacher evaluation to enhance professional practice. Alexandria, VA:Association for Supervision and Curriculum Development.
23. Darling-Hammond, L. (2013). Getting teacher evaluation right:What really matters for effectiveness and improvement. New York, NY:Teachers College Press.
24. Darling-Hammond, L., Wei, R. C., Andree, A., Richardson, N., & Orphanos, S., (2009). Professional learning in the learning profession:A status report on teacher development in the United States and abroad. Palo Alto, CA:National Staff Development Council and The School Redesign Network, Stanford University.
25. Darling-Hammond, L., Wise, A. E., & Pease, S. R., (1983). Teacher evaluation in the organizational context:A review of the literature. Review of Educational Research, 53, 285–328.
26. Downer, J. T., Rimm-Kaufman, S., & Pianta, R. C., (2007). How do classroom conditions and children’s risk for school problems contribute to children’s behavioral engagement in learning? School Psychology Review, 36, 413–432.
27. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J., (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299.
28. Field, A., (2013). Discovering statistics using IBM SPSS statistics (4th ed.). London, England:Sage.
29. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S., (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945.
30. Gargani, J., & Strong, M., (2014). Can we identify a successful teacher better, faster, and cheaper? Evidence for innovating teacher observation systems. Journal of Teacher Education, 65, 389–401.
31. Good, T. L., & Lavigne, A. L., (2015). Rating teachers cheaper, faster, and better:Not so fast. Journal of Teacher Education, 66, 288–293.
32. Grossman, P., Loeb, S., Cohen, J., & Wyckoff, J., (2013). Measure for measure:The relationship between measures of instructional practice in middle school English language arts and teachers’ value-added. American Journal of Education, 199, 445–470.
33. Grossman, P., & McDonald, M., (2008). Back to the future:Directions for research in teaching and teacher education. American Educational Research Journal, 45, 184–205.
34. Gustafsson, J., & Balke, G., (1993). General and specific abilities as predictors of school achievement. Multivariate Behavioral Research, 28, 407–434.
35. Hafen, C. A., Hamre, B. K., Allen, J. P., Bell, C. A., Gitomer, D. H., & Pianta, R. C., (2015). Teaching through interactions in secondary school classrooms:Revisiting the factor structure and practical application of the Classroom Assessment Scoring System–Secondary. Journal of Early Adolescence, 35(5–6), 651–680. doi:10.1177/0272431614537117
36. Hamre, B., Hatfield, B., Pianta, R., & Jamil, F., (2014). Evidence for general and domain-specific elements of teacher–child interactions:Associations with preschool children’s development. Child Development, 85, 1257–1274.
37. Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A. J., Jones, S. M., … Brackett, M. A., (2013). Teaching through interactions:Testing a developmental framework of teacher effectiveness in over 4,000 classrooms. The Elementary School Journal, 113, 461–487.
38. Hamre, B. K., & Pianta, R. C., (2001). Early teacher–child relationships and the trajectory of children’s school outcomes through eighth grade. Child Development, 72, 625–638.
39. Hayton, J. C., Allen, D. G., & Scarpello, V., (2004). Factor retention decisions in exploratory factor analysis:A tutorial on parallel analysis. Organizational Research Methods, 7, 191–205.
40. Herman J. L., & Baker, E. L., (2009). Assessment policy:Making sense of the Babel. In G., Sykes, B., Schneider, & D. N., Plank (Eds.), Handbook of educational policy research (pp. 176–190). New York, NY:Routledge.
41. Hill, H. C., (2007). Learning in the teacher workforce. Future of Children, 17, 111–127.
42. Hill, H. C., (2010). The nature and predictors of elementary teachers’ mathematical knowledge for teaching. Journal for Research in Mathematics Education, 41(5), 513–545.
43. Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L., (2008). Mathematical knowledge for teaching and the mathematical quality of instruction:An exploratory study. Cognition and Instruction, 26, 430–511.
44. Hill, H. C., Charalambous, C. Y., Blazar, D., McGinn, D., Kraft, M. A., Beisiegel, M., … Lynch, K., (2012). Validating arguments for observational instruments:Attending to multiple sources of variation. Educational Assessment, 17, 88–106.
45. Hill, H. C., Charalambous, C. Y., & Kraft, M. A., (2012). When rater reliability is not enough:Teacher observation systems and a case for the generalizability study. Educational Researchers, 41, 56–64.
46. Hill, H. C., & Grossman, P., (2013). Learning from teacher observations:Challenges and opportunities posed by new teacher evaluation systems. Harvard Educational Review, 83, 371–384.
47. Hill, H. C., Kapitula, L., & Umland, K., (2011). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48, 794–831.
48. Ho, A. D., & Kane, T. J., (2013). The reliability of classroom observations by school personnel. Seattle, WA:Measures of Effective Teaching Project, Bill and Melinda Gates Foundation.
49. Hu, L. T., & Bentler, P. M., (1999). Cutoff criteria for fit indexes in covariance structure analysis:Conventional criteria versus new alternatives. Structural Equation Modeling:A Multidisciplinary Journal, 6, 1–55.
50. Jacob, B. A., & Lefgren, L., (2008). Can principals identify effective teachers? Evidence on subjective performance evaluation in education. Journal of Labor Economics, 20, 101–136.
51. Kaiser, H., (1974). An index of factorial simplicity. Psychometrika, 39, 31–36.
52. Kane, T. J., & Staiger, D. O., (2012). Gathering feedback for teaching:Combining high-quality observations with student surveys and achievement gains (Research paper). Seattle, WA:Measures of Effective Teaching Project, Bill and Melinda Gates Foundation.
53. Kane, T. J., Taylor, E. S., Tyler, J. H., & Wooten, A. L., (2011). Identifying effective classroom practices using student achievement data. Journal of Human Resources, 46, 587–613.
54. Kline, P., (1994). An easy guide to factor analysis. London, UK:Routledge.
55. Kline, R. B., (2011). Principles and practice of structural equation modeling. New York, NY:Guilford Press.
56. Lampert, M., (2001). Teaching problems and the problems of teaching. New Haven, CT:Yale University Press.
57. Leinhardt, G., (1993). On teaching. In R., Glaser (Ed.), Advances in instructional psychology (Vol.4, pp. 1–54). Hillsdale, NJ:Erlbaum.
58. Little, J., (2001). Professional development in the pursuit of school reform. In A., Lieberman & L., Miller (Eds.), Teachers caught in the action (pp. 23–44). New York, NY:Teachers College Press.
59. Lockwood, J. R., Savitsky, T. D., & McCaffrey, D. F., (2015). Inferring constructs of effective teaching from classroom observations:An application of Bayesian exploratory factor analysis without restrictions. The Annals of Applied Statistics, 9, 1484–1509.
60. Luckner, A. E., & Pianta, R. C., (2011). Teacher–student interactions in fifth grade classrooms:Relations with children’s peer behavior. Journal of Applied Developmental Psychology, 32, 257–266.
61. Mathematical Quality of Instruction. (2014). Retrieved from http://isites.harvard.edu/icb/icb.do?keyword=mqi_training&pageid=icb.page394761
62. McCaffrey, D. F., Yuan, K., Savitsky, T. D., Lockwood, J. R., & Edelen, M. O., (2015). Uncovering multivariate structure in classroom observations in the presence of rater errors. Educational Measurement:Issues and Practice, 34(2), 34–46.
63. McClellan, C., Donoghue, J., & Park, Y. S., (2013). Commonality and uniqueness in teaching practice observation. Retrieved from http://www.clowderconsulting.com/wp-content/uploads/2016/01/Commonality-and-Uniqueness-in-Teaching-Practice-Observation_paper.pdf
64. Mihaly, K., McCaffrey, D. F., Staiger, D. O., & Lockwood, J. R., (2013). A composite estimator of effective teaching. Seattle, WA:Bill & Melinda Gates Foundation, Measures of Effective Teaching Project.
65. Muijs, D., Kyriakides, L., van der Werf, G., Creemers, B., Timperley, H., & Earl, L., (2014). State of the art:Teacher effectiveness and professional learning. School Effectiveness and School Improvement, 25, 231–256.
66. National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA:Author.
67. National Council of Teachers of Mathematics. (1991). Professional standards for teaching mathematics. Reston, VA:Author.
68. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA:Author.
69. National Council of Teachers of Mathematics. (2014). Principles to actions:Ensuring mathematical success for all. Reston, VA:Author.
70. Odden, A., (2004). Lessons learned about standards-based teacher evaluation systems. Peabody Journal of Education, 79, 126–137.
71. Papay, J. P., (2012). Refocusing the debate:Assessing the purposes and tools of teacher evaluation. Harvard Educational Review, 82, 123–141.
72. Pianta, B., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F., (2008). Classroom effects on children’s achievement trajectories in elementary school. American Educational Research Journal, 45, 365–387.
73. Pianta, R., & Hamre, B. K., (2009). Conceptualization, measurement, and improvement of classroom processes:Standardized observation can leverage capacity. Educational Researcher, 38, 109–119.
74. Pianta, R. C., Hamre, B. K., & Mintz, S., (2010). Classroom Assessment Scoring System (CLASS) Manual:Elementary 4–6 Manual. Retrieved from http://teachstone.com/classroom-assessment-scoring-system-class/
75. Pohl, S., & Steyer, R., (2010). Modeling common traits and method effects in multitrait–multimethod analysis. Multivariate Behavioral Research, 45, 45–72.
76. Rockoff, J. E., & Speroni, C., (2010). Subjective and objective evaluations of teacher effectiveness. American Economic Review, 100(2), 261–266.
77. Rockoff, J. E., Staiger, D. O., Kane, T. J., & Taylor, E. S., (2012). Information and employee evaluation:Evidence from a randomized intervention in public schools. American Economic Review, 102, 3184–3213.
78. Rothstein, J., & Mathis, W. J., (2013). Review of two culminating reports from the MET project. National Education Policy Center, University of Colorado at Boulder. Retrieved from http://nepc.colorado.edu/thinktank/review-MET-final-2013
79. Sandilos, L. E., DiPerna, J. C., & Family Life Project Key Investigators. (2014). Measuring quality in kindergarten classrooms:Structural analysis of the Classroom Assessment Scoring System (CLASS K–3). Early Education and Development, 25, 894–914.
80. Savitsky, T. D., & McCaffrey, D. F., (2014). Bayesian hierarchical multivariate formulation with factor analysis for nested ordinal data. Psychometrika, 79, 275–302.
81. Shavelson, R., & Dempsey-Atwood, N., (1976). Generalizability of measures of teaching behavior. Review of Educational Research, 46, 553–611.
82. Sivo, S. A., Fan, X., Witta, E. L., & Willse, J. T., (2006). The search for “optimal” cutoff properties:Fit index criteria in structural equation modeling. The Journal of Experimental Education, 74, 267–288.
83. Stiggins, R. J., & Chappuis, J., (2012). An introduction to student-involved assessment for learning. Boston, MA:Pearson.
84. Tabachnick, B. G., & Fidell, L. S., (2001). Using multivariate statistics (4th ed.). New York, NY:HarperCollins.
85. Wayne, A. J., Yoon, K. S., Zhu, P., Cronen, S., & Garet, M. S., (2008). Experimenting with teacher professional development:Motives and methods. Educational Researcher, 37, 469–479.
86. Weisberg, D., Sexton, S., Mulhern, J., & Keeling, D., (2009). The widget effect:Our national failure to acknowledge and act on differences in teacher effectiveness. Retrieved from https://tntp.org/assets/documents/TheWidgetEffect_2nd_ed.pdf.
87. Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K., (2007). Reviewing the evidence on how teacher professional development affects student achievement. Washington, DC:U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.
88. Yuan, K.-H., & Bentler, P. M., (2004). On chi-square difference and z tests in mean and covariance structure analysis when the base model is misspecified. Educational and Psychological Measurement, 64, 737–757.