Relationships between observations of elementary mathematics instruction and student achievement: Exploring variability across districts

American Journal of Education

Volumn 123, Issue 4, 2017, Pages 615-646

  Lynch, Kathleen ^a   Chin, Mark ^a   Blazar, David ^b  

^a HARVARD GRADUATE SCHOOL OF EDUCATION   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85025130159     PISSN: 01956744     EISSN: None     Source Type: Journal    
DOI: 10.1086/692662     Document Type: Article

References (81)

1. AERA, APA, and NCME (American Educational Research Association, American Psychological Association, and National Council on Measurement in Education). 1999. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association.
2. Bell, Courtney, Drew Gitomer, Daniel McCaffrey, Bridget Hamre, Robert Pianta, and Yi Qi. 2012. “An Argument Approach to Observation Protocol Validity.” Educational Assessment 17 (2–3): 62–87.
3. Blazar, David. 2015. “Effective Teaching in Elementary Mathematics: Identifying Classroom Practices That Support Student Achievement.” Economics of Education Review 48:16–29.
4. Blazar, David, David Braslow, Charalambos Y. Charalambous, and Heather C. Hill. Forthcoming. “Attending to General and Mathematics-Specific Dimensions of Teaching: Exploring Factors across Two Observation Instruments.” Educational Assessment.
5. Blazar, David, Erica Litke, and Johanna Barmore. 2016. “What Does It Mean to Be Ranked a ‘High’ or ‘Low’ Value-Added Teacher? Observing Differences in Instructional Quality across Districts.” American Educational Research Journal 53 (2): 324–59.
6. Boaler, Jo. 1998. “Open and Closed Mathematics: Student Experiences and Under-standings.” Journal for Research in Mathematics Education 29 (1): 41–62. doi:10.2307/749717.
7. Booher-Jennings, Jennifer. 2005. “Below the Bubble: ‘Educational Triage’ and the Texas Accountability System.” American Educational Research Journal 42 (2): 231–68.
8. Borko, Hilda, Margaret Eisenhart, Catherine A. Brown, Robert G. Underhill, Doug Jones, and Patricia Agard. 1992. “Learning to Teach Hard Mathematics: Do Novice Teachers and Their Instructors Give Up Too Easily?” Journal for Research in Mathematics Education 23 (3): 194–222.
9. Boston, Melissa. 2012. “Assessing Instructional Quality in Mathematics.” Elementary School Journal 113 (1): 76–104.
10. Bridgeman, Brent. 1992. “A Comparison of Quantitative Questions in Open-Ended and Multiple-Choice Formats.” Journal of Educational Measurement 29 (3): 253–71.
11. Carpenter, Thomas P., Elizabeth Fennema, Penelope Peterson, Chi-Pang Chiang, and Megan Loef. 1989. “Using Knowledge of Children’s Mathematics Thinking in Classroom Teaching: An Experimental Study.” American Educational Research Journal 26 (4): 499–531.
12. Charalambous, Charalambos Y., Andreas Komitis, Maria Papacharalambous, and Afroditi Stefanou. 2014. “Using Generic and Content-Specific Teaching Practices in Teacher Evaluation: An Exploratory Study of Teachers’ Perceptions.” Teaching and Teacher Education 41:22–33.
13. Coburn, Cynthia E. 2005. “The Role of Nonsystem Actors in the Relationship between Policy and Practice: The Case of Reading Instruction in California.” Educational Evaluation and Policy Analysis 27:23–52.
14. Coggshall, Jane G., Claudette Rasmussen, Amy Colton, Jessica Milton, and Catherine Jacques. 2012. Generating Teacher Effectiveness: The Role of Job-Embedded Professional Learning in Teacher Evaluation. Washington, DC: National Comprehensive Center for Teacher Quality.
15. Cohen, David K. 1990. “A Revolution in One Classroom: The Case of Mrs. Oublier.” Educational Evaluation and Policy Analysis 12:311–30.
16. Cohen, David K. 2011. Teaching and Its Predicaments. Cambridge, MA: Harvard University Press.
17. Cohen, David K., and Heather C. Hill. 2000. “Instructional Policy and Classroom Performance: The Mathematics Reform in California.” Teachers College Record 102 (2): 294–343.
18. Cohen, David K., Stephen W. Raudenbush, and Deborah Loewenberg Ball. 2003. “Resources, Instruction, and Research.” Educational Evaluation and Policy Analysis 25 (2): 119–42.
19. Daley, Glenn, and Lydia Kim. 2010. A Teacher Evaluation System That Works. Santa Monica, CA: National Institute for Excellence in Teaching.
20. Danielson, Charlotte. 2011. Enhancing Professional Practice: A Framework for Teaching. Alexandria, VA: Association for Supervision and Curriculum Development.
21. Doyle, Walter. 1988. “Work in Mathematics Classes: The Context of Students’ Thinking during Instruction.” Educational Psychologist 23 (2): 167–80.
22. Fennema, Elizabeth, and Megan Loef Franke. 1992. “Teachers’ Knowledge and Its Impact.” In Handbook of Research on Mathematics Teaching and Learning, ed. Douglas A. Grouws. New York: Macmillan.
23. Firestone, William A. 2014. “Teacher Evaluation Policy and Conflicting Theories of Motivation.” Educational Researcher 42 (2): 100–7.
24. Gamse, Beth C., Robin T. Jacob, Megan Horst, Beth Boulay, and Faith Unlu. 2008. Reading First Impact Study. Washington, DC: US Department of Education, Institute of Education Sciences.
25. Goldhaber, Daniel, and Michael Hansen. 2012. “Is It Just a Bad Class? Assessing the Long-Term Stability of Estimated Teacher Performance.” Economica 80 (319): 589–612.
26. Good, Thomas L., and Jere E. Brophy. 2007. Looking in Classrooms. New York: HarperCollins.
27. Grossman, Pam, Julie Cohen, Matthew Ronfeldt, and Lindsay Brown. 2014. “The Test Matters: The Relationship between Classroom Observation Scores and Teacher Value Added on Multiple Types of Assessment.” Educational Researcher 43 (6): 293– 303.
28. Grossman, Tabitha, Ryan Reyna, and Stephanie Shipton. 2011. Realizing the Potential: How Governors Can Lead Effective Implementation of the Common Core State Standards. Washington, DC: National Governors Association Center for Best Practices.
29. Harris, Douglas N., William K. Ingle, and Stacey A. Rutledge. 2014. “How Teacher Evaluation Methods Matter for Accountability: A Comparative Analysis of Teacher Effectiveness Ratings by Principals and Teacher Value-Added Measures.” American Educational Research Journal 51 (1): 73–112.
30. Henningsen, Marjorie, and Mary Kay Stein. 1997. “Mathematical Tasks and Student Cognition: Classroom-Based Factors That Support and Inhibit High-Level Mathematical Thinking and Reasoning.” Journal for Research in Mathematics Education 28 (5): 524–49.
31. Hickman, Judy J., Jianbin Fu, and Heather Hill. 2012. Technical Report: Creation and Dissemination of Upper-Elementary Mathematics Assessment Modules. Princeton, NJ: Educational Testing Service.
32. Hiebert, James, and Douglas A. Grouws. 2007. “The Effects of Classroom Mathematics Teaching on Students’ Learning.” In Second Handbook of Research on Mathematics Teaching and Learning, ed. F. K. Lester Jr. Greenwich, CT: Information Age.
33. Hill, Heather C., David Blazar, and Kathleen Lynch. 2015. “Resources for Teaching: Examining Personal and Institutional Predictors of High-Quality Instruction.” AERA Open 1 (4): 1–23.
34. Hill, Heather C., Merrie L. Blunk, Charalambos Y. Charalambous, Jennifer M. Lewis, Geoffrey C. Phelps, Laurie Sleep, and Deborah Loewenberg Ball. 2008. “Mathe-matical Knowledge for Teaching and the Mathematical Quality of Instruction: An Exploratory Study.” Cognition and Instruction 26 (4): 430–511.
35. Hill, Heather C., Charalambos Y. Charalambous, David Blazar, Daniel McGinn, Mary Beisiegel, Andrea Humez, Matthew Kraft, Erica Litke, and Kathleen Lynch. 2012. “Validating Arguments for Observational Instruments: Attending to Multiple Sources of Variation.” Educational Assessment 17 (2–3): 88–106.
36. Hill, Heather C., Charalambos Y. Charalambous, and Matthew A. Kraft. 2012. “When Rater Reliability Is Not Enough: Teacher Observation Systems and a Case for the Generalizability Study.” Educational Researcher 41 (2): 56–64.
37. Hill, Heather C., Laura Kapitula, and Kristin Umland. 2011. “A Validity Argument Approach to Evaluating Teacher Value-Added Scores.” American Educational Research Journal 48 (3): 794–831.
38. Hill, Heather C., Brian Rowan, and Deborah Loewenberg Ball. 2005. “Effects of Teachers’ Mathematical Knowledge for Teaching on Student Achievement.” American Educational Research Association 42 (2): 371–406.
39. Ho, Andrew D., and Thomas J. Kane. 2013. The Reliability of Classroom Observations by School Personnel. Seattle: Gates Foundation.
40. Kane, Thomas J., and Steven Cantrell. 2010. Learning about Teaching: Initial Findings from the Measures of Effective Teaching Project. Measures of Effective Teaching Project Research Paper 9, Gates Foundation, Seattle.
41. Kane, Thomas J., and Douglas O. Staiger. 2012. Gathering Feedback for Teaching: Combining High-Quality Observations, Student Surveys, and Achievement Gains. Research Paper, Measures of Effective Teaching Project, Gates Foundation, Seattle.
42. Kane, Thomas J., Eric S. Taylor, John H. Tyler, and Amy L. Wooten. 2011. “Identi-fying Effective Classroom Practices Using Student Achievement Data.” Journal of Human Resources 46 (3): 587–613.
43. Koedel, Cory, and Julian Betts. 2011. “Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique.” Education 6 (1): 18–42.
44. Konstantopoulos, Spyros. 2014. “Teacher Effects, Value-Added Models, and Account-ability.” Teachers College Record 116 (1): 1–21.
45. Lloyd, Gwendolyn M., and Melvin Wilson. 1998. “Supporting Innovation: The Impact of a Teacher’s Conceptions of Functions on His Implementation of a Reform Cur-riculum.” Journal for Research in Mathematics Education 29 (3): 248–74.
46. Ma, Liping. 1999. Knowing and Teaching Elementary Mathematics. Mahwah, NJ: Erlbaum.
47. McCaffrey, Daniel F., Tim R. Sass, J. R. Lockwood, and Kata Mihaly. 2009. “The Intertemporal Variability of Teacher Effect Estimates.” Education Finance and Policy 4 (4): 572–606.
48. Milanowski, Anthony. 2004. “The Relationship between Teacher Performance Evaluation Scores and Student Achievement: Evidence from Cincinnati.” Peabody Journal of Education 79 (4): 33–53.
49. Milanowski, Anthony. 2011. Validity Research on Teacher Evaluation Systems Based on the Framework for Teaching. Madison, WI: Consortium for Policy Research in Education.
50. National Governors Association Center for Best Practices, Council of Chief State School Officers. 2010. Common Core State Standards for Mathematics. Washington, DC: National Governors Association Center for Best Practices, Council of Chief State School Officers.
51. National Research Council. 2001. Adding It Up: Helping Children Learn Mathematics. Washington, DC: National Academy Press.
52. NCTM (National Council of Teachers of Mathematics). 1995. Assessment Standards for School Mathematics. Reston, VA: NCTM.
53. NCTM (National Council of Teachers of Mathematics). 2000. Principles and Standards for School Mathematics. Reston, VA: NCTM.
54. O’Connor, Mary Catherine. 1998. “Language Socialization in the Mathematics Classroom: Discourse Practices and Mathematical Thinking.” In Talking Mathematics in School: Studies of Teaching and Learning, ed. Magdalene Lampert and Merrie L. Blunk. New York: Cambridge University Press.
55. Papay, John P. 2011. “Different Tests, Different Answers: The Stability of Teacher Value-Added Estimates across Outcome Measures.” American Educational Research Journal 48 (1): 163–93.
56. Pianta, Robert C., Bridget K. Hamre, and Susan Mintz. 2010. Classroom Assessment Scoring System (CLASS) Manual: Upper Elementary. Charlottesville, VA: Teachstone.
57. Podgursky, Michael J., and Matthew G. Springer. 2007. “Teacher Performance Pay: A Review.” Journal of Policy Analysis and Management 26 (4): 909.
58. Polikoff, Morgan S. 2014. “Does the Test Matter? Evaluating Teachers When Tests Differ in Their Sensitivity to Instruction.” In Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project, ed. Thomas J. Kane, Kerri A. Kerr, and Robert C. Pianta. San Francisco: Jossey-Bass.
59. Raudenbush, Stephen W., and Anthony S. Bryk. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. Thousand Oaks, CA: Sage.
60. Resnick, Lauren B., Robert Rothman, Jean B. Slattery, and Jennifer L. Vranek. 2004. “Benchmarking and Alignment of Standards and Testing.” Educational Assessment 9 (1–2): 1–27.
61. Richland, Lindsey E. 2015. “Linking Gestures: Cross-Cultural Variation during Instructional Analogies.” Cognition and Instruction 33 (4): 295–321.
62. Sawada, Daiyo, Michael D. Piburn, Eugene Judson, Jeff Turley, Kathleen Falconer, Russell Benford, and Irene Bloom. 2002. “Measuring Reform Practices in Science and Mathematics Classrooms: The Reformed Teaching Observation Protocol.” School Science and Mathematics 102 (6): 245–53.
63. Schacter, John, and Yeow Meng Thum. 2004. “Paying for High- and Low-Quality Teaching.” Economics of Education Review 23 (4): 411–30.
64. Schochet, Peter Z., and Hanley S. Chiang. 2013. “What Are Error Rates for Classifying Teacher and School Performance Using Value-Added Models?” Journal of Educational and Behavioral Statistics 38 (2): 142–71.
65. Schoenfeld, Alan H. 2004. “The Math Wars.” Educational Policy 18 (1): 253–86.
66. Seidel, Tina, and Richard J. Shavelson. 2007. “Teaching Effectiveness Research in the Past Decade: The Role of Theory and Research Design in Disentangling Meta-analysis Results.” Review of Educational Research 77 (4): 454–99.
67. Spearman, Charles. 1904. “The Proof and Measurement of Association between Two Things.” American Journal of Psychology 15:72–101.
68. Star, Jon R. 2005. “Reconceptualizing Procedural Knowledge.” Journal for Research in Mathematics Education 36 (5): 404–11.
69. Star, Jon R. 2007. “Foregrounding Procedural Knowledge.” Journal for Research in Mathematics Education 38 (2): 132–35.
70. Star, Jon R., Courtney Pollack, Kelley Durkin, Bethany Rittle-Johnson, Kathleen Lynch, Kristie Newton, and Claire Gogolen. 2015. “Learning from Comparison in Algebra.” Contemporary Educational Psychology 40:41–54. doi:10.1016/j.cedpsych.2014.05.005.
71. Star, Jon R., and Bethany Rittle-Johnson. 2009. “It Pays to Compare: An Experimental Study on Computational Estimation.” Journal of Experimental Child Psychology 102 (4): 408–26.
72. Stein, Mary Kay, Juliet A. Baxter, and Gaea Leinhardt. 1990. “Subject-Matter Knowledge and Elementary Instruction: A Case from Functions and Graphing.” American Educational Research Journal 27 (4): 639–63.
73. Stein, Mary Kay, Randi A. Engle, Margaret S. Smith, and Elizabeth K. Hughes. 2008. “Orchestrating Productive Mathematical Discussions: Five Practices for Helping Teachers Move Beyond Show and Tell.” Mathematical Thinking and Learning 10 (4): 313–40.
74. Stein, Mary Kay, Barbara W. Grover, and Marjorie Henningsen. 1996. “Building Student Capacity for Mathematical Thinking and Reasoning: An Analysis of Mathematical Tasks Used in Reform Classrooms.” American Educational Research Journal 33 (2): 455–88.
75. Stein, Mary Kay, and Suzanne Lane. 1996. “Instructional Tasks and the Development of Student Capacity to Think and Reason: An Analysis of the Relationship between Teaching and Learning in a Reform Mathematics Project.” Educational Research and Evaluation 2:50–80.
76. Stevens, Floraline I. 1993. “Applying an Opportunity-to-Learn Conceptual Framework to the Investigation of the Effects of Teaching Practices via Secondary Analyses of Multiple-Case-Study Summary Data.” Journal of Negro Education 62 (3): 232–48.
77. Stigler, James W., and James Hiebert. 1999. The Teaching Gap: Best Ideas from the World’s Teachers for Improving Education in the Classroom. New York: Free Press.
78. TERC. 2008. Investigations in Number, Data, and Space. Salt Lake City: Pearson.
79. UCSMP (University of Chicago School Mathematics Project). 2007. Everyday Mathematics. New York: McGraw-Hill.
80. Webb, Norman L. 1999. Alignment of Science and Mathematics Standards and Assessments in Four States. Research Monograph No. 18, National Institute for Science Education, Madison, WI.
81. Yackel, Erna, and Paul Cobb. 1996. “Sociomathematical Norms, Argumentation, and Autonomy in Mathematics.” Journal for Research in Mathematics Education 27 (4): 458–77.