Introducing an observational measure of standards-based mathematics teaching practices: Evidence of validity and score reliability

Educational Studies in Mathematics

Volumn 85, Issue 1, 2014, Pages 109-128

  Walkowiak, Temple A ^a   Berry, Robert Q ^b   Meyer, J Patrick ^b   Rimm Kaufman, Sara E ^b   Ottmar, Erin R ^c  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

^c UNIVERSITY OF RICHMOND   (United States)

Author keywords

Mathematics teaching practices; Observational measure; Score reliability; Standards based; Validity

Indexed keywords

EID: 84890889343     PISSN: 00131954     EISSN: 15730816     Source Type: Journal    
DOI: 10.1007/s10649-013-9499-x     Document Type: Article

References (69)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Australian Association of Mathematics Teachers. (2006). Standards for excellence in teaching mathematics in Australian schools. Adelaide, SA: Author.
3. Ball, D. L., & Rowan, B. (2004). Introduction: Measuring instruction. The Elementary School Journal, 105(1), 3-10.
4. Bartolini-Bussi, M. G., & Mariotti, M. A. (2008). Semiotic mediation in the mathematics classroom: Artifacts and signs after a Vygotskian perspective. In L. English, M. Bartolini-Bussi, G. Jones, R. Lesh, & D. Tirosh (Eds.), Handbook of international research in mathematics education (2nd ed., pp. 746-805). Mahwah, NJ: Lawrence Erlbaum.
5. Battistich, V., Schaps, E., Watson, M., & Solomon, D. (1996). Prevention effects of the child development project: Early findings from an ongoing multisite demonstration trial. Journal of Adolescent Research, 11(1), 12-35.
6. Beguin, P., & Rabardel, P. (2000). Designing for instrument-mediated activity. Scandinavian Journal of Information Systems, 12, 173-190.
7. Boaler, J. (2002). Learning from teaching: Exploring the relationship between reform curriculum and equity. Journal for Research in Mathematics Education, 33(4), 239-258.
8. Boero, P., Bazzini, L., & Garuti, R. (2001). Metaphors in teaching and learning mathematics: A http://www. horizon-research. com/insidetheclassroom/reports/looking/appendixa. pdfhttp://www. horizon-research. com/insidetheclassroom/reports/looking/appendixa. pdf case study concerning inequalities. Proceedings of the 25th PME Conference, Utrecht University, the Netherlands, vol. 2, pp. 185-192.
9. Boero, P., Douek, N., Morselli, F., & Pedemonte, B. (2010). Argumentation and proof: A contribution to theoretical perspectives and their classroom implementation. In M. M. F. Pinto & T. F. Kawasaki (Eds.), Proceedings of the 34th Conference of the International Group for the Psychology of Mathematics Education (Vol. 1, pp. 179-204). Belo Horizonte: PME.
10. Borko, H., Stecher, B. M., & Alonzo, A. C. (2005). Artifact packages for characterizing classroom practice: A pilot study. Educational Assessment, 10(2), 73-104.
11. Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds.). (1999). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.
12. Brennan, R. L. (1992). Generalizability theory. Educational Measurement: Issues and Practice, 11(4), 27-34.
13. Cameron, C. E., Connor, C. M. D., & Morrison, F. J. (2005). Effects of variation in teacher organization on classroom functioning. Journal of School Psychology, 43(1), 61-85.
14. Carpenter, T. P., Franke, M. L., Jacobs, V. R., Fennema, E., & Empson, S. B. (1998). A longitudinal study of invention and understanding in children's multidigit addition and subtraction. Journal for Research in Mathematics Education, 29(1), 3-20.
15. Chomat-Mooney, L., Pianta, R. C., Hamre, B. K., Mashburn, A. J., Luckner, A. E., Grimm, K. J., et al. (2008). A practical guide for conducting classroom observations: A summary of issues and evidence for researchers. Unpublished report to the W. T. Grant Foundation, University of Virginia, Charlottesville.
16. Civil, M. (2002). Culture and mathematics: A community approach. Journal of Intercultural Studies, 23(2), 133-148.
17. diSessa, A., Hammer, D., Sherin, B., & Kolpakowski, T. (1991). Inventing graphing: Children's meta-representational expertise. The Journal of Mathematical Behavior, 10(2), 117-160.
18. Duval, R. (2006). A cognitive analysis of problems of comprehension in a learning of mathematics. Educational Studies in Mathematics, 61, 103-131.
19. Duval, R. (2007). Cognitive functioning and the understanding of mathematical processes of proof. In P. Boero (Ed.), Theorems in school: From history, epistemology, and cognition to classroom practice (pp. 137-161). Rotterdam, the Netherlands: Sense Publishers.
20. Embretson, S. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93(1), 179-197.
21. Emmer, E. T., & Stough, L. M. (2001). Classroom management: A critical part of educational psychology, with implications for teacher education. Educational Psychologist, 36(2), 103-112.
22. Eurydice. (2011). Mathematics education in Europe: Common challenges and national policies. Brussels, Belgium: Education, Audiovisual, and Culture Executive Agency.
23. Gordon, R., Kane, T. J., & Staiger, D. O. (2006). Identifying effective teachers using performance on the job (No. 1). Washington, DC: The Brookings Institution.
24. Hamre, B., & Pianta, R. C. (2007). Learning opportunities in preschool and early elementary classrooms. In R. C. Pianta, M. J. Cox, & K. L. Snow (Eds.), School readiness and the transition to kindergarten in the era of accountability. Baltimore, MD: Brookes Publishing.
25. Hand, V. (2012). Seeing culture and power in mathematical learning: Toward a model of equitable instruction. Educational Studies in Mathematics, 80, 233-247.
26. Hiebert, J., & Stigler, J. W. (2000). A proposal for improving classroom teaching: Lessons from the TIMSS video study. The Elementary School Journal, 101(1), 3-20.
27. Hiebert, J., Stigler, J. W., Jacobs, J. K., Givvin, K. B., Garnier, H., Smith, M., et al. (2005). Mathematics teaching in the United States today (and tomorrow): Results from the TIMSS 1999 video study. Educational Evaluation and Policy Analysis, 27(2), 111-132.
28. Hill, H. C., Sleep, L., Lewis, J. M., & Ball, D. L. (2007). Assessing teachers' mathematical knowledge: What knowledge matters and what evidence counts? In F. K. Lester (Ed.), Second handbook of research on mathematics teaching and learning (pp. 111-154). Charlotte, NC: National Council of Teachers of Mathematics, Information Age Publishing.
29. Horizon Research. (2000). Inside the classroom observation and analytic protocol. Retrieved from http://www. horizon-research. com/insidetheclassroom/reports/looking/appendixa. pdf.
30. Hufferd-Ackles, K., Fuson, K. C., & Sherin, M. G. (2004). Describing levels and components of a math-talk learning community. Journal for Research in Mathematics Education, 35(2), 81-116.
31. Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed.). Westport, CT: Praeger Publishers.
32. Knapp, M. S. (1997). Between systemic reforms and the mathematics and science classroom: The dynamics of innovation, implementation, and professional learning. Review of Educational Research, 67(2), 227-266.
33. Lampert, M. (1990). When the problem is not the question and the solution is not the answer: Mathematical knowing and teaching. American Educational Research Journal, 27(1), 29-63.
34. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174.
35. Learning Mathematics for Teaching. (2006). A coding rubric for measuring the quality of mathematics in instruction, Technical Report LMT. Ann Arbor, MI: University of Michigan, School of Education.
36. Lehrer, R., & Schauble, L. (2002). Symbolic communication in mathematics and science: Co-constituting inscription and thought. In E. Amsel & J. P. Byrnes (Eds.), Language, literacy, and cognitive development: The development and consequences of symbolic communication (pp. 167-192). Mahwah, NJ: Lawrence Erlbaum.
37. Lesh, R. A., Post, T. R., & Behr, M. J. (1987). Representations and translations among representations in mathematics learning and problem solving. In C. Janvier (Ed.), Problems of representation in the teaching and learning of mathematics (pp. 33-40). Hillsdale, NJ: Erlbaum.
38. Malara, N., & Zan, R. (2002). The problematic relationship between theory and practice. In L. D. English (Ed.), Handbook of international research in mathematics education (pp. 553-580). Mahwah, NJ: Lawrence Erlbaum.
39. Messick, S. (1993). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed.). Phoenix, AZ: Oryx Press.
40. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-741.
41. Moyer, P. S. (2001). Are we having fun yet? How teachers use manipulatives to teach mathematics. Educational Studies in Mathematics, 47(2), 175-197.
42. National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: National Council of Teachers of Mathematics.
43. National Council of Teachers of Mathematics. (2007). Mathematics teaching today: Improving practice, improving student learning. Reston, VA: National Council of Teachers of Mathematics.
44. North Central Regional Educational Laboratory. (2000). A study of the differences between higher and lower performing Indiana schools in reading and mathematics. Naperville, IL: North Central Regional Educational Laboratory.
45. Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237.
46. Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom effects on children's achievement trajectories in elementary school. American Educational Research Journal, 45(2), 365-397.
47. Pianta, R. C., La Paro, K. M., & Hamre, B. K. (2008). Classroom assessment scoring system manual, K-3. Baltimore, MD: Brookes Publishing Co.
48. Piburn, M., Sawada, D., Turley, J., Falconer, K., Benford, R., Bloom, I., et al. (2000). Reformed teaching observation protocol (RTOP): Reference manual. ACEPT Technical Report No. IN00-3). Tempe, AZ: Arizona Collaborative for Excellence in the Preparation of Teachers.
49. Porter, A. C., Blank, R. K., Smithson, J. L., & Osthoff, E. (2005). Place-based randomized trials to test the effects on instructional practices of a mathematics/science professional development program for teachers. The Annals of the American Academy of Political and Social Science, 599(1), 147-175.
50. Radford, L. (2002). The seen, the spoken, and the written: A semiotic approach to problem of objectification of mathematical knowledge. For the Learning of Mathematics, 22(2), 14-23.
51. Raphael, D., & Wahlstrom, M. (1989). The influence of instructional aids on mathematics achievement. Journal for Research in Mathematics Education, 20(2), 173-190.
52. Reid, D. A. (2002). Conjectures and refutations in grade 5 mathematics. Journal for Research in Mathematics Education, 33(1), 5-29.
53. Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458.
54. Rutter, M., & Maughan, B. (2002). School effectiveness findings 1979-2002. Journal of School Psychology, 40(6), 451-475.
55. Sanders, W. L., & Rivers, J. C. (1996). Cumulative and residual effects of teachers on future student academic achievement. Knoxville, TN: University of Tennessee Value-Added Research and Assessment Center.
56. Sarama, J., & Clements, D. H. (2004). Building blocks for early childhood mathematics. Early Childhood Research Quarterly, 19(1), 181-189.
57. Sherin, M. G. (2002). A balancing act: Developing a discourse community in a mathematics classroom. Journal of Mathematics Teacher Education, 5(3), 205-233.
58. Sowell, E. J. (1989). Effects of manipulative materials in mathematics instruction. Journal for Research in Mathematics Education, 20(5), 498-505.
59. Spillane, J. P., & Zeuli, J. S. (1999). Reform and teaching: Exploring patterns of practice in the context of national and state mathematics reforms. Educational Evaluation and Policy Analysis, 21(1), 1-27.
60. Stecher, B., Hamilton, L., Ryan, G., Le, V. N., Williams, V., Robyn, A., et al. (2002). Measuring reform-oriented instructional practices in mathematics and science. Annual Meeting of the American Educational Research Association, New Orleans, LA.
61. Stein, M. K., Grover, B. W., & Henningsen, M. (1996). Building student capacity for mathematical thinking and reasoning: An analysis of mathematical tasks used in reform classrooms. American Educational Research Journal, 33(2), 455-488.
62. Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standards-based mathematics instruction: A casebook for professional development (2nd ed.). New York: Teachers College Press.
63. Stipek, D. J., Givvin, K. B., Salmon, J. M., & MacGyvers, V. L. (2001). Teachers' beliefs and practices related to mathematics instruction. Teaching and Teacher Education, 17(2), 213-226.
64. Sutton, J., & Krueger, A. (Eds.). (2002). EDThoughts: What we know about mathematics teaching and learning. Aurora, CO: Mid-continent Research for Education and Learning.
65. Toulmin, S. (1969). The uses of argument. Cambridge, England: Cambridge University Press.
66. Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower, E. R., & Heck, D. J. (2003). Looking inside the classroom: A study of K-12 mathematics and science education in the united states. Chapel Hill, NC: Horizon Research, Inc.
67. Xie, M. (2009). Mathematics education reform in mainland China in the past sixty years: Reviews and forecasts. Journal of Mathematics Education, 2(1), 121-130.
68. Yackel, E., & Cobb, P. (1996). Sociomathematical norms, argumentation, and autonomy in mathematics. Journal for Research in Mathematics Education, 27(4), 458-477.
69. Yackel, E., Cobb, P., & Wood, T. (1991). Small-group interactions as a source of learning opportunities in second-grade mathematics. Journal for Research in Mathematics Education, 22(5), 390-408.