Does the Model Matter? Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments

Statistics and Public Policy

Volumn 1, Issue 1, 2014, Pages 28-39

  Goldhaber, Dan ^a   Walch, Joe ^a   Gabele, Brian ^b  

^a UNIVERSITY OF WASHINGTON BOTHELL   (United States)

^b Seattle Public Schools   (United States)

Author keywords

Teacher performance; Teacher quality; Value added models

Indexed keywords

EID: 84910624764     PISSN: None     EISSN: 2330443X     Source Type: Journal    
DOI: 10.1080/2330443X.2013.856169     Document Type: Article

References (71)

1. Aaronson, D., Barrow, L., and Sander, W., (2007), “Teachers and Student Achievement in the Chicago Public High Schools,” Journal of Labor Economics, 25, 95–135. Available at http://www.journals. uchicago.edu/doi/abs/10.1086/508733.
2. Ashenfelter, O., and Krueger, A., (1994), “Estimates of the Economic Return to Schooling From a New Sample of Twins,” The American Economic Review, 84, 1157–1173.
3. Atteberry, A., (2011), Defining School Value-Added: Do Schools that Appear Strong on One Measure Appear Strong on Another? Evanston, IL. Available at http://www.eric.ed.gov/ERICWebPortal/search/detailmini. jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED518222&ERIC ExtSearch_SearchType_0=no&accno=ED518222.
4. Baker, B.D., (2012), “The Toxic Trifecta, Bad Measurement & Evolving Teacher Evaluation Policies.”. Available at http://nepc.colorado.edu/ blog/toxic-trifecta-bad-measurement-evolving-teacher-evaluation-policies.
5. Ballou, D., Mokher, C.G., and Cavalluzzo, L., (2012), “Using Value Added Assessment for Personnel Decisions: How Omitted Variables and Model Specification Influence Teachers’ Outcomes,”. Available at http://www.aefpweb.org/sites/default/files/webform/AEFP-Using VAM for personnel decisions_02-29-12.pdf.
6. Ballou, D., Sanders, W.L., and Wright, P.S., (2004), “Controlling for Student Background in Value-Added Assessment of Teachers,” Journal of Education and Behavioral Statistics, 29, 37–65. Available at http://jeb.sagepub.com/cgi/content/abstract/29/1/37.
7. Barlevy, G., and Neal, D., (2012), “Pay for Percentile,” American Economic Review, 102, 1805–1831.
8. Betebenner, D., (2007), Estimation of Student Growth Percentiles for the Colorado Student Assessment Program, Dover, NH: National Center for the Improvement of Educational Assessment. Available at http://www.cde.state. co.us/ dedocs/Research/PDF/technicalsgppaper_betebenner.pdf.
9. ——— (2009), “Norm- and Criterion-Referenced Student Growth,” Educational Measurement: Issues and Practice, 28, 42–51. Available at http://doi.wiley.com/10.1111/j.1745-3992.2009.00161.x.
10. Boyd, D.J., Grossman, P., Lankford, H., Loeb, S., and Wyckoff, J., (2008), Measuring Effect Sizes: The Effect of Measurement Error. National Conference on Value-Added Modeling, Madison, WI: University of Wisconsin.
11. Boyd, D.J., Lankford, H., Loeb, S., and Wyckoff, J.H., (2010). Teacher Layoffs: An Empirical Illustration of Seniority vs. Measures of Effectiveness, Washington, DC: National Center for Analysis of Longitudinal Data in Education Research. Available at http://www. caldercenter.org/UploadedPDF/1001421-teacher-layoffs.pdf.
12. Briggs, D., and Domingue, B., (2011), Due Diligence and the Evaluation Of Teachers A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by The Los Angeles Times, Boulder, CO: National Education Policy Center. Available at http://greatlakescenter.org/docs/Policy_Briefs/Briggs_LATimes.pdf.
13. Buddin, R., (2010), How Effective are Los Angeles Elementary Teachers and Schools?. Available at http://www.latimes.com/media/acrobat/2010-08/55538493.pdf.
14. Burke, M.A., and Sass, T.R., (2008), “Classroom Peer Effects and Student Achievement,” CALDER working paper #18. Available at http://www.urban.org/UploadedPDF/1001190_peer_effects.pdf.
15. Caldas, S.J., and Bankston, C. (1997), “Effect of School Population Socioeconomic Status on Individual Academic Achievement,” The Journal of Educational Research, 90, 269–277. Available at http://apps.webofknowledge. com.offcampus.lib.washington.edu/full_record.do?product=WOS&search_ mode=GeneralSearch&qid=17&SID=2FC8FA7Hp8kFmOCeOOF&page= 1&doc=2.
16. Castellano, K., (2011). Unpacking Student Growth Percentiles: Statistical Properties of Regression-Based Approaches With Implications for Student and School Classifications, Doctoral Dissertation, University of Iowa. Available at http:/ir.uiowa.edu/etd/931.
17. Chetty, R., Friedman, J.N., and Rockoff, J.E., (2011), “The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood,”. NBER Working Paper no. #17699.
18. Clotfelter, C., Ladd, H., and Vigdor, J., (2006), “Teacher–Student Matching and the Assessment of Teacher Effectiveness,” The Journal of Human Resources, 41, 778–820.
19. Cornelissen, T., (2008), “The Stata Command Felsdvreg to Fit a Linear Model With Two High-Dimensional Fixed Effects,” Stata Journal, 8, 170–189.
20. Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., and Rothstein, J., (2012), “Evaluating Teacher Evaluation: Popular Modes of Evaluating Teachers Are Fraught With Inaccuracies and Inconsistencies, But the Field has Identified Better Approaches,” Phi Delta Kappan, 93, 8–15. Available at http://uwashington.worldcat.org.offcampus.lib.washington.edu/title/evalua ting-teacher-evaluation/oclc/779377670&referer=brief_results.
21. Ehlert, M., Koedel, C., Parsons, E., and Podgursky, M., (2012), “Selecting Growth Measures for School and Teacher Evaluations: Should Proportionality Matter?”. Working Paper no. #1210, University of Missouri.
22. Gabriel, T., (2010, September 2), “A Celebratory Road Trip for Education Secretary,” New York Times, p. A24. Available at http://www.nytimes.com/2010/09/02/education/02duncan.html.
23. Glazerman, S., Loeb, S., Goldhaber, D., Staiger, D., Raudenbush, S., and Whitehurst, G., (2010), Evaluating Teachers: The Important Role of Value-Added. Available at http://www.brookings. edu/∼/media/Files/rc/reports/2010/1117_evaluating_teachers/1117_evalua ting_teachers.pdf.
24. Goldhaber, D., (2007), “Everyone's Doing It, But What Does Teacher Testing Tell Us About Teacher Effectiveness?” Journal of Human Resources, 42, 765–794.
25. ——— (2010), When the Stakes Are High, Can We Rely on Value-Added? Exploring the Use of Value-Added Models to Inform Teacher Workforce Decisions, Washington, DC: Center for American Progress. Available at http://www.americanprogress.org/issues/2010/12/pdf/vam.pdf.
26. Goldhaber, D., and Anthony, E., (2007), “Can Teacher Quality be Effectively Assessed? National Board Certification as a Signal of Effective Teaching,” Review of Economics and Statistics, 89, 134–150.
27. Goldhaber, D., Brewer, D., and Anderson, D., (1999), “A Three-Way Error Components Analysis of Educational Productivity,” Education Economics, 7, 199–208.
28. Goldhaber, D., and Chaplin, D., (2012), “Assessing the “Rothstein Falsification Test.” Does it Really Show Teacher Value-added Models are Biased?”. Mathematica Working Paper. Available at http://www. mathematica-mpr.com/publications/pdfs/education/rothstein_wp.pdf.
29. Goldhaber, D., Cowan, J., and Walch, J., (2012), “Is a Good Elementary Teacher Always Good? Assessing Teacher Performance Estimates Across Contexts and Comparison Groups,”. CEDR Working Paper no. #2012-7.
30. Goldhaber, D., Goldschmidt, P., Sylling, P., and Tseng, F., (2011), “Assessing Value-Added Model Estimates of Teacher Contributions to Student Learning at the High School Level,”. CEDR Working Paper no. #2011-4.
31. Goldhaber, D., and Hansen, M., (2010), “Using Performance on the Job to Inform Teacher Tenure Decisions,” American Economic Review, 100, 250–255.
32. ——— (2013), “Is it Just a Bad Class? Assessing the Long-Term Stability of Estimated Teacher Performance,” Economica, 80, 589–612.
33. Goldhaber, D., and Theobald, R., (2011), “Managing the Teacher Workforce in Austere Times: The Implications of Teacher Layoffs,”. CEDR Working Paper no. #2011-1.2. Available at http://www.cedr.us/papers/working/CEDR WP 2011-1.2 Teacher Layoffs (6-15-2011).pdf.
34. Goldhaber, D., and Walch, J., (2012), “Strategic Pay Reform: A Student Outcomes-Based Evaluation of Denver's ProComp Teacher Pay Initiative,” Economics of Education Review, 31, 1067–1083.
35. Goldhaber, D., Walch, J., and Gabele, B., (2012), “Does the Model Matter: Exploring the Relationship Between Different Student Achievement-Based Teacher Assessments,”. CEDR Working Paper no. #2012-6. Available at http://www.cedr.us/papers/working/CEDR%20WP%202012-6_Does%20the%20Model%20Matter.pdf.
36. Goldschmidt, P., Choi, K., and Beaudoin, J.P., (2012), Growth Model Comparrison Study: A Summary of Results, Washington, DC. Available at http://www. ccsso.org/Documents/2012/Summary_of_Growth_Model_Comparison_ Study_2012.pdf.
37. Gordon, R.J., Kane, T.J., and Staiger, D.O., (2006), Identifying Effective Teachers Using Performance on the Job. Hamilton Project White Paper, Washington, DC: Brookings Institution. Available at http://www.brook.edu/views/papers/200604hamilton_1.htm.
38. Hanushek, E., (1979), “Conceptual and Empirical Issues in the Estimation of Education Production Functions,” Journal of Human Resources, 14, 351–388.
39. ——— (1986), “The Economics of Schooling—Production and Efficiency in Public-Schools,” Journal of Economic Literature, 24, 1141–1177.
40. Hanushek, E.A., and Rivkin, S.G., (2010), “Generalizations About Using Value-added Measures of Teacher Quality,” American Economic Review, 100, 267–271.
41. Harris, D., and Sass, T.R., (2006), “Value-Added Models and the Measurement of Teacher Quality,”. unpublished manuscript.
42. Harris, D., Sass, T., and Semykina, A., (2010), Value-Added Models and the Measurement of Teacher Productivity, Washington, DC, unpublished manuscript.
43. Jackson, C., and Bruegmann, E., (2009), “Teaching Students and Teaching Each Other: The Importance of Peer Learning for Teachers,” American Economic Journal: Applied Economics, 1, 85–108.
44. Jacob, B., and Lefgren, L., (2008), “Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education,” Journal of Labor Economics, 26, 101–136.
45. Kane, T., and Staiger, D., (2008), “Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation,”. NBER Working Paper no. #14607. Available at http://www.nber.org/papers/w14607.pdf?new_window=1.
46. Koedel, C., and Betts, J., (2007), Re-Examining the Role of Teacher Quality in the Educational Production Function, San Diego, CA: University of Missouri. Available at http://economics.missouri. edu/working-papers/2007/WP0708_koedel.pdf.
47. ——— (2011), “Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique,” Education Finance and Policy, 6, 18–42.
48. Koenker, R., (2005), Quantile Regression (Econometric Society Monographs) (p. 366), Cambridge: Cambridge University Press. Available at http://www.amazon.com/Quantile-Regression-Econometric-Society-Monog raphs/dp/0521608279.
49. LA Times (2010), “Study Critical of Teacher Layoffs (2010, December 24),” Los Angeles Times. Available at http://www.latimes. com/news/local/teachers-investigation/la-me-teachers-study-critical-ap,0, 6451518.story.
50. Lavy, V., Silva, O., and Weinhardt, F., (2012), “The Good, the Bad, and the Average: Evidence on Ability Peer Effects in Schools,” Journal of Labor Economics, 30, 367–414.
51. Lockwood, J.R., and McCaffrey, D.F., (2012), “Reducing Bias in Observational Analyses of Education Data by Accounting for Test Measurement Error,”. unpublished manuscript, RAND Corporation.
52. Lockwood, J.R., McCaffrey, D.F., Hamilton, L., Stecher, B., Le, V.-N., and Martinez, J.F., (2007), “The Sensitivity of Value-Added Teacher Effect Estimates to Different Mathematics Achievement Measures,” Journal of Education Measurement, 44, 47–67.
53. McCaffrey, D., Koretz, D., Lockwood, J.R., Louis, T.A., and Hamilton, L.S., (2004), “Models for Value-Added Modeling of Teacher Effects,” Journal of Educational and Behavioral Statistics, 29, 67–101.
54. McCaffrey, D., Sass, T.R., Lockwood, J.R., and Mihaly, K., (2009), “The Intertemporal Variability of Teacher Effect Estimates,” Education Finance and Policy, 4, 572–606.
55. MET (2010), Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project, Bill & Melinda Gates Foundation. Available at http://www.metproject.org/downloads/Preliminary_Findings-Research_ Paper.pdf.
56. Nichols, A., (2008), “FESE: Stata Module to Calculate Standard Errors for Fixed Effects,” Boston College Department of Economics. Available at http://econpapers.repec.org/software/bocbocode/s456914.htm.
57. Papay, J., (2010), “Different Tests, Different Answers: The Stability of Teacher Value-Added Estimates Across Outcome Measures,” American Educational Research Journal, 48, 163–193.
58. Podgursky, M., and Springer, M., (2007), “Teacher Performance Pay: A Review,” Journal of Policy Analysis and Management, 26, 909–949.
59. R Development Core Team (2008), R: A Language and Environment for Statistical Computing, Vienna, Austria: R Foundation for Statistical Computing. Available at http://www.r-project.org.
60. Racine, J., (2011), A Primer on Regression Splines. Available at http://cran.r-project.org/web/packages/crs/vignettes/spline_primer.pdf.
61. Rivkin, S., Hanushek, E., and Kain, J., (2005), “Teachers, Schools, and Academic Achievement,” Econometrica, 73, 417–458.
62. Rockoff, J., and Speroni, C., (2011), “Subjective and Objective Evaluations of Teacher Effectiveness: Evidence From New York City,” Labour Economics, 18, 687–696.
63. Rothstein, J., (2010), “Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement,” Quarterly Journal of Economics, 125, 175–214.
64. Schochet, P., and Chiang, H., (2010), Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains. NCEE 2010-4004. National Center for Education Evaluation and Regional Assistance. Available at http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED511026&ERICExtSearch_SearchType_0=no&accno=ED511026.
65. Toch, T., and Rothman, R., (2008). Rush to Judgment: Teacher Evaluation in Public Education, Washington, DC: Education Sector. Available at http://www.educationsector.org/usr_doc/RushToJudgment_ES_Jan08.pdf.
66. Todd, P., and Wolpin, K., (2003), “On the Specification and Estimation of the Production Function for Cognitive Achievement,” Economic Journal, 113(485), F3–F33.
67. Tucker, P., (1997), “Lake Wobegon: Where All the Teachers are Competent (Or, Have We Come to Terms With the Problem of Incompetent Teachers?),” Journal of Personnel Evaluation in Education, 11, 103–26.
68. Wei, Y., and He, X., (2006), “Conditional Growth Charts,” The Annals of Statistics, 34, 2069–2097.
69. Weisburg, D., Sexton, S., Mulhern, J., and Keeling, D., (2009), The Widget Effect: Our National Failure to Acknowledge and Act of Differences in Teacher Effectiveness, New York: The New Teacher Project.
70. Wright, P., (2010), An Investigation of Two Nonparametric Regression Models for Value-Added Assessment in Education, SAS. Available at http://www.sas.com/resources/whitepaper/wp_16975.pdf.
71. Wright, P., White, J., Sanders, W., and Rivers, J., (2010), “SAS EVAAS Statistical Models,”. SAS White Paper. Available at http://www. sas.com/resources/asset/SAS-EVAAS-Statistical-Models.pdf.