Assessing the “Rothstein Falsification Test”: Does It Really Show Teacher Value-Added Models Are Biased?

Journal of Research on Educational Effectiveness

Volumn 8, Issue 1, 2015, Pages 8-34

  Goldhaber, Dan ^a,b   Chaplin, Duncan Dunbar ^c  

^a UNIVERSITY OF WASHINGTON BOTHELL   (United States)

^b AMERICAN INSTITUTES FOR RESEARCH   (United States)

^c MATHEMATICA POLICY RESEARCH   (United States)

Author keywords

falsification tests; teacher evaluation; Value added

Indexed keywords

EID: 84921314479     PISSN: 19345747     EISSN: 19345739     Source Type: Journal    
DOI: 10.1080/19345747.2014.978059     Document Type: Article

References (40)

1. Ashenfelter, O. (1978). Estimating the effect of training programs on earnings. Review of Economics and Statistics, 60(1), 47–50.
2. Baker, E.L., Barton, P.E., Darling-Hammond, L., Haertel, E., Ladd, H.F., Linn, R.L., Ravitch, D., Rothstein, R., Shavelson, R.J., & Shepard L.A. (2010). Problems with the use of student test scores to evaluate teachers (Briefing Paper #278). Washington, DC: Economic Policy Institute.
3. Briggs, D., & Domingue, B. (2011). Due diligence and the evaluation of teachers: A review of the value-added analysis underlying the effectiveness rankings of Los Angeles Unified School District teachers by the Los Angeles Times. Boulder, CO: National Education Policy Center. Retrieved from http://nepc.colorado.edu/publication/due-diligence
4. Chetty, R., Friedman, J. N., & Rockoff, J.E. (2014). Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9): 2593–2632.
5. Corcoran, S., & Goldhaber, D. (2013). Where you stand depends on where you sit: Debates over value-added and its uses. Education Finance and Policy, 8(3), 218–434.
6. Feng, L. (2010). Hire today, gone tomorrow: New teacher classroom assignments and teacher mobility. Education Finance and Policy, 5(3), 278–316.
7. Fuller, W.A. (1987). Measurement error models. New York, NY: John Wiley & Sons.
8. Gao, N. (2011). School accountability, principal characteristics and teacher assignment. Unpublished manuscript, Florida State University.
9. Goldberger, A.J. (1991). A course in econometrics. Cambridge, MA: Harvard University Press.
10. Goldhaber, D., & Chaplin, D. (2012). Assessing the “Rothstein Falsification Test.” Does it really show teacher value-added models are biased? CEDR Working Paper 2012-1.3. University of Washington, Seattle, WA.
11. Goldhaber, D., & Hansen, M. (2010). Using performance on the job to inform teacher tenure decisions. American Economic Review, 100(2), 250–255.
12. Guarino, C., Reckase, M., Stacy, B., & Wooldridge, J. (2015). Evaluating specification tests in the context of value-added estimation. Journal of Research on Educational Effectiveness, 8, 35–59.
13. Hanushek, E.A, & Rivkin, S.G. (2010). Using value-added measures of teacher quality (Brief 9). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.
14. Harris, D., & Sass, T. (2010). Teacher training, teacher quality, and student achievement (Working Paper). Tallahassee, FL: Florida State University.
15. Harris, D., Sass, T., & Semykina, A. (2010). Value-added models and the measurement of teacher productivity (Working Paper No. 54). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.
16. Heckman, J., & Hotz, J. (1989). Choosing among alternative nonexperimental methods for estimating the impacts of social programs: The case of manpower training. Journal of the American Statistical Association, 84(408), 862–874.
17. Isenberg, E., Max, J., Gleason, P., Potamites, L., Santillano, R., Hock, H., & Hansen, M. (2013). Access to effective teaching for disadvantaged students (NCEE 2014-4001). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
18. Jacob, B.A., & Lefgren, L. (2007). What do parents value in education? An empirical investigation of parents’ revealed preferences for teachers. Quarterly Journal of Economics, 122(4), 1603–1637.
19. Kalogrides, D., Loeb, S., & Beteille, T. (2013). Systematic sorting: Teacher characteristics and class assignments. Sociology of Education, 86(2), 103–123.
20. Kane, T.J., McCaffrey, D.F., Miller, T., & Staiger, D.O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment (Measures of Effective Teaching Research Report). Seattle, WA: Bill & Melinda Gates Foundation.
21. Kane, T.J, & Staiger, D.O. (2008). Estimating teacher impacts on student achievement: An experimental evaluation (Working Paper No. 14607). Cambridge, MA: National Bureau of Economic Research.
22. Kinsler, J. (2012). Assessing Rothstein's critique of teacher value-added models. Quantitative Economics, 3(2), 333–362.
23. Koedel, C., & Betts, J.R. (2009). Does student sorting invalidate value-added models of teacher effectiveness? An extended analysis of the Rothstein critique (Working Paper 09-02). Columbia, MO: University of Missouri.
24. Kraemer, S., Worth, R., & Meyer, R.H. (2011). Classroom assignment practices in urban school districts using teacher level value-added systems (Working Paper). Madison, WI: Value-Added Research Center, University of Wisconsin–Madison.
25. Lockwood, J., & McCaffrey, D. (in press). Should nonlinear functions of test scores be used as covariates in a regression model? In R. Lissitz (Ed.), Value-added modeling and growth modeling with particular application to teacher and school effectiveness. Charlotte, NC: Information Age.
26. McCaffrey, D.F., Koretz, D., Lockwood, J.R., Louis, T.A., & Hamilton, L.S. (2004). Models for value-added modeling of teacher effects. Journal of Educational and Behavioral Statistics, 29(1), 67–101.
27. McCaffrey, D.F., Sass, T.R., Lockwood, J.R., & Mihaly, K. (2009). The intertemporal variability of teacher effect estimates. Education Finance and Policy, 4572–606.
28. Meyer, R.H. (1999). The production of mathematics skills in high school: What works? In S. Mayer & P. Peterson (Eds.), Earning and learning: How schools matter (pp. 169–204). Washington, DC: The Brookings Institute.
29. Potamites, L., Booker, K., Chaplin, D., & Isenberg, E. (2009). Measuring school and teacher effectiveness in the epic charter school consortium—Year 2 (Final Report). Washington, DC: Mathematica Policy Research.
30. Ravitch, D. (2011). The death and life of the great American school system: How testing and choice are undermining education (revised and expanded edition). New York, NY: Basic Books.
31. Rothman, R. (2010). Beyond test scores: Adding value to assessment. The School Administrator, 67(2), 20–24.
32. Rothstein, J. (2009). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Education Finance and Policy, 4(3), 537–571.
33. Rothstein, J. (2010). Teacher quality in educational production: Tracking, decay, and student achievement. Quarterly Journal of Economics 125(1), 175–214.
34. Rothstein, J. (2011). Using value added for teacher accountability: What could go wrong? (PowerPoint Slides). Retrieved from http://www.aera.net/Portals/38/docs/About_AERA/AERA%20-%20NAE%20briefing%20_%20Combined%20Slides%20%289-20%29.pdf
35. Sass, T.R., Semykina, A., & Harris, D.N. (2014). Value-added models and the measurement of teacher productivity. Economics of Education Review, 38, 9–23.
36. Schochet, P., & Chiang, H. (2013). Using value-added models? What are error rates for classifying teacher and school performance. Journal of Educational and Behavioral Statistics, 38(2), 142–171.
37. Spearman, C. (1904). The proof and measurement of association between two things. The American Journal of Psychology, 15(1), 72–101.
38. Theil, H. (1957). Specification errors and the estimation of economic relationships. Review of the International Statistical Institute, 25(1/3), 41–51.
39. Todd, P.E, & Wolpin, K.I. (2003). On the specification and estimation of the production function for cognitive achievement. Economic Journal, 113(485), F3–F33.
40. Viadero, D. (2009). “Value added” gauge of teaching probed. Education Week, 17(11), 6–11.