Missing data in value-added modeling of teacher effects

Annals of Applied Statistics

Volumn 5, Issue 2 A, 2011, Pages 773-797

  McCaffrey, Daniel F ^a   Lockwood, J R ^a  

^a RAND CORPORATION   (United States)

Author keywords

Data missing not at random; Nonignorable missing data; Pattern mixture model; Random effects; Selection models; Student achievement

Indexed keywords

EID: 84870062874     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/10-AOAS405     Document Type: Article

References (58)

1. Ballou, D., Sanders, W. and Wright, P. (2004). Controlling for student background in value-added assessment of teachers. J. Educ. Behav. Statist. 29 37-66.
2. Best, N. G., Cowles, K. and Vines, S. K. (1995). CODA: Convergence diagnosis and output analysis software for Gibbs sampling output, Version 0.3. MRC Biostatistics Unit, Cambridge.
3. Bifulco, R. and Ladd, H. (2004). The impact of charter schools on student achievement: Evidence from North Carolina. Technical report, Univ. Connecticut and Duke Univ.
4. Braun, H. (2005a). Using student progress to evaluate teachers: A primer on value-added models. Technical report, Educational Testing Service, Policy Information Center.
5. Braun, H. (2005b). Value-added modeling: What does due dilligence require? In Value Added Models in Education: Theory and Practice (R. Lissitz, ed.) 19-38. JAM Press, Maple Grove, MN.
6. Browne, W. J., Goldstein, H. and Rasbash, J. (2001). Multiple membership multiple classification (MMMC) models. Statist. Model. 1 103-124.
7. Diggle, P. and Kenward, M. G. (1994). Informative drop-out in longitudinal data analysis. Appl. Statist. 43 49-73.
8. Dunn, M., Kadane, J. B. and Garrow, J. (2003). Comparing the harm done by mobility and class absence: Missing students and missing data. J. Educ. Behav. Statist. 28 269-288.
9. Ellickson, P. L. and McGuigan, K. A. (2000). Early predictors of adolescent violence. Amer. J. Public Health 90 566-572.
10. Fitzmaurice, G. M., Laird, N. M. and Shneyer, L. (2001). An alternative parameterization of the general linear mixture model for longitudinal data with non-ignorable drop-outs. Statist. Med. 20 1009-1021.
11. Follmann, D. and Wu, M. (1995). An approximate generalized linear model with random effects for informative missing data. Biometrics 51 151-168.
12. Gelman, A. and Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences. Statist. Sci. 7 457-472.
13. Goldhaber, A. and Anthony, E. (2004). Can teacher quality be effectively assessed? Unpublished manuscript.
14. Guo, X. and Carlin, B. P. (2004). Separate and joint modeling of longitudinal and event time data using standard computer packages. Amer. Statist. 58 16-24.
15. Hanushek, E. A., Kain, J. F. and Rivkin, S. G. (2002). The impact of charter schools on academic achievement. Technical report, Hoover Institute.
16. Hanushek, E. A., Kain, J. F. and Rivkin, S. G. (2004). Disruption versus tiebout improvement: The costs and benefits of switching schools. J. Public Economics 88 1721-1746.
17. Harris, D. and Sass, T. R. (2006). Value-added models and the measurement of teacher quality. Unpublished manuscript.
18. Hedeker, D. and Gibbons, R. D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods 2 64-78.
19. Hedeker, D. and Gibbons, R. D. (2006). Longitudinal Data Analysis. Wiley, Hoboken, NJ.
20. Holman, R. and Glas, C. A. W. (2005). Modelling non-ignorable missing-data mechanisms with item response theory models. Br. J. Math. Statist. Psychol. 58 1-17.
21. Ibrahim, J. G., Chen, M.-H. and Lipsitz, S. R. (2001). Missing responses in generalised linear mixed models when the missing data mechanism is nonignorable. Biometrika 88 551-564.
22. Jacob, B. and Lefgren, L. (2006). When principals rate teachers. Technical Report No. 2, Education Next.
23. Kane, T. J., Rockoff, J. E. and Staiger, D. O. (2006). What does certification tell us about teacher effectiveness? Evidence from New York City. Unpublished manuscript.
24. Kupermintz, H. (2003). Teacher effects and teacher effectiveness: A validity investigation of the tennessee value added assessment system. Educational Evaluation and Policy Analysis 25 287-298.
25. Le, V., Stecher, B., Lockwood, J. R., Hamilton, L. S., Robyn, A., Williams, V., Ryan, G., Kerr, K., Martinez, J. F. and Klein, S. (2006). Improving Mathematics and Science Education: A Longitudinal Investigation of the Relationship Between Reform-Oriented Instruction and Student Achievement (MG-480-EDU). RAND, Santa Monica, CA.
26. Lissitz, R., ed. (2005). Value Added Models in Education: Theory and Applications. JAM Press, Maple Grove, MN.
27. Little, R. J. A. (1993). Pattern-mixture models for multivariate incomplete data. J. Amer. Statist. Assoc. 88 125-134.
28. Little, R. J. A. (1995). Modeling the drop-out mechanism in repeated measures studies. J. Amer. Statist. Assoc. 90 1112-1121.
29. Little, R. J. A. and Rubin, D. B. (1987). Statistical Analysis with Missing Data, 2nd ed. Wiley, New York.
30. Lockwood, J. R. and McCaffrey, D. F. (2007). Controlling for individual heterogeneity in longitudinal models, with applications to student achievement. Electron. J. Statist. 1 223-252.
31. Lockwood, J. R. and McCaffrey, D. F. (2009). Exploring student-teacher interactions in longitudinal achievement data. Education Finance and Policy 4 439-467.
32. Lockwood, J. R., McCaffrey, D. F., Mariano, L. T. and Setodji, C. (2007). Bayesian methods for scalable multivariate value-added assessment. J. Educ. Behav. Statist. 32 125-150.
33. Lunn, D. J., Thomas, A., Best, N. and Spiegelhalter, D. (2000). WinBUGS-a Bayesian modelling framework: Concepts, structure, and extensibility. Statist. Comput. 10 325-337.
34. McCaffrey, D. F. and Lockwood, J. R. (2010). Supplement to "Missing data in value-added modeling of teacher effects." DOI: 10.1214/10-AOAS405SUPP.
35. McCaffrey, D. F., Lockwood, J. R., Koretz, D. M. and Hamilton, L. S. (2003). Evaluating Value-Added Models for Teacher Accountability (MG-158-EDU). RAND, Santa Monica, CA.
36. McCaffrey, D. F., Lockwood, J. R., Koretz, D., Louis, T. A. and Hamilton, L. (2004). Models for value-added modeling of teacher effects. J. Educ. Behav. Statist. 29 67-101.
37. McCaffrey, D. M., Lockwood, J. R., Mariano, L. T. and Setodji, C. (2005). Challenges for value-added assessment of teacher effects. In Value Added Models in Education: Theory and Applications (R. Lissitz, ed.) 111-144. JAM Press, Maple Grove, MN.
38. Mehana, M. and Reynolds, A. J. (2004). School mobility and achievement: A meta-analysis. Children and Youth Services Review 26 93-119.
39. Moustaki, I. and Knott, M. (2000). Weighting for item non-response in attitude scales by using latent variable models with covariates. J. Roy. Statist. Soc. Ser. A 163 445-459.
40. US General Accounting Office (1994). Elementary school children: Many change schools frequently, harming their education (GAO/HEHS-94-45). Technical report, US General Accounting Office, Washington, DC. Retreived on December 30, 2008 from http://archive.gao.gov/t2pbat4/150724.pdf.
41. Korobko, O. B., Glas, C. A. W., Bosker, R. J. and Luyten, J. W. (2008). Comparing the difficulty of examination subjects with item response theory. Journal of Educational Measurement 45 139-157.
42. O'Muircheartaigh, C. and Moustaki, I. (1999). Symmetric pattern models: A latent variable approach to item non-response in attitude scales. J. Roy. Statist. Soc. Ser. A 162 177-194.
43. R Development Core Team (2007). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0.
44. Raudenbush, S. W. and Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed. Sage, Newbury Park, CA.
45. Rumberger, R. W. (2003). The causes and consequences of student mobility. J. Negro Educ. 72 6-21.
46. Sanders, W. L., Saxton, A. M. and Horn, B. P. (1997). The Tennessee value-added assessment system: A quantitative outcomes-based approach to educational assessment. In Grading Teachers, Grading Schools: Is Student Achievement a Valid Evaluational Measure? (J. Millman, ed.) 137-162. Corwin Press, Thousand Oaks, CA.
47. Schacter, J. and Thum, Y. M. (2004). Paying for high and low-quality teaching. Econ. Educ. Rev. 23 411-430.
48. Searle, S. R., Casella, G. and McCulloch, C. E. (1992). Variance Components. Wiley, New York.
49. Simpson, G. A. and Fowler, M. G. (1994). Geographic mobility and children's emotional/behavioral adjustment and school functioning. Pediatrics 93 303-309.
50. Singer, J. D. and Willett, J. B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford Univ. Press, Oxford.
51. Spiegelhalter, D. J., Best, N. G., Carlin, B. P. and van der Linde, A. (2002). Bayesian measures of model complexity and fit (with discussion). J. R. Statist. Soc. Ser. B Statist. Methodol. 64 583-639.
52. Strand, S. and Demie, F. (2006). Pupil mobility, attainment and progress in primary school. Br. Educ. Res. J. 32 551-568.
53. Ten Have, T. R., Reboussin, B. A., Miller, M. E. and Kunselman, A. (2002). Mixed effects logistic regression models for multiple longitudinal binary functional limitation responses with informative drop-out and confounding by baseline outcomes. Biometrics 58 137-144.
54. Theil, H. (1971). Principals of Econometrics. Wiley, New York.
55. Wood, D., Halfon, N., Scarlata, D., Newacheck, P. and Nessim, S. (1993). Impact of family relocation on children's growth, development, school function, and behavior. J. Amer. Med. Assoc. 270 1334-1338.
56. Wright, S. P. (2004). Advantages of a multivariate longitudinal approach to educational value-added assessment without imputation. Presented at the 2004 National Evaluation Institute, July 8-10, 2004, Colorado Springs, CO.
57. Wu, M. C. and Carroll, R. J. (1988). Estimation and comparison of changes in the presence of informative right censoring by modeling the censoring process. Biometrics 44 175-188.
58. Zimmer, R., Buddin, R., Chau, D., Daley, G., Gill, B., Guarino, C., Hamilton, L., Krop, C., McCaffrey, D., Sandler, M. and Brewer, D. (2003). Charter School Operations and Performance: Evidence from California. RAND, Santa Monica, CA.