Quality of research design moderates effects of grade retention on achievement: A meta-analytic, multilevel analysis

Educational Evaluation and Policy Analysis

Volumn 31, Issue 4, 2009, Pages 480-499

  Allen, Chiharu S ^a   Chen, Qi ^b   Willson, Victor L ^a   Hughes, Jan N ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

^b UNIVERSITY OF NORTH TEXAS   (United States)

Author keywords

Academic achievement; Design quality; Grade retention; Meta analysis

Indexed keywords

EID: 72749101736     PISSN: 01623737     EISSN: 19351062     Source Type: Journal    
DOI: 10.3102/0162373709352239     Document Type: Article

References (77)

1. References marked with an asterisk indicate studies included in the meta-analysis.
2. &z.ast;Alexander, K., Entwisle, D., & Dauber, S. (1994). On the success of failure: A reassessment of the effects of retention in the primary grades. New York: Cambridge University Press.
3. Anderson, G.E., Jimerson, S.R., & Whipple, A.D. (2005). Student ratings of stressful experiences at home and school: Loss of a parent and grade retention as superlative stressors. Journal of Applied School Psychology, 21, 1-20.
4. Bali, V.A., Anagnostopoulos, D., & Roberts, R. (2005). Toward a political explanation of grade retention. Educational Evaluation and Policy Analysis, 27, 133-155.
5. Byrnes, D., & Yamamoto, K. (1986). Views of grade repetition. Journal of Research and Development in Education, 10, 14-20.
6. Campbell, D.T., & Kenny, D.A. (1999). A primer on regression artifacts. New York: Guilford.
7. Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
8. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
9. &z.ast;Dennebaum, J.M., & Kuhlberg, J.M. (1994). Kindergarten retention and transition classrooms: Their relationship to achievement. Psychology in the Schools, 31, 5-12.
10. Educational Research Service. (1998). Information for school leaders. Prepared for the Association of California School Administrators. Arlington, VA: Author.
11. *Ferguson, P. (1991). Longitudinal outcome differences among promoted and transitional at-risk kindergarten students. Psychology in the Schools, 28, 139-146.
12. &z.ast;Ferguson, P., & Mueller Streib, M. (1996). Longitudinal outcome effects of non-at-risk and at-risk transition first-grade samples: A follow-up study and further analysis. Psychology in the Schools, 33, 38-45.
13. Florida Department of Education. (2008). Non-promotions in Florida's Public Schools, 2006-2007. Retrieved October 17, 2008, from http://www.fldoe.org/eias/eiaspubs/pdf/nonpromotions.pdf.
14. Foster, J. (1993). Retaining children in grade. Childhood Education, 70, 38-43.
15. Glass, G.V., McGaw, B., & Smith, M.L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
16. &z.ast;Gleason, K.A., Kwok, O.-M., & Hughes, J.N. (2007). The short-term effect of grade retention on peer relations and academic performance of at-risk first graders. Elementary School Journal, 107, 327-340.
17. &z.ast;Hagborg, W.J., Masella, G., Palladino, P., & Shepardson, J. (1991). A follow-up study of high school students with a history of grade retention. Psychology in the Schools, 28, 310-317.
18. Heinsman, D.T., & Shadish, W.R. (1996). Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments? Psychological Methods, 2, 154-169.
19. Holmes, C.T. (1989). Grade-level retention effects: A meta-analysis of research studies. In L. A. Shepard & M. L. Smith (Eds.), Flunking grades: Research and policies on retention (pp. 16-33). London: Falmer.
20. &z.ast;Hong, G., & Raudenbush, S.W. (2005). Effects of kindergarten retention policy on children's cognitive growth reading and mathematics. Educational Evaluation and Policy Analysis, 27, 205-224.
21. Hopkins, K.D. (1969). Regression and the matching fallacy in quasi-experimental research. Journal of Special Education, 3, 329-336.
22. Jackson, G.B. (1975). The research evidence on the effects of grade retention. Review of Educational Research, 45, 613-635.
23. &z.ast;Jacobs, B.A., & Lefgren, L. (2004). Remedial education and student achievement: A regression-discontinuity analysis. Review of Economics and Statistics, 86, 226-244.
24. Jimerson, S.R. (2001a). Meta-analysis of grade retention research: Implications for practice in the 21st century. School Psychology Review, 30, 420-437.
25. Jimerson, S.R. (2001b). A synthesis of grade retention research: Looking backward and moving forward. California School Psychologist, 6, 47-59.
26. &z.ast;Jimerson, S., Carlson, E., Rotert, M., Egeland, B., & Sroufe, L.A. (1997). A prospective, longitudinal study of the correlates and consequences of early grade retention. Journal of School Psychology, 35, 3-25.
27. Jimerson, S.R., Pletcher, S.M., Sarah, M.W., Graydon, K., Schnurr, B.L., Nickerson, A.B.,et al.(2006). Beyond grade retention and social promotion: Promoting the social and academic competence of students. Psychology in the Schools, 42, 85-97.
28. &z.ast;Johnson, E.R., Merrell, K.W., & Stover, L. (1990). The effects of early grade retention on the academic achievement of fourth-grade students. Psychology in the Schools, 27, 333-338.
29. Light, R.J., & Pillemer, D.B. (1984). Summing up: The science of reviewing research. Cambridge, MA: Harvard University Press.
30. Lipsey, M.W., & Wilson, D.B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
31. Little, R. J., Hyonggin, J., Johanns, J., & Giordani, B. (2000). A comparison of subset selection and analysis of covariance for the adjustment of confounders. Psychological Methods, 5, 459-476.
32. Lorence, J. (2006). Retention and academic achievement research revisited from a United States perspective. International Education Journal, 7, 731-777.
33. &z.ast;Lorence, J., & Dworkin, A.G. (2006). Elementary grade retention in Texas and reading achievement among racial groups: 1994-2002. Review of Policy Research, 23, 999-1033.
34. Mantzicopoulos, P. (1997). Do certain groups of children profit from early grade retention. Psychology in the Schools, 34, 115-127.
35. *Mantzicopoulos, P. (2003). Academic and school adjustment outcomes following placement in a developmental first-grade program. Journal of Educational Research, 97, 90-105.
36. &z.ast;Mantzicopoulos, P., & Morrison, D. (1992). Kindergarten retention: Academic and behavioral outcomes through the end of the second grade. American Educational Research Journal, 29, 182-198.
37. Marascuilo, L.A., & Serlin, R. (1988). Statistical methods for the social and behavioral sciences. New York: Freeman.
38. *McCombs-Thomas, A., Armistead, L., Kempton, T., Lynch, S., Forehand, R., Nousianen, S.,et al.(1992). Early retention: Are there long term beneficial effects? Psychology in the Schools, 29, 342-347.
39. &z.ast;McCoy, A.R., & Reynolds, A.J. (1999). Grade retention and school performance: An extended investigation. Journal of School Psychology, 37, 273-298.
40. &z.ast;Meisels, S.J., & Liaw, F.R. (1993). Failure in grade: Do retained students catch up? Journal of Educational Research, 87, 69-77.
41. &z.ast;Moller, S., Stearns, E., Blau, J.R., & Land, K.C. (2006). Smooth and rough roads to academic achievement: Retention and race/class disparities in high school. Social Science Research, 35, 157-180.
42. Muthén, L.K., & Muthén, B.O. (2006a). Mplus user's guide. Los Angeles: Author.
43. Muthén, L.K., & Muthén, B.O. (2006b). Mplus user's guide: Version 4.2 language addendum. Updated version retrieved October 10, 2009, from http://www.statmodel.com/download/language1.pdf.
44. Muthén, L.K., & Muthén, B.O. (2006c). Scaling of sampling weights for two-level models. Retrieved July 7, 2007, from http://www.statmodel.com/download/Scaling3.pdf.
45. National Association of School Psychologists. (2003). Position statement on student grade retention and social promotion. Bethesda, MD: Author.
46. National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Washington, DC: Government Printing Office.
47. No Child Left Behind Act of 2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002).
48. Owings, W.A., & Kaplan, L.S. (2001). Standards, retention, and social promotion. NASSP Bulletin, 85, 57-66.
49. Owings, W.A., & Magliaro, S. (1998). Grade retention: A history of failure. Educational Leadership, 56, 86-88.
50. Pagani, L., Tremblay, R.E., Vitaro, F., Boulerice, B., & McDuff, P. (2001). Effect of grade retention on academic performance and behavioral development. Development and Psychopathology, 13, 297-315.
51. Peterson, L.S., & Hughes, J.N. (2008, August). Differences between retained and promoted children in educational services received. Paper presented at the American Psychological Association annual conference, Boston.
52. &z.ast;Phelps, L., Dowdell, N., Rizzo, F.G., Ehrlich, P., & Wilczenski, F. (1992). Five to ten years after placement: The long-term efficacy of retention and pre-grade transition. Journal of Psychoeducational Assessment, 10, 116-123.
53. Picklo, D.M., & Christenson, S.L. (2005). Alternatives to retention and social promotion: The availability of instructional options. Remedial and Special Education, 26, 258-268.
54. &z.ast;Pierson, L.H., & Connell, J.P. (1992). Effect of grade retention on self-system processes, school engagement and academic performance. Journal of Educational Psychology, 84, 300-307.
55. Raudenbush, S.W., & Bryk, A.S. (2002). Linear models. Applications and data analysis methods. Thousand Oaks, CA: Sage.
56. *Reynolds, A.J. (1992). Grade retention and school adjustment: An exploratory analysis. Educational Evaluation and Policy Analysis, 14, 101-121.
57. Reynolds, A.J., & Bezruczko, N. (1993). School adjustment of children at risk through fourth grade. Merrill-Palmer Quarterly, 39, 457-480.
58. Roderick, M. (1994). Grade retention and school dropout: Investigating the association. American Educational Research Journal, 31, 729-759.
59. &z.ast;Roderick, M., & Nagaoka, J. (2005). Retention under Chicago's high-stakes testing program: Helpful, harmful, or harmless? Educational Evaluation and Policy Analysis, 27, 309-340.
60. Rosenbaum, P.R. (2002). Attributing effects to treatment in matched observational studies. Journal of the American Statistical Association, 97, 183-192.
61. Rosenbaum, P.R., & Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
62. &z.ast;Rust, J., & Wallace, K. (1993). Effects of grade level retention for four years. Journal of Instructional Psychology, 20, 162-166.
63. Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
64. Shadish, W.R., Luellen, J.K., & Clark, M.H. (2006). Propensity scores and quasi-experiments: A testimony to the practical side of Lee Sechrest. In R. R. Bootzin & P. E. McKnight (Eds.), Strengthening research methodology: Psychological measurement and evaluation (pp. 143-157). Washington, DC: American Psychological Association.
65. Shepard, L.A., Smith, M.L., & Marion, S.F. (1996). Failed evidence on grade retention [Review of the book On the success of failure: A reassessment of the effects of retention in the primary grades]. Psychology in the Schools, 33, 251-261.
66. Silberglitt, B., Appleton, J.J., Burns, M.K., & Jimerson, S.R. (2006). Examining the effects of grade retention on student reading performance: A longitudinal study. Journal of School Psychology, 44, 255-270.
67. Silberglitt, B., Jimerson, S.R., Burns, M.K., & Appleton, J.J. (2006). Does the timing of grade retention make a difference? Examining the effects of early versus late retention. School Psychology Review, 35, 134-141.
68. Sipple, J.W., Killeen, K., & Monk, D.H. (2004). Adoption and adaptation: School district responses to state imposed learning and graduation requirements. Educational Evaluation and Policy Analysis, 26, 143-168.
69. Snijders, T.A.B., & Bosker, R.J. (1993). Standard errors and sample sizes for two-level research. Journal of Educational Statistics, 18, 237-259.
70. Texas Education Agency. (2006). Texas Education Agency academic excellence indicator system. Retrieved June 8, 2006, from http://www.tea.state.tx.us/perfreport/aeis/2001/state.html.
71. Texas Education Agency. (2007). Grade-level retention in Texas public schools, 2005-06 (GE08 601 01). Retrieved October 17, 2008, from http://ritter.tea.state.tx.us/research/pdfs/retention_2005-06.pdf.
72. Tomchin, E.M., & Impara, J.C. (1992). Unraveling teachers' beliefs about grade retention. American Educational Research Journal, 29, 199-123.
73. U.S. Department of Education. (1999). Taking responsibility for ending social promotion. Retrieved January 23, 2007, from http://www.ed.gov/PDFDocs/socialprom.pdf.
74. U.S. Department of Education. (2006). Improving data quality for Title I standards, assessment, and accountability reporting. Retrieved September 23, 2007, from http://www.ed.gov/policy/elsec/guid/standardsassessment/nclbdataguidance.pdf.
75. Willson, V.L., & Hughes, J.N. (2006). Retention of Hispanic/Latino students in first grade: Child, parent, teacher, school, and peer predictors. Journal of School Psychology, 44, 31-49.
76. Wu, W., West, S.G., & Hughes, J.N. (2008). Short-term effects of grade retention on the growth rate of Woodcock-Johnson III Broad Math and Reading Scores. Journal of School Psychology, 46, 85-105.
77. Wu, W., West, S.G., & Hughes, J.N. (in press). Effect of retention in first grade on children's achievement trajectories over four years: A piecewise growth analysis using propensity score matching. Journal of Educational Psychology.