Is test-driven external accountability effective? Synthesizing the evidence from cross-state causal-comparative and correlational studies

Review of Educational Research

Volumn 78, Issue 3, 2008, Pages 608-644

  Lee, Jaekyung ^a  

^a UNIVERSITY AT BUFFALO   (United States)

Author keywords

Accountability; Achievement; High stakes testing; Meta analysis; NAEP

Indexed keywords

EID: 68049131017     PISSN: 00346543     EISSN: None     Source Type: Journal    
DOI: 10.3102/0034654308324427     Document Type: Review

References (87)

1. Achieve. (2004). The expectations gap: A 50-state review of high school graduation requirements. Retrieved January 9, 2005, from http://www.achieve.org/files/coursetaking.pdf
2. Adams, J. E., & Kirst, M. W. (1999). New demands and concepts for educational accountability: Striving for results in an era of excellence. In J. Murphy & K. S. Louis (Eds.), Handbook of research on educational administration (pp. 463-490). San Francisco, CA: Jossey-Bass.
3. American Educational Research Association, American Psychological Association, & National Council on Educational Measurement. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4. Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning. Education Policy Analysis Archives, 10(18). Retrieved June 14, 2003, from http://epaa.asu.edu/epaa/v10n18/
5. Amrein-Beardsley, A. A., & Berliner, D.C. (2003). Re-analysis of NAEP math and reading scores in states with and without high-stakes tests: Responses to Rosenshine. Education Policy Analysis Archives, 11(25). Retrieved August 24, 2003, from http://epaa.asu.edu/epaa/v11n25/
6. Baker, E. L (2003). Multiple measures: Toward tiered systems. Educational Measurement: Issues and Practice, 22(2), 13-17.
7. Benveniste, G. (1985). The design of school accountability systems. Educational Evaluation and Policy Analysis, 7(3), 261-279.
8. Berliner, D. C., & Biddle, B. J. (1995). The manufactured crisis: Myth, fraud, and the attack on America's public schools. Reading, MA: Addison-Wesley.
9. Betts, J. R., & Danenberg, A. (2002). School accountability in California: An early evaluation. In D. Ravitch (Ed.), Brookings papers on education policy 2002 (pp. 123-180). Washington, DC: Brookings Institution.
10. Bishop, J. H., Mane, F., Bishop, M., & Moriatry, J. (2001). The role of end-of-course exams and minimum competency exams in standards-based reforms. In D. Ravitch (Ed.), Brookings papers on education policy 2001 (pp. 267-330). Washington, DC: Brookings Institution.
11. Braun, H. (2004, January 5). Reconsidering the impact of high-stakes testing. Education Policy Analysis Archives, 12(1). Retrieved March 10, 2004, from http://epaa.asu.edu/epaa/v12n1/
12. Bryk, A. (2003). No Child Left Behind, Chicago-style. In P. E. Peterson & M. R. West (Eds.), No Child Left Behind? The politics and practice of school accountability (pp. 242-268). Washington, DC: Brookings Institution.
13. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
14. Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? Educational Evaluation and Policy Analysis, 24(4), 305-331.
15. Carnoy, M., Loeb, S., & Smith, T. (2001). Do higher scores in Texas make for better high school outcomes? (CPRE Research Report No. RR-047). Philadelphia: Consortium for Policy Research in Education.
16. Clotfelter, C. T., & Ladd, H. F. (1996). Recognizing and rewarding success in public schools. In H. Ladd (Ed.), Holding schools accountable (pp. 23-63). Washington, DC: Brookings Institution.
17. Cohen, D. K., & Haney, W. (1980). Minimums, competency testing, and social policy. In R. M. Jaeger & K. T. Carol (Eds.), Minimum competency achievement testing (chap. 1). Berkeley, CA: McCutchan.
18. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
19. Commission on Instructionally Supportive Assessment. (2001). Building tests that support instruction and accountability: A guide for policymakers. Washington, DC: Author.
20. Council of Chief State School Officers. (1996). Key state education policies on K-12 education. Washington, DC: Author.
21. Darling-Hammond, L. (1989). Accountability for professional practice. Teachers College Record, 91, 59-80.
22. Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Education Policy Analysis Archives, 8(1). Retrieved June 14, 2003 from http://epaa.asu.edu/epaa/v8n1/
23. Dorn, S. (1998). The political legacy of school accountability systems. Education Policy Analysis Archives, 6(1). Retrieved November 5, 2003, from http://epaa.asu.edu/epaa/v6n1/
24. Eckstein, M. A., & Noah, H. J. (1993). Secondary school examinations: International perspectives on policies and practices. New Haven, CT: Yale University Press.
25. Elmore, R. F. (2002, September-October). Testing trap. Harvard Magazine Forum. Retrieved May 31, 2006, from http://www.harvard-magazine.com
26. Erpenbach, W. J., Forte-Fast, E., & Potts, A. (2003). Statewide educational accountability under NCLB: Central issues arising from an examination of state accountability workbooks and U.S. Department of Education reviews under the No Child Left Behind Act of 2001. Washington, DC: Council of Chief State School Officers.
27. Fredericksen, N. (1994). The influence of minimum competency tests on teaching and learning. Princeton, NJ: Educational Testing Service.
28. Fuller, B., Gesicki, K., Kang, E. & Wright, J. (2006). Is the No Child Left Behind Act working? The reliability of how states track achievement (PACE Working Paper 06-1). Berkeley: University of California.
29. Goertz, M. E. (1986). State educational standards: A 50-state survey. Princeton, NJ: Educational Testing Service. (ERIC Document Reproduction Service No. ED 275 726)
30. Grissmer, D., & Flanagan, A. (1998). Exploring rapid achievement gains in North Carolina and Texas. Washington, DC: National Education Goals Panel.
31. Grissmer, D., Flanagan, A., Kawata, J., & Williamson, S. (2000). Improving student achievement: What state NAEP test scores tell us. Santa Monica, CA: RAND.
32. Haney, W. (2000). The myth of the Texas miracle in education. Educational Policy Analysis Archives, 8. Retrieved March 3, 2001, from http://epaa.asu.edu/epaa/v8n41/
33. Hanushek, E. A., & Raymond, M. E. (2004). Does school accountability lead to improved performance? Journal of Policy Analysis and Management, 24(2), 297-327.
34. Harris, D. N., & Herrington, C. D. (2006). Accountability, standards, and the growing achievement gap: Lessons from the past half-century. American Journal of Education, 112, 209-238.
35. Hauser, R. M. (2002). Comment by Robert M. Hauser. In D. Ravitch (Ed.), Brookings papers on education policy 2002 (pp. 184-194). Washington, DC: Brookings Institution.
36. Hedges, L. V. (1990). Directions for future methodology. In K. W. Watcher & M. L. Straf (Eds.), The future of meta-analysis (pp. 11-26). New York: Russell Sage.
37. Hedges, L. V., Laine, R. D., & Greenwald, R. (1994). Does money matter? A meta-analysis of studies of the effects of differential school inputs on student outcomes. Educational Researcher, 23(3), 5-14.
38. Heubert, J. P. (2000). Graduation and promotion testing: Potential benefits and risks for minority students, English-language learners, and students with disabilities. Poverty and Race, 9(5), 1-2, 5-7.
39. Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.
40. Hoxby, C. M. (2002). The cost of accountability (National Bureau of Economic Research Working Paper 8855). Retrieved September 10, 2003, from http://www.nber.org/papers/w8855/
41. Jacob, B. A. (2001). Getting tough? The impact of high school graduation exams. Educational Evaluation and Policy Analysis, 23(2), 99-121.
42. Jacob, B. A. (2003). A closer look at achievement gains under high-stakes testing in Chicago. In P. E. Peterson & M. R. West (Eds.), No Child Left Behind? The politics and practice of school accountability (pp. 269-291). Washington, DC: Brookings Institution.
43. Kirkland, M. C. (1971). The effects of tests on students and schools. Review of Educational Research, 41, 303-350.
44. Kane, T. J., & Staiger, D. O. (2002). Volatility in school test scores. In D. Ravitch (Ed.), Brookings papers on education policy 2002 (pp. 235-284). Washington, DC: Brookings Institution.
45. Klein, S. P., Hamilton, L. S., McCaffrey, D. F., & Stecher, B. M. (2000). What do test scores in Texas tell us? Santa Monica, CA: RAND.
46. Koretz, D., & Barron, S. I. (1998). The validity of gains on the Kentucky Instructional Results Information System (KIRIS) (MR-792-PCT/FF). Santa Monica, CA: RAND.
47. Ladd, H. F. (1999). The Dallas school accountability and incentive program: An evaluation of its impact on student outcomes. Economics of Education Review, 18, 1-16.
48. Langenfeld, K. L., Thurlow, M. L., & Scott, D. L. (1996). High stakes testing for students: Unanswered questions and implications for students with disabilities (Synthesis Report No. 26). Retrieved January 10, 2005, from http://education.umn.edu/NCEO/OnlinePubs/Synthesis26.htm
49. Lee, J. (1997). State activism in education reform: Applying the Rasch model to measure trends and examine policy coherence. Educational Evaluation and Policy Analysis, 19(1), 29-43.
50. Lee, J. (1998). State policy correlates of the achievement gap among racial and social groups. Studies in Educational Evaluation, 24(2), 137-152.
51. Lee, J. (2002). Racial and ethnic achievement gap trends: Reversing the progress toward equity? Educational Researcher, 31(1), 3-12.
52. Lee, J. (2006a). Input-guarantee vs. performance-guarantee approaches to school accountability: Cross-state comparisons of policies, resources, and outcomes. Peabody Journal of Education, 81(4), 43-64.
53. Lee, J. (2006b). Tracking achievement gaps and assessing the impact of NCLB on the gaps: An in-depth look into national and state reading and math outcome trends. Cambridge, MA: The Civil Rights Project at Harvard University.
54. Lee, J. (2007). Revisiting the impact of high-stakes testing on student outcomes from an international perspective. In L. Deretchin & C. Craig (Eds.), International research on the impact of accountability systems (pp. 65-82). Lanham, MD: Rowman & Littlefield.
55. Lee, J., & Coladarci, T. (2002). Using multiple measures to evaluate the performance of students and schools: Learning from the cases of Kentucky and Maine. Orono: University of Maine.
56. Lee, J., & Wong, K. K. (2004). The impact of accountability on racial and socioeconomic equity: Considering both school resources and achievement outcomes. American Educational Research Journal, 41(4), 797-832.
57. Linn, R. L. (2001). The design and evaluation of educational assessment and accountability systems (CSE Technical Report 539). Los Angeles: Center for the Study of Evaluation.
58. Linn, R. L. (2003). Accountability: Responsibility and reasonable expectations. Educational Researcher, 32(7), 3-13.
59. Linn, R. L., Baker, E. L., & Betebenner, D. W. (2002). Accountability systems: Implications of requirements of the No Child Left Behind Act of 2001. Educational Researcher, 31, 3-16.
60. Linn, R. L., & Haug, C. (2002). Stability of school-building accountability scores and gains. Educational Evaluation and Policy Analysis, 24(1), 29-36.
61. Madaus, G. (1988). The influence of testing on the curriculum. In L. Tanner (Ed.), Critical issues in curriculum: 87th yearbook of the NSSE Part 1. Chicago: University of Chicago Press. (ERIC Document Reproduction Service No. 263 183)
62. Madaus, G., & Greaney, V. (1985). The Irish experience in competency testing: Implications for American education. American Journal of Education, 93(2), 268-293.
63. Marion, S. F., White, C., Carlson, D., Erpenbach, W. J., Rabinowitz, S., Sheinker, J. (2002). Making valid and reliable decisions in the determination of adequate yearly progress. Washington, DC: Council of Chief State School Officers.
64. McLendon, M. K., Hearn, J. C., & Deaton, R. (2006). Called to account: Analyzing the origins and spread of state performance-accountability policies for higher education. Educational Evaluation and Policy Analysis, 28(1), 1-24.
65. National Commission on Excellence in Education. (1983). A nation at risk: The imperative for education reform. Washington, DC: U.S. Government Printing Office.
66. Newmann, F. M., King, M. B., & Rigdon, M. (1997). Accountability and school performance: Implications from restructuring schools. Harvard Educational Review, 67(1), 41-74.
67. Nichols, S. L., Glass, G. V, & Berliner, D. C. (2006). High-stakes testing and student achievement: Does accountability pressure increase student learning? Education Policy Analysis Archives, 14(1). Retrieved May 1, 2006, from http://epaa.asu.edu/epaa/v14n1/
68. North Central Regional Educational Laboratory. (1996). State student assessment programs database. Oak Brook, IL: Author.
69. O'Day, J. (2002). Complexity, accountability, and school improvement. Harvard Educational Review, 72(3), 293-329.
70. Olson, L. (2007, January 11). New bills would prod states to take national view on standards. Education Week. Retrieved January 29, 2007, from http://www.edweek.org
71. Orfield, G., & Kornhaber, M. (Eds.) (2001). Raising standards or raising barriers? Inequality and high-stakes testing in public education. New York: Century Foundation.
72. Osborne, D., & Gaebler, T. (1992). Reinventing government: How the entrepreneurial spirit is transforming the public sector. Reading, MA: Addison-Wesley.
73. Peterson, P. E. (Ed.). (2006). Generational change: Closing the test score gap. Lanham, MD: Rowman & Littlefield.
74. Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90(1), 175-181.
75. Phelps, R. P. (2005). The rich, robust research literature on testing's achievement benefits. In R. P. Phelps (Ed.), Defending standardized tests (pp. 55-90). Mahwah, NJ: Lawrence Erlbaum.
76. Porter, A., & Chester, M. (2002). Building a high-quality assessment and accountability program: The Philadelphia example. In D. Ravitch (Ed.), Brookings papers on education policy 2002 (pp. 285-315). Washington DC: Brookings Institution.
77. Raymond, M. E., & Haushek, E. A. (2003, Summer). High-stakes research. Education Next. Retrieved January 20, 2004, from http://www.educationnext.org
78. Rosenshine, B. (2003). High-stakes testing: Another analysis. Education Policy Analysis Archives, 11(24). Retrieved December 8, 2004, from http://epaa.asu.edu/epaa/v11n24/
79. Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 231-244). New York: Russell Sage.
80. Rowan, B., & Miskel, C. G. (1999). Institutional theory and the study of educational organizations. In J. Murphy & K. S. Louis (Eds.), Handbook of research on educational administration (pp. 359-384). San Francisco: Jossey-Bass.
81. Shavelson, R. J., & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC: National Academy Press.
82. Shepard, L. A. (1991). Will national tests improve student learning? Phi Delta Kappan, 72, 232-238.
83. Skrla, L., Scheurich, J. J., Johnson, J. F., & Koschoreck, J. W. (2004). Accountability for equity: Can state policy leverage social justice? In L. Skrla & J. J. Scheurich (Eds.), Educational equity and accountability: Paradigms, policies, and politics (pp. 51-78). New York: Routledge Falmer.
84. Slavin, R. E. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15-21.
85. Swanson, C. B., & Stevenson, D. L. (2002). Standards-based reform in practice: Evidence on state policy and classroom instruction from the NAEP state assessments. Educational Evaluation and Policy Analysis, 24(1), 1-28.
86. Valencia, R. R., Valenzuela, A., Sloan, K., & Foley, D. (2004). Let's treat the cause, not the symptoms: Equity and accountability in Texas revisited. In L. Skrla & J. J. Scheurich (Eds.), Educational equity and accountability: Paradigms, policies, and politics (pp. 29-38). New York: Routledge Falmer.
87. Wise, A. E. (1979). Legislated learning: The bureaucratization of the American classroom. Berkeley: University of California Press.