Raising the Bar

Evaluation Review

Volumn 41, Issue 3, 2017, Pages 212-239

  Seftor, Neil ^a  

^a MATHEMATICA POLICY RESEARCH   (United States)

Author keywords

content area; education; methodological development

Indexed keywords

DECISION MAKING; EDUCATION; EVIDENCE BASED PRACTICE; FACTUAL DATABASE; UNITED STATES;

DATABASES, FACTUAL; DECISION MAKING; EDUCATION; EVIDENCE-BASED PRACTICE; SYSTEMATIC REVIEWS AS TOPIC; UNITED STATES;

EID: 85019833632     PISSN: 0193841X     EISSN: 15523926     Source Type: Journal    
DOI: 10.1177/0193841X16665023     Document Type: Article

References (57)

1. Angrist J., Pischke J.-S., (2010). The credibility revolution in empirical economics: How better research design is taking the con out of econometrics. Journal of Economic Perspectives, 24, 3–30.
2. Avellar S., Dion M., Clarkwest A., Zaveri H., Asheer S., Borradaile K., …Zukiewicz M., (2011). Catalog of research: Programs for low-income fathers (OPRE Report #2011-20). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
3. Averch H., Carroll S., Donaldson T., Kiesling H., Pincus J., (1972). How effective is schooling? A critical review and synthesis of research findings. The Rand Corporation. Retrieved from http://www.rand.org/pubs/reports/2006/R956.pdf
4. Baron J., (2013). Randomized controlled trials commissioned by the Institute of Education Sciences since 2002: How many found positive versus weak or no effects. Washington, DC: Coalition for Evidence-Based Policy.
5. Bero L., Rennie D., (1995). The Cochrane Collaboration: Preparing, maintaining, and disseminating systematic reviews of the effects of health care. Journal of the American Medical Association, 274, 1935–1938.
6. Boruch R., de Moya D., Snyder B., (2002). The importance of randomized field trials in education and related areas. In Mosteller F., Boruch R., (Eds.), Evidence matters: Randomized field trials in education research (pp. 50–79). Washington, DC: Brookings Institution Press.
7. Brown H., Card D., Dickerson K., Greenhouse J., Kling J., Littell J., (2008). Report of the What Works Clearinghouse expert panel. Retrieved from http://ies.ed.gov/director/board/pdf/panelreport.pdf
8. Burwell S., Muñoz C., Holdren J., Krueger A., (2013). Next steps in the evidence and innovation agenda[Memorandum]. Washington, DC: Office of Management and Budget.
9. Campbell Collaboration. (2011). C2 review template. Retrieved from http://www.campbellcollaboration.org/artman2/uploads/1/C2_Review_template_v1.doc
10. Campbell D., Stanley J., (1966). Experimental and quasi-experimental designs for research. Chicago, IL: Rand McNally.
11. Cochrane A., (1979). 1931–1971: A critical review, with particular reference to the medical profession. In Teeling-Smith G., Wells N., (Eds.), Medicines for the year 2000 (pp. 1–11). London, England: Office of Health Economics.
12. Consolidated Standards of Reporting Trials. (2010). CONSORT 2010 checklist. Retrieved from http://www.consort-statement.org/consort-statement/
13. Cook D., Mulrow C., Haynes R., (1997). Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126, 376–380.
14. Eisenhart M., Towne L., (2003). Contestation and change in national policy on “scientifically based” education research. Education Researcher, 32, 31–38.
15. Eldridge S., Ashby D., Bennett C., Wakelin M., Feder G., (2008). Internal and external validity of cluster randomised trials: Systematic review of recent trials. British Medical Journal, 336, 876–880.
16. Epstein D., Klerman J. A., (2012). When is a program ready for rigorous impact evaluation? Evaluation Review, 36, 373–399.
17. Flexner A., (1916). A modern school. American Review of Reviews, 53, 465–474.
18. Goesling B., Oberlander S., Trivits L. (in press). High-stakes systematic reviews: A case study from the field of teen pregnancy prevention. Evaluation Review.
19. Government Accountability Office. (2010). Improved dissemination and timely product release would enhance the usefulness of the What Works Clearinghouse (GAO-10.644). Washington, DC: U.S. Government Accountability Office.
20. Gueron J., Rolston H., (2013). Fighting for reliable evidence. New York, NY: Russell Sage Foundation.
21. Haskins R., Baron J., (2011). Building the connection between policy and evidence: The Obama evidence-based initiatives. London, England: NESTA.
22. Haskins R., Margolis G., (2014). Show me the evidence: Obama’s fight for rigor and results in social policy. Washington, DC: Brookings Institution Press.
23. Higgins J. P. T., Green S., (Eds.). (2011). Cochrane handbook for systematic reviews of interventions (Version 5.1.0, updated March 2011). London, England: The Cochrane Collaboration. Retrieved from www.cochrane-handbook.org
24. Hsieh P., Hsieh Y., Chung W., Acee T., Thomas G., Kim H., …Robinson D. (2005). Is educational intervention research on the decline? Journal of Educational Psychology, 97, 523–529.
25. Institute of Education Sciences. (2008a). Rigor and relevance redux: Director’s biennial report to Congress (IES 2009-6010). Washington, DC: U.S. Department of Education.
26. Institute of Education Sciences. (2008b). What Works Clearinghouse evidence standards for reviewing studies (Version 1.0). Washington, DC: U.S. Department of Education.
27. Institute of Education Sciences. (2008c). What Works Clearinghouse procedures and standards handbook (Version 2.0). Washington, DC: U.S. Department of Education.
28. Institute of Education Sciences. (2011). What Works Clearinghouse procedures and standards handbook (Version 2.1). Washington, DC: U.S. Department of Education.
29. Institute of Education Sciences. (2012). What Works Clearinghouse reporting guide for study authors. Washington, DC: U.S. Department of Education.
30. Institute of Education Sciences. (2014a). Assessing attrition bias (Version 3.0). Washington, DC: U.S. Department of Education.
31. Institute of Education Sciences. (2014b). What Works Clearinghouse procedures and standards handbook (Version 3.0). Washington, DC: U.S. Department of Education.
32. Institute of Education Sciences. (2015). Guidance for the review of studies that present a complier average causal effect. Washington, DC: U.S. Department of Education.
33. Knowles T., Ludwig J., (2013). Strengthening the education sciences reform act: Fostering innovation efficiency through mechanism experiments. In Leveraging learning: The evolving role of federal policy in education research (pp. 31–33). Washington, DC: The Aspen Institute. Retrieved from http://files.eric.ed.gov/fulltext/ED545131.pdf
34. Lagemann E., (2002). The troubling history of education research. Chicago, IL: University of Chicago Press.
35. Manna P., Petrilli M., (2008). Double standard? “Scientifically based research” and the no child left behind act. In Hess F. M., (Ed.), When research matters: How scholarship influences education policy (pp. 63–88). Cambridge, MA: Harvard Education Press.
36. Mastri A., Sama-Miller E., Clarkwest A., (2015). Employment strategies for low-income adults evidence review: Standards and methods. Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
37. McCall R. B., (2009). Evidence-based programming in the context of practice and policy. SRCD Social Policy Report, 23, 3–18.
38. National Research Council. (1999). Improving student learning: A strategic plan for education research and its utilization. Washington, DC: The National Academies Press.
39. Neuhoff A., Axworthy S., Glazer S., Berfond D., (2015). The What Works marketplace helping leaders use evidence to make smarter choices: Results for America. Boston, MA: The Bridgespan Group.
40. Newman M., Elbourne D., (2004). Improving the usability of education research: Guidelines for the REPOrting of primary empirical research studies in education (the REPOSE guidelines). Evaluation & Research in Education, 18, 201–212.
41. Orr L., 2015 2014 Rossi Award lecture: Beyond internal validity. Evaluation Review, 39, 587–624.
42. Orszag P., (2009). Increased emphasis on program evaluations [Memorandum]. Washington, DC: Office of Management and Budget.
43. Orszag P., (2010). Evaluating programs for efficacy and cost-efficiency [Memorandum]. Washington, DC: Office of Management and Budget.
44. Paulsell D., Thomas J., Monahan S., Seftor N., (in press). A trusted source of information: How systematic reviews can support user decisions about adopting evidence-based programs. Evaluation Review.
45. Preferred Reporting Items for Systematic Reviews and Meta-Analyses. (2009). PRISMA 2009 checklist. Retrieved from http://www.prisma-statement.org/statement.htm
46. Schulz K. F., Altman D. G., Moher D., (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 8, 18.
47. Slavin R., (2003). A reader’s guide to scientifically based research. Educational Leadership, 60, 12–16.
48. Slavin R. (2015). Evidence-based vs. evidence-proven [Web log post]. Retrieved from http://www.huffingtonpost.com/robert-e-slavin/evidence-based-vs-evidenc_b_7614582.html.
49. Supplee L., Paulsell D., Avellar S., (2012). What works in home visiting programs? In Curtis P., Alexander G., (Eds.), What works in child welfare (pp. 39–61). Washington, DC: Child Welfare League of American Press.
50. The Federal Role in Education Research: Providing relevant information to students, parents, and educators: Hearing before the Subcommittee on Early Childhood, Elementary, and Secondary Education, of the House Committee on Education and the Workforce, 112th Cong. (2011) (testimony of Grover Whitehurst, PhD).
51. The Reauthorization of the Office of Educational Research and Improvement: Hearing before the Subcommittee on Education Reform, of the House Committee on Education and the Workforce, 107th Cong. (2002) (testimony of Grover Whitehurst, PhD).
52. Tranfield D., Denyer D., Smart P., (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207–222.
53. Valentine J. C., Wilson S. J., Rindskopf D., Lau T. S., Tanner-Smith E. E., Yeide M., LaSota R., Foster L., (in press). The challenge of synthesis when there are only a few studies to review. Evaluation Review.
54. West M. (2015). From evidence-based programs to an evidence-based system: Opportunities under the Every Student Succeeds Act [Web log post]. Retrieved from http://www.brookings.edu/research/papers/2016/02/05-evidence-based-system-opportunities-under-essa-west.
55. Westbrook T., Avellar S., Seftor N. (in press). Reviewing the reviews: Examining similarities and differences between federally funded evidence reviews. Evaluation Review.
56. Zients J., (2012). Use of evidence and evaluation in the 2014 budget [Memorandum]. Washington, DC: Office of Management and Budget.
57. Zorzela L., Golder S., Liu Y., Pilkington K., Hartling L., Joffe A., …Vohra S. (2014). Quality of reporting in systematic reviews of adverse events: Systematic review. British Medical Journal, 348, f7668.