When is a Program Ready for Rigorous Impact Evaluation? The Role of a Falsifiable Logic Model

Evaluation Review

Volumn 36, Issue 5, 2012, Pages 375-401

  Epstein, Diana ^a   Klerman, Jacob Alex ^b  

^a AMERICAN INSTITUTES FOR RESEARCH   (United States)

^b ABT ASSOCIATES   (United States)

Author keywords

design and evaluation of programs and policies; outcome evaluation; program design and development

Indexed keywords

COST BENEFIT ANALYSIS; ECONOMICS; EDUCATIONAL MODEL; EVALUATION STUDY; HUMAN; METHODOLOGY; PROGRAM EVALUATION; REVIEW; STATISTICAL MODEL;

COST-BENEFIT ANALYSIS; EVALUATION STUDIES AS TOPIC; HUMANS; LOGISTIC MODELS; MODELS, EDUCATIONAL; PROGRAM EVALUATION;

EID: 84874215431     PISSN: 0193841X     EISSN: 15523926     Source Type: Journal    
DOI: 10.1177/0193841X12474275     Document Type: Review

References (42)

1. AndersonD.2010. "Proven Programs are the Exception, not the Rule [Web log post]."Accessed January 3, 2013, http://blog.givewell.org/2008/12/18/guest-post-proven-programs-are-the-exception-not-the-rule/.
2. Azurdia G.,Barnes Z.The Employment Retention and Advancement Projects: Impacts for Portland's Career Builders Program. New York: Manpower Demonstration Research Corporation; 2008:.
3. Berk R.Evaluation Review. 2011:191-203.
4. Bernstein L.,Dun Rappaport C.,Olsho L.,Hunt D.,Levin M.2009:.
5. Bos J. M.,Scrivener S.,Snipes J.,Hamilton G.Improving Basic Skills: The Effects of Adult Education in Welfare-to-Work Programs. New York: Manpower Demonstration Research Corporation; 2002:.
6. Brown A.,Klerman J. A.Evaluation Review. 2012:185-2197.
7. Campbell D. T."Reforms as Experiments.".American Psychologist. 1969;24:409-29.
8. Campbell D. T."Can we be Scientific in Applied Social Science?".Evaluation Studies Review Annual. 1984;9:26-48.
9. Coalition for Evidence Based Policy. 2009. "The Coalition is Pleased with GAO's Confirmation of the Top Tier Initiative's Adherence to Rigorous Standards and Overall Transparency." Accessed January 3, 2013, http://coalition4evidence.org/wordpress/?page_id=410.
10. Connell J. P.,Kubisch A. C."Applying a Theory of Change Approach to the Evaluation of Comprehensive Community Initiatives: Progress, Prospects, and Problems.".The Aspen Institute. 1998;:.
11. Conrad K. J.,Randolph F.,Kirby M.,Bebout R. R."Creating and using Logic Models-Four Perspectives.".Alcoholism Treatment Quarterly. 1999;17:17-31.
12. GiveWell. (n.d.). Social Programs that Just don't Work."Accessed January 3, 2013, http://www.givewell.org/giving101/Social-Programs-That-Just-Dont-Work.
13. Ioannidis J. P. A.,Haidich A. B.,Pappa M.,Pantazis N.,Kokori S. I.,Tektonidou M. G.,Contopoulos-Ionnidis D. G.,Lau J."Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies.".Journal of the American Medical Association. 2001;286:821-30.
14. Jalbert S.,Rhodes W.Reduced Caseloads Improve Probation Outcomes in Oklahoma City. Cambridge, MA: Abt Associates; 2010:.
15. James-Burdumy S.,Deke J.,Lugo-Gil J.,Crey N.,Hershey A.,Gersten R.,Newman-Gonchar R.,Dimino J.,Haymond K.,Faddis B.2010:.
16. Judkins D.,St.Pierre R. B.,Goodson B.,Gutmann B.,von Glatz A.,Hamilton J.,Webber A.,Troppe P.,Rimdzius T.2008:.
17. Klerman J. A."Contracting for Independent Evaluation: Approaches to an Inherent Tension.".Evaluation Review. 2010;34 (4, August): 299-333.
18. Leviton L. C.,Gutman M. A.The Systematic Screening and Assessment Method: Finding Innovations Worth Evaluating. New Directions for Evaluation. Leviton L. C.Kettel Khan L.Dawkins N., ed. 2010:7-31.
19. McDonald S.-K.Handbook of Education Policy Research. Sykes G.Schneider B.Plank D. N., ed. New York: Routledge; 2009:191-208.
20. McDonald S.-K."Developmental Stages for Evaluating Scale.".The Evaluation Exchange. 2010;15:.
21. McLaughlin J. A.,Jordan G. B."Logic Models: A Tool for Telling your Program's Performance Story.".Evaluation and Program Planning. 1999;22:65-72.
22. Meckstroth A.,Burwick A.,Ponza M.,Marsh S.,Novak T.,Phillips S.,Diaz-Tena N.,Ng J.2006:.
23. Miller C.,Martin V.,Hamilton G.The Employment Retention and Advancement Projects: Findings for the Cleveland Achieve Model: Implementation and Early Impacts of an Employer-Based Approach to Encourage Employment Retention among Low-Wage Workers. New York, NY: Manpower Demonstration Research Corporation; 2008:.
24. Molina F.,Cheng W.-L.,Hendra R.The Employment Retention and Advancement Project: Results from the Valuing Individual Success and Increasing Opportunities Now (VISION) Program in Salem, Oregon. New York: Manpower Demonstration Research Corporation; 2008:.
25. ObamaB.2009. "Remarks of President Obama on Community Solutions Agenda."Accessed January 3, 2013, http://www.nationalservice.gov/about/newsroom/statements_detail.asp?tbl_pr_id=1828.
26. OrszagP. R.2009a"Building Rigorous Evidence to Drive Policy [Web log post]."Accessed January 3, 2013, http://www.whitehouse.gov/omb/blog/09/06/08/BuildingRigorousEvidencetoDrivePolicy.
27. OrszagP. R.2009b. "Increased Emphasis on Program Evaluation [Memorandum]."Accessed January 3, 2013, http://www.whitehouse.gov/omb/assets/memoranda_2010/m10-01.pdf.
28. Pawson R.,Tilley N.Realistic Evaluation. London, England: Sage; 1997:.
29. Ravallion M."Should the Randomistas Rule?" The Economists'.Voice. 2008;:.
30. Rogers P.,Petrosino A.,Huebner T.,Hacsi T."Program Theory Evaluation: Practice, Promise, and Problems.".New Directions in Evaluation. 2000;87:5-13.
31. Rogers P. J.Encyclopedia of Evaluation. Mathison S., ed. Beverly Hills, CA: Sage; 2005:232.
32. Sawhill I. V.,Baron J."We Need a New Start for Head Start.".Education Week. 2010;29:22-23.
33. Scrivener S.,Azurdia G.,Page J.The Employment Retention and Advancement Projects: Results from the South Carolina ERA site. New York: Manpower Demonstration Research Corporation; 2005:.
34. Shadish W. R.,Cook T. D.,Campbell D. T.Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin; 2001:.
35. Shah, Shivam Mallick,Jolin MicheleAmerica Achieves and the Center for American Progress; 2012:.
36. Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination. 2004:.
37. USAID Evaluation Policy. Bureau for Policy, Planning, and Learning. 2011:.
38. Logic Model Handbook 2008. 2008:.
39. Weiss C.Evaluation. 1997. Evaluation, 2nd ed. New York: Prentice Hall.New York: Prentice Hall; 1997:.
40. Wholey J.Handbook of Practical Program Evaluation. Hatry H. P.Wholey J. S.Newcomer K. E., ed. San Francisco, CA: Jossey-Bass; 1994:.
41. Wood R. G.,McConnell S.,Moore Q.,Clarkwest A.,Hseuh J.Princeton, NJ: Mathematica Policy Research; 2010:.
42. Using Logic Models to Bring Together Planning, Evaluation and Action: Logic Model Development Guide. Battle Creek, MI: W.W. Kellogg Foundation; 2004:.