An evaluator's balancing act: Making decisions about methodological rigor

New Directions for Evaluation

Volumn 2008, Issue 120, 2008, Pages 71-86

  Braverman, Marc T ^a   Arnold, Mary E ^b  

^a OREGON STATE UNIVERSITY   (United States)

^b OREGON STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77950723946     PISSN: 10976736     EISSN: 1534875X     Source Type: Journal    
DOI: 10.1002/ev.277     Document Type: Article

References (26)

1. Bennett, C. F., & Rockwell, K. (1996). Targeting Outcomes of Programs (TOP): An integrated approach to planning and evaluation. Washington, DC: CSREES, USDA.
2. Block, A., Laughlin, S., Del Tredici, A., & Omelich, C. (1985). California EFNEP Evaluation Study. Berkeley: Cooperative Extension, University of California Division of Agriculture and Natural Resources.
3. Braverman, M. T. (1996). Sources of survey error: Implications for evaluation studies. In M. T. Braverman & J. K. Slater (Eds.), Advances in survey research. New Directions for Evaluation, 70, 17–28.
4. Chatterji, M. (2007). Grades of evidence: Variability in quality of findings in effectiveness studies of complex field interventions. American Journal of Evaluation, 28(3), 239–255.
5. Chelimsky, E. (2007). Factors influencing the choice of methods in federal evaluation practice. In G. Julnes & D. J. Rog (Eds.), Informing federal policies on evaluation methodology. New Directions for Evaluation, 113, 13–33.
6. Cooperative State Research, Education, and Extension Service. (2007). Research studies of the Expanded Food and Nutrition Education Program: 1989–2006.
7. U.S. Department of Agriculture. Retrieved April 11, 2008, from http://www.csrees.usda.gov/nea/food/efnep/pdf/research_studies.pdf
8. Del Tredici, A. M., Joy, A. B., Omelich, C. L., & Laughlin, S. G. (1988). Evaluation study of the California Expanded Food and Nutrition Education Program: 24-hour food recall data. Journal of the American Dietetic Association, 88(2), 185–190.
9. Dennis, M. L. (1994). Ethical and practical randomized field experiments. In J. S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of practical program evaluation (pp. 155–197). San Francisco: Jossey-Bass.
10. Institute of Education Sciences. (2008). What Works Clearinghouse. Retrieved April 12, 2008, from http://ies.ed.gov/ncee/wwc/
11. Lipsey, M. W. (1993). Theory as method: Small theories of treatments. In L. B. Sechrest & A.G. Scott (Eds.), Understanding causes and generalizing about them. New Directions for Program Evaluation, 57, 5–38.
12. Lipsey, M. W. (1998). Design sensitivity: Statistical power for applied experimental research. In L. Bickman & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 39–68). Thousand Oaks, CA: Sage.
13. Minnesota Department of Education. (2007). Minnesota Student Survey. Retrieved April 11, 2008, from http://education.state.mn.us/MDE/Learning_Support/Safe_and_Healthy_Learners/Minnesota_Student_Survey
14. Minnesota 4-H Program. (2005). Minnesota 4-H Youth Survey. Retrieved April 11, 2008, from http://www.fourh.umn.edu/evaluation/evaluating/youthsurvey.html
15. Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage.
16. Preskill, H., & Torres, R. T. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage.
17. Randall, M. J., Brink, M. S., & Joy, A. B. (1989). EFNEP: An investment in America's future. Journal of Nutrition Education, 21(6), 276–279.
18. Rogers, P. J., Hacsi, T. A., Petrosino, A., & Huebner, T. A. (Eds.). (2000). Program theory in evaluation: Challenges and opportunities. New Directions for Evaluation, 87.
19. Russ-Eft, D., & Preskill, H. (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. Cambridge, MA: Perseus.
20. Schweigert, F. J. (2006). The meaning of effectiveness in assessing community initiatives. American Journal of Evaluation, 27(4), 416–436.
21. Shadish, W. R., Cook, T. D., & Campbell D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin.
22. Society for Prevention Research. (2004). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Retrieved February 9, 2008, from http://www.preventionresearch.org/StandardsofEvidencebook.pdf
23. St. Pierre, R. G. (2004). Using randomized experiments. In J. S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of practical program evaluation (2nd ed., pp. 150–175). San Francisco: Jossey-Bass.
24. Substance Abuse and Mental Health Services Administration. (2008). National Registry of Evidence-based Programs and Practices. Retrieved April 12, 2008, from http://nrepp.samhsa.gov/
25. Trochim, W. (2007). Evolutionary perspectives in evaluation: Theoretical and practical implications. Retrieved February 9, 2008, from http://www.socialresearchmethods.net/research/EERS2007/Evolutionary%20Perspectives%20in%20Evaluation%20Theoretical%20and%20Practical%20Implications.pdf
26. Weiss, C. H., Murphy-Graham, E., Petrosino, A., & Gandhi, A. G. (2008). The Fairy Godmother—and her warts: Making the dream of evidence-based policy come true. American Journal of Evaluation, 29(1), 29–47.