Research on evaluation use: A review of the empirical literature from 1986 to 2005

American Journal of Evaluation

Volumn 30, Issue 3, 2009, Pages 377-410

  Johnson, Kelli ^a   Greenseid, Lija O ^a   Toal, Stacie A ^a   King, Jean A ^a   Lawrenz, Frances ^a   Volkov, Boris ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b UNIVERSITY OF NORTH DAKOTA   (United States)

Author keywords

Evaluation influence; Evaluation use; Literature review; Research on evaluation; Stakeholder involvement

Indexed keywords

EID: 70249147676     PISSN: 10982140     EISSN: 15570878     Source Type: Journal    
DOI: 10.1177/1098214009341660     Document Type: Article

References (65)

1. Alkin, M. C., & Taut, S. M. (2003). Unbundling evaluation use. Studies in Educational Evaluation, 29, 1-12.
2. Altschuld, J. W., Yoon, J. S., & Cullen, C. (1993). The utilization of needs assessment results. Evaluation and Program Planning, 16, 279-285.
3. Ayers, T. D. (1987). Stakeholders as partners in evaluation: A stakeholder-collaborative approach. Evaluation and Program Planning, 10, 263-271.
4. Bamberger, M. (2004). Influential evaluations: Evaluations that improved performance and impacts of development programs. Washington, DC: Operations Evaluation Department, The World Bank.
5. Barrios, N. B. (1986). Utilization of evaluation information: A case study approach investigating factors related to evaluation utilization in a large state agency. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 47, 1704 (UMI 8616880).
6. Bober, C. E., & Bartlett, K. R. (2004). The utilization of training program evaluation in corporate universities. Human Resource Development Quarterly, 15, 363-383.
7. Boyer, J. F., & Langbein, L. I. (1991). Factors influencing the use of health evaluation research in Congress. Evaluation Review, 18, 507-532.
8. Brown-McGowan, S. (1992). Effects of decision maker and context variables on evaluation utilization. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 53, 2261 (UMI 9226505).
9. Cai, M. (1996). An empirical examination of participatory evaluation: Teachers' perceptions of their involvement and evaluation use. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 57, 1921 (UMI 9629749).
10. Callahan, C. M., Tomlinson, C. A., Hunsaker, S. L., Bland, L. C., & Moon, T. (1995). Instruments and evaluation designs used in gifted programs. The National Research Center on the Gifted and Talented, The University of Virginia. Research Report: RM-95132.
11. Carpinello, S. E. (1989). The effect of power, consequence, and experience on nurse decision-makers' utilization of evaluation information. State University of New York at Albany. Dissertation Abstracts International: Section B: The Sciences & Engineering, 50, 3395.
12. Chin, M. C. (2003). An investigation into the impact of using poetry and cartoons as alternative representational forms in evaluation reporting. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 64, 394 (UMI 3081434).
13. Christie, C. A. (2007). Reported influence of evaluation data on decision makers' actions: An empirical examination. American Journal of Evaluation, 28, 8-25.
14. Combs, W. L. A. (1999). The predictive validity of teachers' attitudes towards inclusive education on the conceptual use of program evaluation information. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 60, 3208 (UMI 9946401).
15. Cousins, J. B. (1995). Assessing program needs using participatory evaluation: A comparison of high and marginal success cases. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 55-71). London: Routledge.
16. Cousins, J. B. (1996). Consequences of researcher involvement in participatory evaluation. Studies in Educational Evaluation, 22, 3-27.
17. Cousins, J. B. (2003). Utilization effects of participatory evaluation. In T. Kelleghan, & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 245-265). Great Britain: Kluwer Academic Publishers.
18. Cousins, J. B., Goh, S. C., Clark, S., & Lee, L. E. (2004). Integrating evaluative inquiry into the organizational culture: A review and synthesis of the knowledge base. Canadian Journal of Program Evaluation, 19, 99-141.
19. Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research on evaluation utilization. Review of Educational Research, 56, 331-364.
20. Crotti, J. G. (1993). Evaluation utilization: A study of administrators' perceptions of the uses of the long-range plan evaluation process in Pennsylvania. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 54, 4315 (UMI 9414267).
21. Earl, L. M. (1995). District-wide evaluation of school improvement: A system partners approach. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 21-32), London: Routledge.
22. Eisendrath, A. (1988). The use of development project evaluation information: A study of state agencies in India. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 49, 1572 (UMI 8810460).
23. Forss, K., Cracknell, B., & Samset, K. (1994). Can evaluation help an organization to learn? Evaluation Review, 18, 574-591.
24. Greene, J. C. (1987). Stakeholder participation in evaluation design: Is it worth the effort? Evaluation and Program Planning, 10, 379-394.
25. Greene, J. C. (1988). Communication of results and utilization in participatory program evaluation. Evaluation and Program Planning, 11, 341-351.
26. Guarino, C., Santibañez, L., Daley, G., & Brewer, D. (2004, May). A review of the research literature on teacher recruitment and retention. Technical report TR-164-EDU. Santa Monica, CA: RAND.
27. Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.
28. Haddock, R. E. (1998). State legislative program evaluation: An assessment of recent claims of direct utilization in the states. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 60, 881 (UMI 9921400).
29. Henry, G. T., & Mark, M. M. (2003a). Beyond use: Understanding evaluation's influence on attitudes and actions. American Journal of Evaluation, 24, 293-314.
30. Henry, G. T., & Mark, M. M. (2003b). Toward an agenda for research on evaluation. New Directions for Evaluation, 97, 69-80.
31. Hofstetter, C. H., & Alkin, M. C. (2003). Evaluation use revisited. In T. Kelleghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 197-222). Great Britain: Kluwer Academic Publishers.
32. Hopstock, P., Young, M., & Zehler, A. (1993). Serving different masters: Title VII evaluation practice and policy. Vol. I - Final report. Arlington, VA: Development Associates IncReport: ED/OPP93-32.
33. Johnson, K. W. (1980). Academia and practice. Knowledge: Creation, Diffusion, Utilization, 2, 237-261.
34. Johnson, R. B. (1993). An exploratory conjoint measurement study of selected variables related to innovative educational evaluation participation and instrumental utilization. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 55, 64 (UMI 9416264).
35. Johnston, W. P., Jr. (1986). A study of the acceptance of management performance evaluation recommendations by federal agencies: Lessons from GAO reports issued in FY 1983. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 48, 2157 (UMI 8725055).
36. King, J. A. & Pechman, E. M. (1984). Pinning a wave to the shore: Conceptualizing evaluation use in school systems. Educational Evaluation and Policy Analysis, 6, 241-451.
37. Kirkhart, K. E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. New Directions for Evaluation, 88, 5-23.
38. Kirkhart, K. E., Morgan, R. O., & Sincavage, J. (1991). Assessing evaluation performance and use: Test-Retest. Evaluation Review, 15(4), 482-502.
39. Lafleur, C. (1995). A participatory approach to district-level program evaluation: The dynamics of internal evaluation. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 33-54). London: Falmer.
40. Lee, L. E., & Cousins, J. B. (1995). Participation in evaluation of funded school improvement: Effects and supporting conditions. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in evaluation use and organizational learning (pp. 72-85). London: Routledge.
41. Leviton, L. C., & Hughes, E. F. X. (1981). Research on the utilization of evaluations: A review and synthesis. Evaluation Review, 5, 525-549.
42. Malen, B., Murphy, M. J., & Geary, S. (1988). The role of evaluation information in legislative decision making: A case study of a loose cannon on deck. Theory into Practice, 27, 111-125.
43. Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programs. San Francisco: Jossey-Bass, Inc.
44. Mark, M. M., & Henry, G. T. (2004). The mechanisms and outcomes of evaluation influence. Evaluation, 10, 35-57.
45. Marra, M. (2003). Dynamics of evaluation use as organizational knowledge: The case of the World Bank. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 64, 1070 (UMI 3085545).
46. Marsh, D. D., & Glassick, J. M. (1988). Knowledge utilization in evaluation efforts: The role of recommendations. Knowledge, 9, 323-341.
47. McCormick, E. R. (1997). Factors influencing the use of evaluation results. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 58, 4187 (UMI 9815051).
48. Newman, D., Brown, R., Rivers, L., & Glock, R. (1983). School boards' and administrators' use of evaluation information: Influencing factors. Evaluation Review, 7(1), 110-125.
49. Newman, D. L., Brown, R. D., & Rivers, L. (1987). Factors influencing the decision-making process: An examination of the effect of contextual variables. Studies in Educational Evaluation, 13, 199-209.
50. Nunneley, R. D. (2008). The danger of theorizing under the influence: An analysis of the arguments in Henry and Mark (2003) and Mark and Henry (2004). Unpublished manuscript. Minneapolis, MN: University of Minnesota.
51. Patton, M. Q. (1997). Utilization-focused evaluation. Thousand Oaks, CA: Sage.
52. Potts, S. A. K. (1998). Impact of mixed method designs on knowledge gain, credibility, and utility of program evaluation findings. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 59, 1942 (UMI 9837695).
53. Preskill, H., & Caracelli, V. (1997). Current and developing conceptions of use: Evaluation use TIG survey results. Evaluation Practice, 18, 209-225.
54. Rinne, C. (1994). The impact of anxiety as a mediating variable on health educators' utilization of evaluation results. Dissertation Abstracts International: Section B: The Sciences & Engineering, 54, 3554.
55. Rockwell, S. K., Dickey, E. C., & Jasa, P. J. (1990). The personal factor in evaluation use: A case study of a steering committee's use of a conservation tillage survey. Evaluation and Program Planning, 13, 389-394.
56. Russon, C., & Koehly, L. (1995). Construction of a scale to measure the persuasive impact of qualitative and quantitative evaluation reports. Evaluation and Program Planning, 18(2), 165-177.
57. Santhiveeran, J. (1995). Factors influencing the utilization of evaluation findings in mental health centers: A national survey. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 56, 3311 (UMI 9539597).
58. Scriven, M. (2007). Activist evaluation. Journal of Multi Disciplinary Evaluation, 4(7), i-ii.
59. Shea, M. P. (1991). Program evaluation utilization in Canada and its relationship to evaluation process, evaluator and decision context variables. University of Windsor (Canada). Dissertation Abstracts International: Section B: The Sciences & Engineering, 53, 597.
60. Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research, and practice since 1986. Evaluation Practice, 18, 195-208.
61. Sleezer, C. M. (1987). The relationship between types of evaluation reports and support for the training function by corporate managers. Project number twenty-one. Department of Vocational and Technical Education, University of Minnesota.
62. Sperlazza, J. (1995). Involving school professionals in program evaluation in an urban school district. Dissertation Abstracts International: Section A: The Humanities and Social Sciences, 56, 3406 (UMI 9601924).
63. Turnbull, B. (1999). The mediating effect of participation efficacy on evaluation use. Evaluation and Program Planning, 22, 131-140.
64. Weeks, E. C. (1979). The managerial use of evaluation findings. In H.C. Schulberg & J.M. Jerrell (Eds.), The evaluator and management (pp. 137-156). Beverly Hills, CA: Sage.
65. Weiss, C. H., Murphy-Graham, E., & Birkeland, S. (2005). An alternate route to policy influence: How evaluations affect DARE. American Journal of Evaluation, 26, 12-30.