Using evaluation theory in priority setting and resource allocation

Journal of Health, Organisation and Management

Volumn 26, Issue 5, 2012, Pages 655-671

  Smith, Neale ^a   Mitton, Craig ^b   Cornelissen, Evelyn ^c   Gibson, Jennifer ^d   Peacock, Stuart ^e  

^a VANCOUVER COASTAL HEALTH RESEARCH INSTITUTE   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^d at the University of Toronto Wagner is also a faculty member of the Canadian Medical Associations' Physician Management Institute   (Canada)

^e BRITISH COLUMBIA CANCER AGENCY   (Canada)

Author keywords

Evaluation; Health planning and policy; Priority setting; Resource allocation; Social welfare planning; Theory

Indexed keywords

EID: 84865725197     PISSN: 14777266     EISSN: None     Source Type: Journal    
DOI: 10.1108/14777261211256963     Document Type: Review

References (86)

1. Alkin, M.C. and Christie, C.A. (2004), "An evaluation theory tree" in Alkin, M.C. (Ed.), Evaluation Roots: Tracing Theorists' Views and Influences, Sage, Thousand Oaks, CA, pp. 12-65.
2. Anell, A. (2004), "Priority setting for pharmaceuticals" in European Journal of Health Economics, Vol. 5, pp. 28-35.
3. Armstrong, K., Mitton, C., Carleton, B. and Shoveller, J. (2008), "Drug formulary decision-making in two regional health authorities in British Columbia, Canada" in Health Policy, Vol. 88, pp. 308-16.
4. Baltussen, R. and Niessen, L. (2006), "Priority setting of health interventions: the need for multi-criteria decision analysis" in Cost Effectiveness and Resource Allocation, Vol. 4, p. 14.
5. Bate, A., Donaldson, C. and Murtagh, M.J. (2007), "Managing to manage healthcare resources in the English NHS? What can health economics teach? What can health economics learn?" in Health Policy, Vol. 84, pp. 249-61.
6. Bruni, R.A., Laupacis, A., Levinson, W. and Martin, D.K. (2007), "Public involvement in the priority setting activities of a wait time management initiative: a qualitative case study" in BMC Health Services Research, Vol. 7, p. 186.
7. Chen, H. (1990), Theory-driven Evaluations, Sage, Newbury Park, CA.
8. Chen, H. and Rossi, P. (1983), "Evaluating with sense: the theory-driven approach" in Evaluation Review, Vol. 7, pp. 283-302.
9. Chen, H. and Rossi, P. (1987), "The theory-driven approach to validity" in Evaluation and Program Planning, Vol. 10, pp. 95-103.
10. Christie, C.A. and Alkin, M.C. (2008), "Evaluation theory tree re-examined" in Studies in Educational Evaluation, Vol. 34, pp. 131-5.
11. Christie, C.A. and Masyn, K.E. (2008), "Latent profiles of evaluators' self-reported practices" in Canadian Journal of Program Evaluation, Vol. 23, pp. 225-54.
12. Culyer, A. (2006), "NICE's use of cost-effectiveness as an exemplar of a deliberative process" in Health Economics, Policy and Law, Vol. 1, pp. 299-318.
13. Davis, P. (2005), "The limits of realist evaluation" in Evaluation, Vol. 11, pp. 275-95.
14. Davis, T.S. (2002), "The federal GPRA evaluation mandate" in The Social Policy Journal, Vol. 1, pp. 51-74.
15. Dionne, F., Mitton, C., Smith, N. and Donaldson, C. (2009), "Evaluation of the impact of program budgeting and marginal analysis in Vancouver Island Health Authority" in Journal of Health Services Research and Policy, Vol. 14, pp. 234-42.
16. Donaldson, C., Bate, A., Mitton, C., Peacock, S. and Ruta, D. (2009), "Priority setting in the public sector: turning economics into a management process" Hartley, J., Donaldson, C., Skelcher, C. and Wallace, M. (Eds.), Managing Improvement in Public Service Delivery: Progress and Challenges, Cambridge University Press, Cambridge.
17. Donaldson, S. (2003), "Theory-driven program evaluation in the new millennium" in Donaldson, S.I. and Scriven, M. (Eds.), Evaluating Social Programs and Problems: Visions for the New Millennium, Lawrence Erlbaum Associates, Mahwah, NJ, pp. 109-41.
18. Donaldson, S. and Lipsey, M. (2006), "Roles for theory in contemporary evaluation practice: developing practical knowledge" in Shaw, I.F., Greene, J.C. and Mark, M.M. (Eds.), The SAGE Handbook of Evaluation, Sage, London, pp. 56-75.
19. Eddama, O. and Coast, J. (2008), "A systematic review of the use of economic evaluation in local decision making" in Health Policy, Vol. 86, pp. 129-41.
20. Fetterman, D. (1996), "Empowerment evaluation: an introduction to theory and practice" in Fetterman, D.M., Kaftarian, S.J. and Wandersman, A. (Eds.), Empowerment Evaluation: Knowledge and Tools for Self-assessment and Accountability, Sage, Thousand Oaks, CA, pp. 3-46.
21. Fitzpatrick, J.L. (2004), "Exemplars as case studies: reflections on the links between theory, practice, and context" in American Journal of Evaluation, Vol. 25, pp. 541-59.
22. Geva-May, I. and Pal, L. (1999), "Good fences make good neighbours: policy evaluation and policy analysis - exploring the differences" in Evaluation, Vol. 5, pp. 259-77.
23. Gibson, J.L., Martin, D.K. and Singer, P.A. (2004a), "Evidence, economics, and ethics: resource allocation in health services organizations" in Healthcare Quarterly, Vol. 8, pp. 50-9.
24. Gibson, J.L., Martin, D.K. and Singer, P.A. (2004b), "Setting priorities in health care organizations: criteria, processes, and parameters of success", BMC Health Services Research, Vol. 4.
25. Greener, I. and Powell, J. (2003), "Health authorities, priority-setting and resource allocation: a study of decision-making in New Labour's NHS" in Social Policy & Administration, Vol. 37, pp. 35-48.
26. Guba, E. and Lincoln, Y. (1989), Fourth Generation Evaluation, Sage, Newbury Park, CA.
27. Hatry, H., van Houten, T., Plantz, M.C. and Greenway, M.T. (1996), Measuring Program Outcomes: A Practical Approach, United Way of America, Alexandria, VA.
28. Heberger, A.E., Christie, C.A. and Alkin, M.C. (2010), "A bibliometric analysis of the academic influences of and on evaluation theorists' published works" in American Journal of Evaluation, Vol. 31, pp. 24-44.
29. Henry, G.T. (2000), "Why not use?" in New Directions for Evaluation, Vol. 88, pp. 85-98.
30. Hood, C. (1991), "A public administration for all seasons" in Public Administration, Vol. 69, pp. 3-19.
31. Hood, C. and Peters, G. (2004), "The middle aging of new public management: into the age of paradox" in Journal of Public Administration Research and Theory, Vol. 14, pp. 267-82.
32. House, E. and Howe, K. (1999), Values in Evaluation and Social Research, Sage, Thousand Oaks, CA.
33. House, E. and Howe, K. (2000), "Deliberative democratic evaluation" in New Directions for Evaluation, Vol. 85, pp. 3-12.
34. Howe, K. and Ashcraft, C. (2005), "Deliberative democratic evaluation: successes and limitations of an evaluation of school choice" in Teachers College Record, Vol. 107, pp. 2275-98.
35. Jacobson, B. and Bowling, A. (1995), "Involving the public: practical and ethical issues" in British Medical Bulletin, Vol. 51, pp. 869-75.
36. Joyce, P. (2001), "Governmentality and risk: setting priorities in the new NHS" in Sociology of Health & Illness, Vol. 23, pp. 594-614.
37. Kapiriri, L. and Martin, D.K. (2010), "Successful priority setting in low and middle income countries: a framework for evaluation" in Healthcare Analysis, Vol. 18, pp. 129-47.
38. Kazi, M. (2003), "Realist evaluation for practice" in British Journal of Social Work, Vol. 33, pp. 803-18.
39. Kitzhaber, J.A. (1993), "Prioritising health services in an era of limits: the Oregon experience" in BMJ, Vol. 307, pp. 373-7.
40. Lewis, S. and Kouri, D. (2004), "Regionalization: making sense of the Canadian experience" in Healthcare Papers, Vol. 5, pp. 12-31.
41. Lincoln, Y.S. (2003), "Fourth generation evaluation in the new millennium" in Donaldson, S.I. and Scriven, M. (Eds.), Evaluating Social Programs and Problems: Visions for the New Millennium, Lawrence Erlbaum Associates, Mahwah, NJ, pp. 77-90.
42. MacNeil, C. (2002), "Evaluator as steward of citizen deliberation" in American Journal of Evaluation, Vol. 23, pp. 45-54.
43. Maluka, S., Kamuzora, P., SanSebastian, M., Byskov, J., Ndawi, B., Olsen, O.E. and Hurtig, A.-K. (2011), "Implementing accountability for reasonableness framework at district level in Tanzania: a realist evaluation" in Implementation Science, Vol. 6, p. 11.
44. Mark, M.M., Henry, G.T. and Julnes, G. (1998), "A realist theory of evaluation practice" in New Directions for Evaluation, Vol. 78, pp. 3-32.
45. Martin, D.K., Giacomini, M. and Singer, P.A. (2002), "Fairness, accountability for reasonableness, and the views of priority setting decision-makers" in Health Policy, Vol. 61, pp. 279-90.
46. Menon, D., Stafinski, T. and Martin, D. (2007), "Priority-setting for healthcare: who, how much, and is it fair?" in Health Policy, Vol. 84, pp. 220-33.
47. Mitton, C. and Donaldson, C. (2001), "Twenty-five years of programme budgeting and marginal analysis in the health sector, 1974-1999" in Journal of Health Services Research & Policy, Vol. 6, pp. 239-48.
48. Mitton, C. and Donaldson, C. (2003), "Setting priorities and allocating resources in health regions: lessons from a project evaluating program budgeting and marginal analysis (PBMA)" in Health Policy, Vol. 64, pp. 335-48.
49. Mitton, C. and Donaldson, C. (2004), Priority Setting Toolkit: A Guide to the Use of Economics in Healthcare Decision Making, BMJ Books, London.
50. Mitton, C. and Prout, S. (2004), "Setting priorities in the south west of Western Australia: where are we now?" in Australian Health Review, Vol. 28, pp. 301-10.
51. Mitton, C., Smith, N., Peacock, S., Evoy, B. and Abelson, J. (2009), "Public participation in health care priority setting: a scoping review" in Health Policy, Vol. 91, pp. 219-28.
52. Mooney, G. (1998), "'Communitarian claims' as an ethical basis for allocating health care resources" in Social Science and Medicine, Vol. 47, pp. 1171-80.
53. Mooney, G. (2005), "Communitarian claims and community capabilities: furthering priority setting?" in Social Science and Medicine, Vol. 60, pp. 247-55.
54. Muller-Clemm, W.J. and Barnes, M.P. (1997), "A historical perspective on federal program evaluation in Canada" in Canadian Journal of Program Evaluation, Vol. 12, pp. 47-70.
55. Nevitte, N. (1996), The Decline of Deference: Canadian Value Change in Cross-national Perspective, University of Toronto Press, Toronto.
56. Noorani, H., Husereau, D.R., Boudreau, R. and Skidmore, B. (2007), "Priority setting for health technology assessments: a systematic review of current practical approaches" in International Journal of Technology Assessment in Health Care, Vol. 23, pp. 310-5.
57. Oberlander, J., Marmor, T. and Jacobs, L. (2001), "Rationing medical care: rhetoric and reality in the Oregon Health Plan" in Canadian Medical Association Journal, Vol. 164, pp. 1583-7.
58. Olsen, J.A. (1997), "Theories of justice and their implications for priority setting in health care" in Journal of Health Economics, Vol. 16, pp. 625-39.
59. Osborne, D. and Gaebler, T. (1992), Reinventing Government, Addison-Wesley, Reading, MA.
60. Pal, L. (2010), Beyond Policy Analysis: Public Issue Management in Turbulent Times, 4th ed., Nelson Education Limited, Toronto.
61. Patton, M.Q. (2007), "Process use as a usefulism" in New Directions for Evaluation, Vol. 116, pp. 99-112.
62. Patton, M.Q. (2008), Utilization-focused Evaluation, 4th ed., Sage, Thousand Oaks, CA.
63. Pawson, R. and Tilley, N. (1997), Realistic Evaluation, Sage, Thousand Oaks, CA.
64. Peacock, S. (1998), An Evaluation of Program Budgeting and Marginal Analysis Applied in South Australian Hospitals, Centre for Health Program Evaluation, Melbourne.
65. Peacock, S., Richardson, J., Carter, R. and Edwards, D. (2007), "Priority setting in health care using multi-attribute utility theory and program budgeting and marginal analysis PBMA" in Social Science and Medicine, Vol. 64, pp. 897-910.
66. Posner, P.L. and Fantone, D.M. (2007), "Assessing federal program performance: observations on the US Office of Management and Budget's program assessment rating tool and its use in the budget process" in Public Performance and Management Review, Vol. 30, pp. 351-68.
67. Rossi, P.H. and Freeman, H.E. (1993), Evaluation: A Systematic Approach, 5th ed., Sage, Newbury Park, CA.
68. Sabik, L.M. and Lie, R.K. (2008), "Priority setting in health care: lessons from the experiences of eight countries", International Journal for Equity in Health, Vol. 7.
69. Schick, A. (1973), "A death in the bureaucracy: the demise of federal PPB" in Public Administration Review, Vol. 33, pp. 146-56.
70. Shadish, W.R., Cook, T.D. and Leviton, L.C. (1991), Foundations of Program Evaluation: Theories of Practice, Sage, Newbury Park, CA.
71. Sibbald, S.L., Singer, P.A., Upshur, R. and Martin, D.K. (2009), "Priority setting: what constitutes success? A conceptual framework for successful priority setting", BMC Health Services Research, Vol. 9.
72. Sibbald, S.L., Gibson, J.L., Singer, P.A., Upshur, R. and Martin, D.K. (2010), "Evaluating priority setting success in healthcare: a pilot study", BMC Health Services Research, Vol. 10.
73. Smith, N., Mitton, C. and Peacock, S. (2009a), "Qualitative methodologies in health-care priority setting research" in Health Economics, Vol. 18, pp. 1163-75.
74. Smith, N., Mitton, C., Peacock, S., Cornelissen, E. and MacLeod, S. (2009b), "Identifying research priorities for health care priority setting: a collaborative effort between managers and researchers", BMC Health Services Research, Vol. 9.
75. Stake, R. (2000), "A modest commitment to the promotion of democracy" in New Directions for Evaluation, Vol. 85, pp. 97-106.
76. Stufflebeam, D. (2001), "Evaluation models" in New Directions for Evaluation, Vol. 89, pp. 7-98.
77. Syrett, K. (2002), "NICE work? Rationing, review and the 'legitimacy problem' in the new NHS" in Medical Law Review, Vol. 10, pp. 1-27.
78. Treasury Board of Canada (2009), "Policy on evaluation", available at: www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024 (accessed 2 November 2010).
79. Weiss, C.H. (1979), "The many meanings of research utilization" in Public Administration Review, Vol. 39, pp. 426-31.
80. Weiss, C.H. (1997), "How can theory-based evaluation make greater headway?" in Evaluation Review, Vol. 21, pp. 501-24.
81. Weiss, C.H. (1998), Evaluation, 2nd ed., Prentice Hall, Upper Saddle River, NJ.
82. Weiss, C.H. (2003), "On theory-based evaluation: winning friends and influencing people" in Evaluation Exchange, Vol. IX, No. 4, pp. 2-3.
83. West, W.F., Lindquist, E. and Mosher-Howe, K.N. (2009), "NOAA's resurrection of program budgeting: déjà vu all over again?" in Public Administration Review, Vol. 69, pp. 435-47.
84. Williams, I. and Bryan, S. (2007), "Understanding the limited impact of economic evaluation in health care resource allocation: a conceptual framework" in Health Policy, Vol. 80, pp. 135-43.
85. Williams, I., Bryan, S. and McIver, S. (2007), "How should cost-effectiveness analysis be used in health technology coverage decisions? Evidence from the National Institute for Health and Clinical Excellence approach" in Journal of Health Services Research and Policy, Vol. 12, pp. 73-9.
86. Wiseman, V. (2005), "Comparing the preferences of health professionals and members of the public for setting healthcare priorities: experiences from Australia" in Applied Health Economics and Health Policy, Vol. 4, pp. 129-37.