Performance measurement and improvement frameworks in health, education and social services systems: A systematic review

International Journal for Quality in Health Care

Volumn 22, Issue 1, 2009, Pages 44-69

  Klassen, Anne ^a   Miller, Anton ^b,c   Anderson, Nancy ^c   Shen, Jane ^c   Schiariti, Veronica ^b,c   O'Donnell, Maureen ^b,c  

^a MCMASTER UNIVERSITY   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Child; Chronic disease; Healthcare system; Quality indicators; Quality measurement; Special needs

Indexed keywords

CHRONIC DISEASE; CONCEPTUAL FRAMEWORK; EXERCISE; HEALTH CARE; HEALTH CARE QUALITY; HEALTH CARE SYSTEM; HEALTH EDUCATION; PERFORMANCE MEASUREMENT SYSTEM; PRIORITY JOURNAL; REVIEW; SOCIAL WORK; SYSTEMATIC REVIEW; TOTAL QUALITY MANAGEMENT;

EID: 77952254816     PISSN: 13534505     EISSN: 14643677     Source Type: Journal    
DOI: 10.1093/intqhc/mzp057     Document Type: Review

References (122)

1. Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press, 2001.
2. Zelman WN, Pink GH, Matthias CB. Use of the balanced scorecard in health care. J Health Care Finance 2003;29: 1-16.
3. Ryan GW, Bernard HR. Data management and analysis methods. In: Denzin N, Lincoln Y (eds). Handbook of Qualitative Research. 2nd edn. Thousand Oaks, CA: Sage Publications, 2000, 769-802.
4. Ryan GW, Bernard HR. Techniques to identify themes. Field Methods 2003;15:85-109.
5. Canadian Institute for Health Information (CIHI). Health Indicators 2007. Ottawa, ON: CIHI, 2007.
6. Institute of Medicine (IOM). Envisioning the National Health Care Quality Report. Washington, DC: National Academy Press, 2001.
7. Arah OA, Westert GP, Hurst J et al. A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care 2006;18:5-13.
8. Agency for Healthcare Research and Quality. National Healthcare Disparities Report. Background on the Measures Development Process. Rockville, MD: AHRQ, 2005.
9. Agency for Healthcare Research and Quality. Medicaid Home and Community-Based Services Measure Scan: Project Methodology. Rockville, MD: AHRQ, 2007.
10. Schoen C, Davis K, How SKH et al. Health system performance: a national scorecard. Health Affairs Web Exclusive 2006;e457-e475.
11. Leatherman S, McCarthy D. Quality of Health Care for Children and Adolescents: A Chartbook. New York, NY: The Commonwealth Fund, 2004.
12. National Association of Children's Hospitals and Related Institutions (NACHRI), Child Health Corporation of America (CHCA), Medical Management Planning Inc. (MMP). National Pediatric Practices & Measures: Focus on PICU. Alexandria, VA: NACHRI, CHCA, MMP, 2005.
13. Health Quality Council of Alberta. Alberta Quality Matrix for Health.http://www.hqca.ca/index.php?id=35 (31 January 2008, date last accessed).
14. Storey A. Performance management in schools: Could the balanced scorecard help? Sch Leader Manag 2002;22:321-38.
15. Huang SH, Chen PL, Yang MC et al. Using a balanced score-card to improve the performance of an emergency department. Nurs Econ 2004;22:140-6.
16. Thalman JJ, Malinowski TP, Thomas P. Balanced scoreboard for methodology for total performance improvement. Respir Care Clin N Am 2004;10:253-68.
17. Wachtel TL, Hartford CE, Hughes JA. Building a balanced scorecard for a burn center. Burns 1999;25:431-7.
18. Jones MLH, Filip SJ, Smith H et al. Strategy management system in perinatal services: the role of a patient resource manager. Lippincotts Case Manage 2002;7:27-45.
19. Chen XY, Yamauchi K, Kato K et al. Using the balanced scorecard to measure Chinese and Japanese hospital performance. Int J Health Care Qual Assur Inc Leaders Health Serv 2006;19:339-50.
20. Goodspeed SW. Metrics help rural hospitals achieve world-class performance. J Healthc Qual 2006;28:28-32, 55.
21. Yang MC, Tung YC. Using path analysis to examine causal relationships among balanced scorecard performance indicators for general hospitals: the case of a public hospital system in Taiwan. Health Care Manage Rev 2006;31:280-8.
22. Wells R, Weiner B. Using the balanced scorecard to characterize benefits of integration in the safety net. Health Serv Manage Res 2005;18:109-23.
23. Marr TJ, Mullen DM. Balanced scorecards for specialists: a tool for quality improvement. Minn Med 2004;87:46-50.
24. Macdonald M. Using the balanced scorecard to align strategy and performance in long-term care. Healthc Manage Forum 1998;11:33-8.
25. Radford A, Pink G, Ricketts T. A comparative performance scorecard for federally funded community health centers in North Carolina. J Healthc Manag 2007;52:20-31.
26. Harber BW. The balanced scorecard solution at Peel Memorial Hospital. Hosp Q 1998;1:59-61.
27. Devitt R, Klassen W, Martalog J. Strategic management system in a healthcare setting-moving from strategy to results. Healthc Q 2005;8:58-65.
28. Pieper SK. Reading the right signals: how to strategically manage with scorecards. Healthc Exec 2005;20:8-14.
29. Shortell SM, Schmittdiel J, Wang MC et al. An empirical assessment of high-performing medical groups: results from a national study. Med Care Res Rev 2005;62:407-34.
30. Robinson VA, Hunter D, Shortt SE. Accountability in public health units: using a modified nominal group technique to develop a balanced scorecard for performance measurement. Can J Public Health 2003;94:391-6.
31. Santiago JM. Use of the balanced scorecard to improve the quality of behavioral health care. Psychiatr Serv 1999;50:1571-6.
32. Peters DH, Noor AA, Singh LP et al. A balanced scorecard for health services in Afghanistan. Bull World Health Organ 2007;85:146-51.
33. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2007: Acute Care. Ottawa, ON: CIHI, 2007.
34. Goalen S, Soparlo L, Teare G. Whole system measures of quality. Paper presented at: 2007 Saskatchewan Association of Health Organizations (SAHO) Conference, 22 April 2007, Saskatoon, SK.
35. Pink GH, McKillop I, Schraa EG et al. Creating a balanced scorecard for a hospital system. J Health Care Finance 2001;27: 1-20.
36. ten Asbroek AH, Arah OA, Geelhoed J et al. Developing a national performance indicator framework for the Dutch health system. Int J Qual Health Care 2004;16:65-71.
37. Donnelley RR. Mental Health Project Final Report: National Benchmarking Project Report 2. Edinburgh, Scotland: Scottish Government, 2008.
38. Biro LA, Moreland ME, Cowgill DE. J Healthc Qual 2003; 25:33-9.
39. Inamdar SN, Kaplan RS, Jones ML et al. The Balanced Scorecard: a strategic management system for multi-sector collaboration and strategy implementation. Qual Manag Health Care 2000;8:21-39.
40. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2007: Emergency Department Care. Ottawa, ON: CIHI, 2007.
41. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2007: Complex Continuing Care. Ottawa, ON: CIHI, 2007.
42. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2007: Rehabilitation. Ottawa, ON: CIHI, 2007.
43. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2004: Mental Health. Toronto, ON: Hospital Report Research Collaborative, 2004.
44. Hospital Report Research Collaborative, Government of Ontario, Ontario Hospital Association, Canadian Institute for Health Information. Hospital Report 2001: Preliminary Studies-Volume Two Exploring Nursing & Population Health. Toronto, ON: Hospital Report Research Collaborative, 2001.
45. Kaplan RS, Norton DP. The balanced scorecard-measures that drive performance. Harv Bus Rev 1992;70: 71-9.
46. Donabedian A. Explorations in Quality Assessment and Monitoring. Vol. 1: The Definition of Quality and Approaches to Its Assessment. Chicago: Health Administration Press, 1982.
47. Hampshire AJ, Blair ME, Crown NS et al. Assessing the quality of preschool child health surveillance in primary care: a pilot study in one health district. Child Care Health Dev 2002;28:239-49.
48. Wakefield DS, Ward MM, Wakefield BJ. A 10-Rights frame-work for patient care quality and safety. Am J Med Qual 2007;22:103-11.
49. O'Meara P. A generic performance framework for ambulance services: an Australian health services perspective. J Emerg Prim Health Care 2005;3.
50. Campbell SM, Roland MO, Buetow SA. Defining quality of care. Soc Sci Med 2000;51:1611-25.
51. Munton AG, Mooney A, Rowland L. Deconstructing quality: a conceptual framework for the new paradigm in day care provision for the under eights. Early Child Dev Care 1995;114:11-23.
52. Derose SF, Schuster MA, Fielding JE et al. Public health quality measurement: concepts and challenges. Annu Rev Public Health 2002;23:1-21.
53. Handler A, Issel M, Turnock B. A conceptual framework to measure performance of the public health system. Am J Public Health 2001;91:1235-9.
54. Stewart AL, Teno J, Patrick DL et al. The concept of quality of life of dying persons in the context of health care. J Pain Symptom Manage 1999;17:93-108.
55. Al-Qutob R, Mawajdeh S, Bin Raad F. The assessment of reproductive health services: a conceptual framework for prenatal care. Health Care Women Int 1996;17:423-34.
56. Health Quality Council Saskatchewan, National Primary Care Development Team (UK). Quality Improvement Toolbook. http://www.hqc.sk.ca/download.jsp?KfoYhitywMAvRhvh6/qyPDBIzBf0QfLQkUwK4QBZaJuITevfxQg6jQ== (27 November 2008, date last accessed).
57. Department of Health, Government of Western Australia. Paving the Way: Promoting Safer Health Care in WA 2002-2007. East Perth: The Office of Safety and Quality in Health Care, Department of Health, Government of Western Australia, 2007.
58. Ministry of Health, New Zealand. Improving Quality (IQ): A Systems Approach for the New Zealand Health and Disability Sector. Wellington: Ministry of Health, New Zealand, 2003.
59. Baldrige National Quality Program. Health Care Criteria for Performance Excellence 2007. Gaithersburg, MD: Baldrige National Quality Program, National Institute of Standards and Technology (NIST), 2007.
60. Chang LC, Lin SW, Northcott DN. The NHS performance assessment framework: a 'balanced scorecard' approach? J Manag Med 2002;16:345-58.
61. Goh J, Singh B. The National Health Performance Framework: an evaluation model for use by psychiatrists. Australas Psychiatry 2005;13:111-5.
62. Goldstein SM, Schweikhart SB. Empirical support for the Baldrige Award framework in U.S. hospitals. Health Care Manage Rev 2002;27:62-75.
63. Health Services Utilization and Research Council. System Performance Indicators: Toward a Goal-Based Health System. Saskatoon, SK: Health Services Utilization and Research Commission, 2000.
64. Hermann RC, Leff HS, Lagodmos G. Selecting Process Measures for Quality Improvement in Mental Healthcare. Cambridge, MA: The Evaluation Center, HSRI, Centre for Quality Assessment & Improvement in Mental Health (CQAIMH), 2002.
65. McKenzie K, White D, Murray G et al. Quality here, there and everywhere: the application of a multi-dimensional learning tool to learning disability health services. Health Bull (Edinb) 1999;57:134-40.
66. Naar-King S, Siegel PT, Smyth M et al. A model for evaluating collaborative health care programs for children with special needs. Child Serv Soc Policy Res Pract 2000;3:233-45.
67. Nabitz U, Klazinga N, Walburg J. The EFQM excellence model: European and Dutch experiences with the EFQM approach in health care. Int J Qual Health Care 2000;12: 191-201.
68. Nabitz U, Schramade M, Schippers G. Evaluating treatment process redesign by applying the EFQM Excellence Model. Int J Qual Health Care 2006;18:336-45.
69. National Health Performance Committee (NHPC). National Health Performance Framework Report: A Report to the Australian Health Minister's Conference. Brisbane: Queensland Health, 2001.
70. National Health Service (UK). The New NHS Modern and Dependable: A National Framework for Assessing Performance-consultation Document. London: Department of Health, 1998.
71. NMHWG Information Strategy Committee Performance Indicator Drafting Group. Key Performance Indicators for Australian Public Mental Health Services. Canberra: Australian Government Department of Health and Ageing, 2005.
72. Turan JM, Bulut A, Nalbant H et al. The quality of hospital-based antenatal care in Istanbul. Stud Fam Plann 2006;37: 49-60.
73. Vallejo P, Saura RM, Sunol R et al. A proposed adaptation of the EFQM fundamental concepts of excellence to health care based on the PATH framework. Int J Qual Health Care 2006;18:327-35.
74. Weisman CS, Grason HA, Strobino DS. Quality management in public and community health: examples from women's health. Qual Manag Health Care 2001;10:54-64.
75. Department of Health, Government of Western Australia. Western Australian Strategic Plan for Safety and Quality in Health Care 2003-2008. East Perth: Office of Safety and Quality in Health Care, Department of Health, 2003.
76. Department of Health, Government of Western Australia. Western Australian Clinical Governance Guidelines. East Perth: The Office of Safety and Quality in Health Care, Department of Health, Government of Western Australia, 2005.
77. Department of Health, New South Wales Government. A Framework for Managing the Quality of Health Services in New South Wales. North Sydney: Department of Health, New South Wales Government, 2005.
78. General Practitioners Committee, British Medical Association. Quality and Outcomes Framework. London: British Medical Association, 2006.
79. Saskatchewan Health. Saskatchewan Framework for Health Service and Outcome Indicators. July/003 53M ed. Regina, SK: Saskatchewan Health, 2000.
80. Accreditation Canada (formerly Canadian Council on Health Services Accreditation). Special Edition on the New Accreditation Program. In Touch: A Newsletter for Surveyors, 2007, 1-5.
81. Bruce J. Fundamental elements of the quality of care: a simple framework. Stud Fam Plann 1990;21:61-91.
82. Clarke EB, Curtis JR, Luce JM et al. Quality indicators for end-of-live care in the intensive care unit. Crit Care Med 2003;31:2255-62.
83. Commission on Accreditation of Rehabilitation Facilities. CARF Standards for Performance Frameworks. http://www.carf.org/Providers.aspx?content=/content/publications/online/econnection/julaug05/build.htm (6 November 2008, date last accessed).
84. Dershin H. Dimensions of quality: a framework for quality management in healthcare. J Healthc Qual 1996;18:11-6.
85. Engels Y, Campbell S, Dautzenberg M et al. Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract 2005;22:215-22.
86. Evans DB, Edejer TT, Lauer J et al. Measuring quality: from the system to the provider. Int J Qual Health Care 2001;13: 439-46.
87. Glass AP. Nursing home quality: a framework for analysis. J Appl Gerontol 1991;10:5-18.
88. Greenberg A, Angus H, Sullivan T et al. Development of a set of strategy-based system-level cancer care performance indicators in Ontario, Canada. Int J Qual Health Care 2005;17:107-14.
89. Hermann RC, Palmer H, Leff S et al. Achieving consensus across diverse stakeholders on quality measures for mental healthcare. Med Care 2004;42:1246-53.
90. Hulton L, Matthews Z, Stones RW. A Framework for the Evaluation of Quality of Care in Maternity Services. Southampton: University of Southampton, 2000.
91. Kaatz T, Sargeant M, Kay R et al. Balancing the perfect scorecard. Inquiry as intervention in physician practice management. Med Group Manage J 2000;47:30-4.
92. Kizer KW. The 'new VA': a national laboratory for health care quality management. Am J Med Qual 1999;14:3-20.
93. Koch HCH. Implementing and Sustaining Total Quality Management in Health Care. Harlow, Essex: Longman, 1992.
94. Moscovice I, Christianson JB, Wellever A. Measuring and evaluating the performance of vertically integrated rural health networks. J Rural Health 1995;11:9-21.
95. Murray CJL, Frenk J. A framework for assessing the performance of health systems. Bull World Health Organ 2000;78:717-31.
96. Nabitz U, van den Brink W, Walburg J. A quality framework for addiction treatment programs. Addict Behav 2005;30: 1254-60.
97. Ontario Health Quality Council (OHQC). Reporting Framework: The Attributes of a High-Performing Health System. Toronto, ON: Ontario Health Quality Council, 2005.
98. Proctor S, Campbell J. A developmental performance frame-work for primary care. Int J Health Care Qual Assur Inc Leaders Health Serv 1999;12:279-86.
99. Reedy AM, Luna RG, Olivas GS et al. Local public health performance measurement: implementation strategies and lessons learned from aligning program evaluation indicators with the 10 essential public health services. J Public Health Manag Pract 2005;11:317-25.
100. Rosenbluth L, Morehead MA, Grossi M et al. A model for evaluating the delivery of pediatric primary care services. Qual Assur Util Rev 1991;6:2-7.
101. Rotondi AJ, Sirio CA, Angus DC et al. A new conceptual framework for ICU performance appraisal and improvement. J Crit Care 2002;17:16-28.
102. Smith G. Defining a quality strategy for learning disability services. Lifecare NHS Trust establishes its core values for care and quality. Int J Health Care Qual Assur 1995;8:14-8.
103. Tregunno D, Baker GR, Barnsley J et al. Competing values of emergency department performance: balancing multiple stakeholder perspectives. Health Serv Res 2004;39: 771-91.
104. Valentine NM. Quality measures essential to the transformation of the Veterans Health Administration: implications for nurses as co-creators of change. J Nurs Care Qual 2001; 15:48-59.
105. Veillard J, Champagne F, Klazinga N et al. A performance assessment framework for hospitals: The WHO regional office for Europe PATH project. Int J Qual Health Care 2005;17:487-96.
106. Standards Site, Department for Children, Schools and Families. National Quality Standards in Gifted and Talented Education. London: Department for Children, Schools and Families, 2007.
107. Baldrige National Quality Program. Education Criteria for Performance Excellence 2007. Gaithersburg, MD: Baldrige National Quality Program, National Institute of Standards and Technology (NIST), 2007.
108. Commission of the European Communities. Detailed Work Programme on the Follow-up of the Objectives of Education and Training Systems in Europe. Brussels: Commission of the European Communities, 2002.
109. Karathanos D, Karathanos P. Applying the balanced scorecard to education. J Educ Bus 2005;80:222-30.
110. National Institute for Urban School Improvement Communities. Systemic Change Framework. Rubrics Assessment Handbook: Rubrics for Urban Schools: Assessing Education for All. Denver, CO: National Institute for Urban School Improvement Communities, 2006.
111. Organisation for Economic Cooperation and Development. Education at a Glance 2007: OECD Indicators. Paris: OECD, 2007.
112. Maes B, Geeraert L, Van den Bruel B. Developing a model for quality evaluation in residential care for people with intellectual disability. Intellect Disabil Res 2000;44: 544-52.
113. Division of Mental Health, Developmental Disabilities and Substance Abuse Services, North Carolina Department of Health and Human Services. Mental Health, Developmental Disabilities and Substance Abuse Services Statewide System Performance Report. Raleigh, NC: North Carolina Department of Health and Human Services, 2008.
114. Centers for Medicare & Medicaid Services. HCBS Quality Framework. Baltimore, MD: Centers for Medicare & Medicaid Services, 2002.
115. DuPlessis HM, Inkelas M, Halfon N. Assessing the performance of community systems for children. Health Serv Res 1998;33:1111-42.
116. Lawthers AG, Pransky GS, Peterson LE et al. Rethinking quality in the context of persons with disability (see comment). Int J Qual Health Care 2003;15:287-99.
117. Wu AW, Johansen KS. Lessons from Europe on quality improvement: Report on the Velen Castle WHO Meeting. Jt Comm J Qual Improv 1999;25:16-29.
118. UK Department for Children,Schools and Families. Setting up Multi-agency services. http://www.dcsf.gov.uk/everychildmatters/strategy/deliveringservices1/multiagencyworking/multiagencyworking/ (14 September 2009, date last accessed).
119. Maxwell RJ. Dimensions of quality revisited: from thought to action. Qual Health Care 1992;1:171-7.
120. Donabedian A. The quality of care. How can it be assessed? Arch Pathol Lab Med 1997;121:1145-50.
121. Wagner EH, Glasgow RE, Davis C et al. Quality improvement in chronic illness care: a collaborative approach. Jt Comm J Qual Improv 2001;27:63-80.
122. Hroscikoski MC, Solberg LI, Sperl-Hillen JM et al. Challenges of change: a qualitative study of chronic care model implementation. Ann Fam Med 2006;4:317-26.