Research methods used in developing and applying quality indicators in primary care

Quality and Safety in Health Care

Volumn 11, Issue 4, 2002, Pages 358-364

  Campbell, Stephen M ^a   Braspenning, J ^c   Hutchinson, A ^b   Marshall, M ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

^c RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; EUROPE; EVIDENCE BASED MEDICINE; HEALTH CARE QUALITY; MEDICAL DECISION MAKING; MEDICAL RESEARCH; PRACTICE GUIDELINE; PRIMARY MEDICAL CARE;

EID: 0036901660     PISSN: 09638172     EISSN: None     Source Type: Journal    
DOI: 10.1136/qhc.11.4.358     Document Type: Article

References (112)

1. Forrest D, Hoskins A, Hussey. Clinical guidelines and their implementation. Postgrad Med J 1996;72:19-22.
2. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet 1993;342:1317-22.
3. Donabedian A. Explorations in Quality Assessment and Monitoring. Volume 1: The Definition of Quality and Approaches to its Assessment. Ann Arbor, Michigan: Health Administration Press, 1980.
4. Irvine D. Managing for Quality in General Practice. London: King's Fund Centre, 1990.
5. Juran JM. Juran on Planning for Quality. New York: Free Press, 1988.
6. McGlynn EA, Asch SM. Developing a clinical performance measure. Am J Prevent Med 1998;14:14-21.
7. Department of Health. A National Service Framework for Coronary Heart Disease. London: Department of Health, 2000.
8. Seddon ME, Marshall MN, Campbell SM, et al. Systematic review of studies of clinical care in general practice in the United Kingdom, Australia and New Zealand. Qual Health Care 2001;10:152-8.
9. Lawrence M, Olesen F, et al. Indicators of quality health care. Eur J Gen Pract 1997;3:103-8.
10. Marshall M, Campbell SM. Introduction to quality assessment in general practice. In: Marshall M, Campbell SM, Hacker J, Roland MO, eds. Quality Indicators for General Practice: A Practical Guide for Health Professionals and Managers. London: Royal Society of Medicine, 2002:1-6.
11. Buck D, Godfrey C, Morgan A. Performance Indicators and Health Promotion Targets. Discussion paper 150. York: Centre for Health Economics, University of York, 1996.
12. Ovretveit J. Health Service Quality: An Introduction to Quality Methods for Health Services. Oxford: Blackwell Scientific Publications, 1992.
13. McGlynn EA. Six challenges in measuring the quality of health care. Health Aff 1997;16:7-21.
14. Joss R, Kogan M. Advancing Quality: Total Quality Management in the National Health Service. Buckingham: Open University Press, 1995.
15. Davies HTO, Crombie IK. Assessing the quality of care. BMJ 1995;311:766.
16. Eddy DM. Performance measurement: Problems and solutions. Health Aff 1998;17:7-26.
17. Mant J, Hicks N. Detecting differences in quality of care: The sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ 1995;311:793-6.
18. Palmer RH. Process-based measures of quality: The need for detailed clinical data in large health care databases. Ann Intern Med 1997;127:733-8.
19. Tarlov AR, Ware JE, Greenfield S, et al. The Medical Outcomes Study: An application of methods for monitoring the results of medical care. JAMA 1989;262:925-30.
20. McGlynn EA. The outcomes utility index: Will outcomes data tell us what we want to know? Int J Qual Health Care 1998;10:485-90.
21. Agency for Healthcare Research and Quality. Computerized Needs: Oriented Quality Measurement Evaluation System. Rockville: Agency for Healthcare Research and Quality, 1999 (www.ahrq.gov/qual/conqix.htm).
22. NHS Executive. Quality and Performance in the NHS: High Level Performance Indicators. London: Department of Health, 1999.
23. Roland MO, Holden J, Campbell SM. Quality Assessment for General Practice: Supporting Clinical Governance in Primary Care Groups. Manchester: National Primary Care Research and Development Centre, 1998.
24. Wensing M, Jung HP, Mainz J, et al. A systematic review of the literature on patient priorities for general practice care. Part 1: Description of the research domain. Soc Sci Med 1998;47:1573-88.
25. Campbell SM, Hann M, Hacker J, et al. Identifying predictors of high quality care in English general practice: An observational study. BMJ 2001;323:784-7.
26. Ramsay J, Campbell JL, Schroter S, et al. The General Practice Assessment Survey (GPAS): Tests of data quality and measurement properties. Fam Pract 2000;17:372-9.
27. Wensing M, Vedsted P, Kersnik J, et al. Patient satisfaction with availability of general practice: An international comparison. Int J Qual Health Care 2002;14:111-8.
28. National Committee for Quality Assurance. Narrative: What's in it and Why it Matters. Volume 1. HEDIS 3.0/1998. Washington: National Committee for Quality Assurance, 1998.
29. Van den Hombergh P, Grol R, van den Hoogen HJ, et al. Practice visits as a tool in quality improvement: Mutual visits and feedback by peers compared with visits and feedback by non-physician observers. Qual Health Care 1999;8:161-6.
30. American Academy of Family Physicians. The Family Practice Management Practice Self-Test. 2001 (available at: www.aafp.org/fpm/20010200/41thef.html)
31. Royal Australian College of General Practitioners. Standards for General Practice. Royal Australian College of General Practitioners, 2000.
32. Brook RH, McGlynn EA, Cleary PD. Measuring quality of care. N Engl J Med 1996;335:966-70.
33. Howie JG, Heaney DJ, Maxwell M. Measuring quality in general practice. Pilot study of a needs, process and outcome measure. Occasional Paper of the Royal College of General Practitioners 1997;75:1-32.
34. Wilson A, Childs S. Systematic Review on Consultation Length in General Practice. A report to the Scientific Foundation Board of the RCGP, University of Leicester, Leicester, 2001.
35. Campbell SM, Roland MO, Shekelle PG, et al. Development of review criteria for assessing the quality of management of stable angina, adult asthma and non-insulin dependent diabetes in general practice. Qual Health Care 1999;8:6-15.
36. Brook RH, McGlynn EA, Shekelle PG. Defining and measuring quality of care: A perspective from US researchers. Int J Qual Health Care 2000;12:281-95.
37. Pringle M. Preventing ischaemic heart disease in one general practice: From one patient, through clinical audit, needs assessment, and commissioning into quality improvement. BMJ 1998;317:1120-4.
38. Pringle M. Clinical governance in primary care. Participating in clinical governance. BMJ 2000;321:737-40.
39. NHS Executive. Quality and Performance in the NHS: High Level Performance Indicators. London: Department of Health, 1999.
40. Hearnshaw HM, Harker RM, Cheater FM, et al. Expert consensus on the desirable characteristics of review criteria for improvement of health quality. Qual Health Care 2001;10:173-8.
41. Campbell SM, Cantrill JA. Consensus methods in prescribing research. J Clin Pharm Ther 2001;26:5-14.
42. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess 1998;2(3).
43. Baker R, Fraser RC. Is ownership more important than the scientific credibility of audit protocols? A survey of medical audit advisory groups. Fam Pract 1997;14:107-11.
44. McColl A, Roderick P, Gabbay J, et al. Performance indicators for primary care groups: An evidence-based approach. BMJ 1998;317:1354-60.
45. Naylor CD. Grey zones in clinical practice: Some limits to evidence based medicine. Lancet 1995;345:840-2.
46. Black N, Murphy M, Lamping D, et al. Consensus development methods: A review of best practice in creating clinical guidelines. J Health Serv Res Policy 1999;4:236-48.
47. Jones JJ, Hunter D. Consensus methods for medical and health services research. BMJ 1995;311:376-80.
48. Shekelle PG, Kahan JP, Bernstein SJ, et al. The reproducibility of a method to identify the overuse and underuse of procedures. N Eng J Med 1998;338:1888-95.
49. Campbell SM, Roland MO, Buetow S. Defining quality of care. Soc Sci Med 2000;51:1611-25.
50. Ibrahim JE. Performance indicators form all perspectives. Int J Qual Health Care 2001;13:431-2.
51. Linstone HA, Turoff M. The Delphi Survey. Method, Techniques and Applications. Reading, Massachusetts: Addison-Wesley, 1975.
52. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Advan Nurs 2000;32:1008-15.
53. Fink A, Kosecoff J, Chassin M, et al. Consensus methods: Characteristics and guidelines for use. Am J Public Health 1984;74:979-83.
54. Brook RH, Chassin MR, Fink A, et al. A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care 1986;2:53-63.
55. Braspenning J, Drijver R, Schiere AM. Quality Indicators for General Practice (in Dutch). Nijmegen/Utrecht: Centre for Quality of Care Research/Dutch College of General Practitioners, 2001.
56. Delbecq AL, Van de Ven AH, Gustafson D. Group Techniques for Programme Planning: A Guide to Nominal Group and Delphi Processes. Glenview, Illinois: Scott, Foresman & Company, 1975.
57. Department of Health. Treatment Choice in the Psychological Therapies and Counselling. London: Department of Health, 2001.
58. Cantrill JA, Sibbald B, Buetow S. Indicators of the appropriateness of long term prescribing in general practice in the United Kingdom: Consensus development, face and content validity, feasibility and reliability. Qual Health Care 1998;7:130-5.
59. Shield TL. Quality Indicators for Mental Health Care in Primary Care. Personal correspondence, NPCRDC, Manchester, 2002.
60. Rockwell MA. The Delphi procedure: Knowledge from goat viscera? N Engl J Med 1973;288:1298-9.
61. Campbell SM, Cantrill JA, Richards D. Prescribing indicators for UK general practice: Delphi consultation study. BMJ 2000;321:1-5.
62. Campbell SM, Roland MO, Quayle JA, et al. Quality indicators for general practice. Which ones can general practitioners and health authority managers agree are important and how useful are they? J Public Health Med 1998;20:414-21.
63. Normand SL, McNeil BJ, Peterson LE, et al. Eliciting expert opinion using the Delphi technique: Identifying performance indicators for cardiovascular disease. Int J Qual Health Care 1998;10:247-60.
64. Munro N, Hornung RI, McAteer S. What are the key attributes to of a good general practice trainer? A Delphi study. Educ Gen Pract 1998;9:263-70.
65. Naylor CD. What is appropriate care? N Engl J Med 1998;338:1918-20.
66. Hicks NR. Some observations on attempts to measure appropriateness of care. BMJ 1994;309:730-3.
67. Ayanian JZ, Landrum MB, Normand SLT, et al. Rating the appropriateness of coronary angiography - Do practicing physicians agree with an expert panel and with each other? N Engl J Med 1998;338:1896-904.
68. Campbell SM. Defining, Measuring and Assessing Quality of Care in General Practice. PhD Thesis, University of Manchester, Manchester, 2002.
69. Leape LL, Hilborne LH, Schwartz JS, et al. The appropriateness of coronary artery bypass graft surgery in academic medical centres. Ann Intern Med 1996;125:8-18.
70. Kahn KL, Rogers WH, Rubenstein LV, et al. Measuring quality of care with explicit process criteria before and after implementation of a DRG-based prospective payment system. JAMA 1990;264:1969-73.
71. Shekelle PG, MacLean CH, Morton SC, et al. Assessing care of vulnerable elders: Methods for developing quality indicators. Ann Intern Med 2001;135:647-52.
72. Gray D, Hampton JR, Bernstein SJ, et al. Audit of coronary angiography and bypass surgery. Lancet 1990;335:1317-20.
73. Scott EA, Black N. Appropriateness of cholecystectomy in the United Kingdom: A consensus panel approach. Gut 1991;32:1066-70.
74. Hemingway H, Crook AM, Feder G, et al. Underuse of coronary revascularization procedures in patients considered appropriate candidates for revascularization. N Engl J Med 2001;344:645-54.
75. Campbell SM, Hann M, Hacker J, et al. Quality assessment for three common conditions in primary care: Validity and reliability of review criteria developed by expert panels for angina, asthma and type 2 diabetes. Qual Saf Health Care 2002;11:125-30.
76. Shekelle PG, Kahan JP, Park RE, et al. Assessing appropriateness by expert panels: How reliable? J Gen Intern Med 1996;10:81.
77. Eccles M, Clapp Z, Grimshaw J, et al. North of England evidence based guidelines development project: Methods of guideline development. BMJ 1996;312:760-2.
78. Agency for Healthcare Research and Quality. Using Clinical Practice Guidelines to Evaluate Quality of Care. Volume 2, Methods. Rockville: Agency for Healthcare Research and Quality, 1995.
79. CBO Quality Institute for Health Care. Handbook. Development of Indicators on Vidence-based Guidelines (in Dutch). Utrecht: Quality Institute for Health Care, 2002.
80. Hutchinson A, Anderson JP, McIntosh A, et al. Evidence Based Review Criteria for Coronary Heart Disease. Sheffield: Royal College of General Practitioners Effective Clinical Practice Unit, University of Sheffield, 2000.
81. Hadorn DC, Baker DW, Kamberg CJ, et al. Phase II of the AHCPR-sponsored heart failure guideline: Translating practice recommendations into review criteria. J Qual Improve 1996;22:266-75.
82. National Institute for Clinical Excellence. Management of Type 2 Diabetes: Renal Disease-Prevention and Early Management. London: National Institute for Clinical Excellence, 2002.
83. Smeele IJ, Grol RP, van Schayck CP, et al. Can small group education and peer review improve care for patients with asthma/chronic obstructive pulmonary disease? Qual Health Care 1999;8:92-8.
84. Grol R, Dalhuijsen J, Thomas S, et al. Attributes of clinical guidelines that influence use of guidelines in genral practice: Observational study. BMJ 1998;317:858-61.
85. Spies TH, Mokkink HGA. Using Guidelines in Clinical Practice (in Dutch). Nijmegen/Utrecht: Centre for Quality of Care Research/Dutch College of General Practitioners, 1999.
86. Schers H, Braspenning J, Drijver R, et al. Low back pain in general practice: Reported manegement and reasons for not adhering to the guidelines in the Netherlands. Br J Gen Pract 2000;50:640-4.
87. Frijling BD, Spies TH, Lobo CM, et al. Blood pressure control in treated hypertensive patients: Clinical performance of general practitioners. Br J Gen Pract 2001;51:9-14.
88. Campbell SM, Hann M, Roland MO, et al. The effect of panel membership and feedback on ratings in a two-round Delphi survey. Med Care 1999;37:964-8.
89. Coulter I, Adams A, Shekelle P. Impact of varying panel membership on ratings of appropriateness in consensus panels: A composition of a multi- and single disciplinary panel. Health Serv Res 1995;30:577-91.
90. Fraser GM, Pilpel D, Kosecoff J, et al. Effect of panel composition on appropriateness ratings. Int J Qual Health Care 1994;6:251-5.
91. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and Use. Oxford: Oxford Medical Publications, 1995.
92. Huff ED. Comprehensive reliability assessment and comparison of quality indicators and their components. J Clin Epidemiol 1997;50:1395-404.
93. Siu AL, McGlynn EA, Morgenstern H, et al. Choosing quality of care measures based on the expected impact of improved care on health. Health Serv Res 1992;27:619-50.
94. Thomson R, Lally J. Clinical indicators: Do we know what we're doing? Qual Health Care 1998;7:122.
95. Enthoven AC. A promising start, bur fundamental reform is needed. BMJ 2000;320:1329-21.
96. Wilkinson EK, McColl A, Exworthy M, et al. Reactions to the use of evidence-based performance indicators in primary care: A qualitative study. Qual Health Care 2000;9:166-74.
97. Baker R. Managing quality in primary health care: The need for valid information about performance. Qual Health Care 2000;9:83.
98. Craddock J, Young A, Sullivan G. The accuracy of medical record documentation in schizophrenia. J Behav Health Serv Res 2001;28:456-66.
99. Wyatt JC, Wright P. Design should help use of patients' data. Lancet 1998;352:1375-8.
100. Wu L, Ashton CM. Chart review: A need for reappraisal. Evaluating Health Professionals 1997;20:146-63.
101. Luck J, Peabody JW, Dresselhaus TR, et al. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med 2000;108:642-9.
102. Salzer MS, Nixon CT, Schut LJA, et al. Validating quality indicators: Quality as relationship between structure, process and outcome. Evaluation Rev 1997;21:292-309.
103. Kahn KL, Park RE, Vennes J, et al. Assigning appropriateness ratings for diagnostic upper gastrointestinal endoscopy using two different approaches. Med Care 1992, 30:1016-28.
104. Phelps CE. The methodologic foundations of studies of the appropriateness of medical care. N Engl J Med 1993;329:1241-5.
105. Merrick NJ, Fink A, Park RE, et al. Derivation of clinical indications for carotid endarterectomy by an expert panel. Am J Public Health 1987;77:187-90.
106. Kravitz RL, Laouri M, Kahan JP, et al. Validity of criteria used for detecting underuse of coronary revascularization. JAMA 1995;274:632-8.
107. Selby JV, Fireman BH, Lundstrom. Variation among hospitals in coronary-angiography practices and outcomes after myocardial infarction in a large health maintenance organisation. N Engl J Med 1996;335:1888-96.
108. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: A framework for change. Milbank Quarterly 2001;79:281-315.
109. Campbell SM, Sweeney GM. The role of clinical governance as a strategy for quality improvement in primary care. Br J Gen Pract 2002 (in press).
110. Donabedian A. Explorations in Quality Assessment and Monitoring. Volume 2. The Criteria and Standards of Quality. Ann Arbor, Michigan: Health Administration Press, 1982.
111. Donabedian A. The quality of medical care. Science 1978;2000:856-64.
112. Lomas J, Anderson G, Enkin M, et al. The role of evidence in the consensus process. Results from a Canadian consensus exercise. JAMA 1988;259:3001-5.