Measuring physicians' performance in clinical practice: Reliability, classification accuracy, and validity

Evaluation and the Health Professions

Volumn 33, Issue 3, 2010, Pages 302-320

  Weng, Weifeng ^a   Hess, Brian J ^a   Lynn, Lorna A ^a   Holmboe, Eric S ^a   Lipner, Rebecca S ^a  

^a AMERICAN BOARD OF INTERNAL MEDICINE   (United States)

Author keywords

classification accuracy; physician performance assessment; reliability of composite measures; reproducibility; validity

Indexed keywords

LOW DENSITY LIPOPROTEIN CHOLESTEROL;

ADULT; AGED; ARTICLE; CLINICAL COMPETENCE; CONFIDENCE INTERVAL; FEASIBILITY STUDY; FEMALE; HEALTH CARE QUALITY; HUMAN; INTERNAL MEDICINE; MALE; MIDDLE AGED; NONPARAMETRIC TEST; ORGANIZATION AND MANAGEMENT; PRODUCTIVITY; PSYCHOMETRY; REPRODUCIBILITY; STANDARD; STATISTICS; TASK PERFORMANCE; VALIDATION STUDY;

ADULT; AGED; CHOLESTEROL, LDL; CLINICAL COMPETENCE; CONFIDENCE INTERVALS; EFFICIENCY; EFFICIENCY, ORGANIZATIONAL; FEASIBILITY STUDIES; FEMALE; HUMANS; INTERNAL MEDICINE; MALE; MIDDLE AGED; PSYCHOMETRICS; QUALITY OF HEALTH CARE; REPRODUCIBILITY OF RESULTS; STATISTICS AS TOPIC; STATISTICS, NONPARAMETRIC; TASK PERFORMANCE AND ANALYSIS; YOUNG ADULT;

EID: 77956082612     PISSN: 01632787     EISSN: 15523918     Source Type: Journal    
DOI: 10.1177/0163278710376400     Document Type: Article

References (30)

1. American Board of Internal Medicine. (n.d.). ABIM HIPAA business associate agreement. Retrieved March 3, 2010, from http://www.abim.org/pdf/hipaa/hipaa_compliance.pdf.
2. American Diabetes Association. (2009). Standards of medical care in diabetes- 2009. Diabetes Care, 32, S13-S61.
3. Bridges to Excellence. (2009). Diabetes care link. Retrieved March 3, 2010, from http://bridgestoexcellence.org/Documents/DCL%20Measures.pdf.
4. Brennan, T.A., Horwitz, R.I., Duffy, F.D., Cassel, C.K., Goode, L.D., & Lipner, R.S. (2004). The role of physician specialty board certification status in the quality movement. Journal of the American Medical Association, 292, 1038-1043.
5. Doran, T., Fullwood, C., Reeves, D., Gravelle, H., & Roland, M. (2008). Exclusion of patients from pay-for-performance targets by English physicians. New England Journal of Medicine, 359, 274-284.
6. Duffy, F.D., Lynn, L.A., Didura, H., Hess, B., Caverzagie, K., Grosso, L., &... Holmboe, E.S. (2008). Self-assessment of practice performance: Development of the ABIM Practice Improvement Module (PIMSM). Journal of Continuing Education in the Health Professions, 28, 38-46.
7. Dunn-Rankin, P. (1983). Scaling methods. Hillsdale, NY: Lawrence Erlbaum.
8. Epstein, A.M. (2007). Pay for performance at the tipping point. New England Journal of Medicine, 356, 515-517.
9. Greenfield, S., Kaplan, S.H., Kahn, R., Ninomiya, J., & Griffith, J.L. (2002). Profiling care provided by different groups of physicians: Effects of patient case-mix (bias) and physician-level clustering on quality assessment results. Annals of Internal Medicine, 136, 111.
10. Hess, B.J., Weng, W., Lynn, L.A., Holmboe, E.S., & Lipner, R.S. (in press). Setting a fair performance standard for physicians' quality of patient care. Manuscript submitted for publication.
11. Holmboe, E.S., Wang, Y., Meehan, T.P., Tate, J.P., Ho, S.-Y., Starkey, K.S., & Lipner, R.S. (2008). Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries. Archives of Internal Medicine, 168, 1396-1403.
12. Holmboe, E.S., Wang, Y., Tate, J.P., & Meehan, T.P. (2006). The effects of patient volume on the quality of diabetic care for Medicare beneficiaries. Medical care, 44, 1073-1077.
13. Kaplan, S., Griffith, J., Price, L., Pawlson, L., & Greenfield, S. (2009). Improving the reliability of physician performance assessment: Identifying the "physician effect" on quality and creating composite measures. Medical Care, 47, 378-387.
14. Kerr, E.A., Hofer, T.P., Hayward, R.A., Adams, J.L., Hogan, M.M., McGlynn, E.A., & Asch, S.M. (2007). Quality by any other name?: A comparison of three profiling systems for assessing health care quality. Health services research, 42, 2070-2087.
15. Landon, B.E., & Normand, S.-L.T. (2008). Performance measurement in the small office practice: Challenges and potential solutions. Annals of Internal Medicine, 148, 353-377.
16. Landon, B.E., Normand, S.-L.T., Blumenthal, D., & Daley, J. (2003). Physician clinical performance assessment: Prospects and barriers. Journal of the American Medical Association, 290, 1183-1189.
17. Lee, W.-C., Brennan, R.L., & Wan, L. (2009). Classification consistency and accuracy for complex assessments under the compound multinomial model. Applied Psychological Measurement, 33, 374-390.
18. Lipner, R.S., Weng, W., Arnold, G.K., Duffy, F.D., Lynn, L.A., & Holmboe, E.S. (2007). A three-part model for measuring diabetes care in physician practice. Academic Medicine, 82, S48-S52.
19. McGlynn, E.A., Asch, S.M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., & Kerr, E.A. (2003). The quality of health care delivered to adults in the United States. New England Journal of Medicine, 348, 2635-2645.
20. Miller, T.P., Brennan, T.A., & Milstein, A. (2009). How can we make more progress in measuring physicians' performance to improve the value of care? Health Affairs, 28, 1429-1437.
21. Mosier, C. (1943). On the reliability of a weighted composite. Psychometrika, 8, 161-168.
22. Norcini, J., Lipner, R., & Kimball, H. (2001). The certification status of generalist physicians and the mortality of their patients after acute myocardial infarction. Academic Medicine, 76, S21-S23.
23. Norcini, J.J., Holmboe, E.S., & Hawkins, R. (2008). Evaluation challenges in the era of the outcomes-based education. In E. S. Holmboe & R. Hawkins (Eds.), A practical approach to the evaluation of clinical competence (pp. 1-9). Philadelphia, PA: Mosby-Elsevier.
24. O'Kane, M.E. (2007). Performance-based measures: The early results are in. Journal of Managed Care Pharmacy, 13, S3-S6.
25. Pham, H.H., Schrag, D., Hargraves, J.L., & Bach, P.B. (2005). Delivery of preventive services to older adults by primary care physicians. Journal of the American Medical Association, 294, 473-481.
26. Reeves, D., Campbell, S.M., Adams, J., Shekelle, P.G., Kontopantelis, E., & Roland, M.O. (2007). Combining multiple indicators of clinical quality: An evaluation of different analytic approaches. Medical Care, 45, 489-496.
27. Rittenhouse, D.R., & Shortell, S.M. (2009). The patient-centered medical home. Journal of the American Medical Association, 301, 2038-2040.
28. Rosenthal, M.B. (2007). Pay-for-performance: Will the latest payment trend improve care? Journal of the American Medical Association, 297, 740-744.
29. Scholle, S.H., Roski, J., Adams, J.L., Dunn, D.L., Kerr, E.A., Dugan, D.P., & Jensen, R.E. (2008). Benchmarking physician performance: Reliability of individual and composite measures. American Journal of Managed Care, 14, 829-838.
30. Tamblyn, R., Abrahamowicz, M., Dauphinee, W.D., Hanley, J.A., Norcini, J., Girard, N., &... Brailovsky, C. (2002). Association between licensure examination scores and practice in primary care. The Journal of the American Medical Association, 288, 3019-3026.