Methods to achieve high interrater reliability in data collection from primary care medical records

Annals of Family Medicine

Volumn 9, Issue 1, 2011, Pages 57-62

  Lidd, Clare ^a,b,c   Wiens, Miriam ^b,c   Hogg, William ^a,b,c,d  

^a UNIVERSITY OF OTTAWA   (Canada)

^b UNIVERSITY OF OTTAWA   (Canada)

^c BRUYÈRE RESEARCH INSTITUTE   (Canada)

^d Institute of Population Health   (Canada)

Author keywords

Chart abstraction; Primary care; Quantitative methods: Measurement issues instrument development

Indexed keywords

ARTICLE; DOCUMENTATION; HUMAN; INFORMATION PROCESSING; MEDICAL RECORD; OBSERVER VARIATION; PRIMARY HEALTH CARE; QUALITY CONTROL; REPRODUCIBILITY; SAMPLE SIZE; STANDARD; STATISTICS;

ABSTRACTING AND INDEXING AS TOPIC; DATA COLLECTION; HUMANS; MEDICAL RECORDS; OBSERVER VARIATION; PRIMARY HEALTH CARE; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS; SAMPLE SIZE;

EID: 78951481656     PISSN: 15441709     EISSN: 15441717     Source Type: Journal    
DOI: 10.1370/afm.1195     Document Type: Review

References (31)

1. Schoen C, Osborn R, Huynh P, Doty M, Peugh J, Zapert K. On the Front Lines of Care: Primary Care Doctors' Offi ce Systems, Experiences, and Views in Seven Countries. Washington, DC: The Commonwealth Fund; 2006.
2. Nagurney JT, Brown DF, Sane S, Weiner JB, Wang AC, Chang Y. The accuracy and completeness of data collected by prospective and retrospective methods. Acad Emerg Med. 2005;12(9):884-895.
3. Wu L, Ashton CM. Chart review. A need for reappraisal. Eval Health Prof. 1997;20(2):146-163.
4. Allison JJ, Wall TC, Spettell CM, et al. The art and science of chart review. Jt Comm J Qual Improv. 2000;26(3):115-136.
5. Cassidy LD, Marsh GM, Holleran MK, Ruhl LS. Methodology to improve data quality from chart review in the managed care setting. Am J Manag Care. 2002;8(9):787-793.
6. Eder C, Fullerton J, Benroth R, Lindsay SP. Pragmatic strategies that enhance the reliability of data abstracted from medical records. Appl Nurs Res. 2005;18(1):50-54.
7. Gearing RE, Mian IA, Barber J, Ickowicz A. A methodology for conducting retrospective chart review research in child and adolescent psychiatry. J Can Acad Child Adolesc Psychiatry. 2006;15(3):126-134.
8. Panacek EA. Performing chart review studies. Air Med J. 2007;26(5): 206-210.
9. Yawn BP, Wollan P. Interrater reliability: completing the methods description in medical records review studies. Am J Epidemiol. 2005; 161(10):974-977.
10. Lowenstein SR. Medical record reviews in emergency medicine: the blessing and the curse. Ann Emerg Med. 2005;45(4):452-455.
11. Worster A, Bledsoe RD, Cleve P, Fernandes CM, Upadhye S, Eva K. Reassessing the methods of medical record review studies in emergency medicine research. Ann Emerg Med. 2005;45(4):448-451.
12. Worster A, Haines T. Advanced statistics: understanding medical record review (MRR) studies. Acad Emerg Med. 2004;11(2):187-192.
13. Donner A, Eliasziw M. A goodness-of-fi t approach to inference procedures for the kappa statistic: confi dence interval construction, signifi cance-testing and sample size estimation. Stat Med. 1992;11(11):1511-1519.
14. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005; 85(3):257-268.
15. Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews in emergency medicine research: where are the methods? Ann Emerg Med. 1996;27(3):305-308.
16. Engel L, Henderson C, Fergenbaum J, Colantonio A. Medical Record Review Conduction Model for improving interrater reliability of abstracting medical-related information. Eval Health Prof. 2009;32(3):281-298.
17. To T, Estrabillo E, Wang C, Cicutto L. Examining intra-rater and inter-rater response agreement: a medical chart abstraction study of a community-based asthma care program. BMC Med Res Methodol. 2008;8:29.
18. Pan L, Fergusson D, Schweitzer I, Hebert PC. Ensuring high accuracy of data abstracted from patient charts: the use of a standardized medical record as a training tool. J Clin Epidemiol. 2005;58(9): 918-923.
19. Engel L, Henderson C, Colantonio A. Eleven steps to improving data collection: guidelines for a retrospective medical record review. Occupational Ther Now. 2008;10(1):17-20.
20. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.
21. Hogg W, Gyorfi-Dyke E, Johnston S, et al. Conducting chart audits in practice-based primary care research: a user's guide. Can Fam Physician. 2010;56(5):495-496.
22. Alberta Health and Wellness. Ambulatory Care Re-abstraction Study: Executive Report. Alberta, Canada: Alberta Health and Wellness; 2006.
23. Reisch LM, Fosse JS, Beverly K, et al. Training, quality assurance, and assessment of medical record abstraction in a multisite study. Am J Epidemiol. 2003;157(6):546-551.
24. Gwet K. Computing Inter-rater Reliability with the SAS System. Statistical Methods for Inter-rater Reliability Assessment [series]. Gaithersburg, MD: STATAXIS Consulting; 2002. Report 3.
25. Crewson PE. A correction for unbalanced kappa tables. In: Proceedings of the 26th Annual SAS Users Group International Conference. Cary, NC: SAS Institute Inc; 2001:1-3.
26. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. J Clin Epidemiol. 1993;46(5):423-429.
27. Norman GR, Streiner DL. Biostatistics: The Bare Essentials. 3rd ed. Hamilton, ON: B.C. Decker Inc; 2008.
28. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol. 1990;43(6):543-549.
29. Szklo M, Nieto FJ. Quality Assurance and Control. Epidemiology Beyond the Basics. Sudbury, MA: Jones and Bartlett; 2004:343-404.
30. Stange KC, Zyzanski SJ, Smith TF, et al. How valid are medical records and patient questionnaires for physician profi ling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36(6):851-867.
31. Gow RM, Barrowman NJ, Lai L, Moher D. A review of fi ve cardiology journals found that observer variability of measured variables was infrequently reported. J Clin Epidemiol. 2008;61(4):394-401.