Methods and dimensions of electronic health record data quality assessment: Enabling reuse for clinical research

Journal of the American Medical Informatics Association

Volumn 20, Issue 1, 2013, Pages 144-151

  Weiskopf, Nicole Gray ^a   Weng, Chunhua ^a  

^a Och Spine at New York Presbyterian Hospitals   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CLINICAL RESEARCH; DATA PROCESSING; ELECTRONIC MEDICAL RECORD; MEDICAL RECORD REVIEW; METHODOLOGY; QUALITY CONTROL; VALIDITY;

DATA COLLECTION; ELECTRONIC HEALTH RECORDS; HUMANS; INFORMATION DISSEMINATION; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 84871882786     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1136/amiajnl-2011-000681     Document Type: Article

References (110)

1. Safran C, Bloomrosen M, Hammond WE, et al. Toward a national framework for the secondary use of health data: an American Medical Informatics Association White Paper. J Am Med Inform Assoc 2007;14: 1-9.
2. Hersh WR. Adding value to the electronic health record through secondary use of data for quality assurance, research, and surveillance. Am J Manag Care 2007;13: 277-8.
3. National Center for Research Resources (US). Widening the Use of Electronic Health Record Data for Research. 2009. http://videocast.nih.gov/summary.asp?live=8062 (accessed 9 Oct 2011).
4. Weiner MG, Embi PJ. Toward reuse of clinical data for research and quality improvement: the end of the beginning? Ann Intern Med 2009;151: 359-60.
5. Burnum JF. The misinformation era: the fall of the medical record. Ann Intern Med 1989;110: 482-4.
6. van der Lei J. Use and abuse of computer-stored medical records. Methods Inf Med 1991;30: 79-80.
7. Juran JM, Gryna FM. Juran's Quality Control Handbook. 4th edn. New York: McGraw-Hill, 1988.
8. Hogan WR, Wagner MM. Accuracy of data in computer-based patient records. J Am Med Inform Assoc 1997;4: 342-55.
9. Thiru K, Hassey A, Sullivan F. Systematic review of scope and quality of electronic patient record data in primary care. BMJ 2003;326: 1070.
10. Chan KS, Fowles JB, Weiner JP. Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev 2010;67: 503-27.
11. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2005;27: 237-46.
12. Ayoub L, Fu L, Pena A, et al. Implementation of a data management program in a pediatric cancer unit in a low income country. Pediatr Blood Cancer 2007;49: 23-7.
13. Barrie JL, Marsh DR. Quality of data in the Manchester orthopaedic database. BMJ 1992;304: 159-62.
14. Hohnloser JH, Fischer MR, Konig A, et al. Data quality in computerized patient records. Analysis of a haematology biopsy report database. Int J Clin Monit Comput 1994;11: 233-40.
15. Mikkelsen G, Aasly J. Consequences of impaired data quality on information retrieval in electronic patient records. Int J Med Inform 2005;74: 387-94.
16. Ricketts D, Newey M, Patterson M, et al. Markers of data quality in computer audit: the Manchester Orthopaedic Database. Ann R Coll Surg Engl 1993;75: 393-6.
17. Roukema J, Los RK, Bleeker SE, et al. Paper versus computer: feasibility of an electronic medical record in general pediatrics. Pediatrics 2006;117: 15-21.
18. Wallace CJ, Stansfield D, Gibb Ellis KA, et al. Implementation of an electronic logbook for intensive care units. Proc AMIA Symp 2002: 840-4.
19. Kaboli PJ, McClimon BJ, Hoth AB, et al. Assessing the accuracy of computerized medication histories. Am J Manag Care 2004;10: 872-7.
20. Porter SC, Mandl KD. Data quality and the electronic medical record: a role for direct parental data entry. Proc AMIA Symp 1999: 354-8.
21. Whitelaw FG, Nevin SL, Milne RM, et al. Completeness and accuracy of morbidity and repeat prescribing records held on general practice computers in Scotland. Br J Gen Pract 1996;46: 181-6.
22. Pyper C, Amery J, Watson M, et al. Patients' experiences when accessing their on-line electronic patient records in primary care. Br J Gen Pract 2004;54: 38-43.
23. Staroselsky M, Volk LA, Tsurikova R, et al. An effort to improve electronic health record medication list accuracy between visits: patients' and physicians' response. Int J Med Inform 2008;77: 153-60.
24. Staroselsky M, Volk LA, Tsurikova R, et al. Improving electronic health record (EHR) accuracy and increasing compliance with health maintenance clinical guidelines through patient access and input. Int J Med Inform 2006;75: 693-700.
25. Weingart SN, Cleary A, Seger A, et al. Medication reconciliation in ambulatory oncology. Jt Comm J Qual Patient Saf 2007;33: 750-7.
26. Logan JR, Gorman PN, Middleton B. Measuring the quality of medical records: a method for comparing completeness and correctness of clinical encounter data. Proc AMIA Symp 2001: 408-12.
27. Smith PC, Araya-Guerra R, Bublitz C, et al. Missing clinical information during primary care visits. JAMA 2005;293: 565-71.
28. Bentsen BG. The accuracy of recording patient problems in family practice. J Med Educ 1976;51: 311-16.
29. Berner ES, Kasiraman RK, Yu F, et al. Data quality in the outpatient setting: impact on clinical decision support systems. AMIA Annu Symp Proc 2005: 41-5.
30. Peabody JW, Luck J, Jain S, et al. Assessing the accuracy of administrative data in health information systems. Med Care 2004;42: 1066-72.
31. Lewis JD, Brensinger C. Agreement between GPRD smoking data: a survey of general practitioners and a population-based survey. Pharmacoepidemiol Drug Saf 2004;13: 437-41.
32. Staes CJ, Bennett ST, Evans RS, et al. A case for manual entry of structured, coded laboratory data from multiple sources into an ambulatory electronic health record. J Am Med Inform Assoc 2006;13: 12-15.
33. Madsen KM, Schonheyder HC, Kristensen B, et al. Can hospital discharge diagnosis be used for surveillance of bacteremia? A data quality study of a Danish hospital discharge registry. Infect Control Hosp Epidemiol 1998;19: 175-80.
34. Aronsky D, Haug PJ. Assessing the quality of clinical data in a computer-based record for calculating the pneumonia severity index. J Am Med Inform Assoc 2000;7: 55-65.
35. Wagner MM, Hogan WR. The accuracy of medication data in an outpatient electronic medical record. J Am Med Inform Assoc 1996;3: 234-44.
36. Ndira SP, Rosenberger KD, Wetter T. Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): a qualitative and quantitative comparative study. Methods Inf Med 2008;47: 489-98.
37. Olola CH, Narus S, Poynton M, et al. Patient-perceived usefulness of an emergency medical card and a continuity-of-care report in enhancing the quality of care. Int J Qual Health Care 2011;23: 60-7.
38. Pearson N, O'Brien J, Thomas H, et al. Collecting morbidity data in general practice: the Somerset morbidity project. BMJ 1996;312: 1517-20.
39. Lo Re V 3rd, Haynes K, Forde KA, et al. Validity of The Health Improvement Network (THIN) for epidemiologic studies of hepatitis C virus infection. Pharmacoepidemiol Drug Saf 2009;18: 807-14.
40. Dambro MR, Weiss BD. Assessing the quality of data entry in a computerized medical records system. J Med Syst 1988;12: 181-7.
41. Nazareth I, King M, Haines A, et al. Accuracy of diagnosis of psychosis on general practice computer system. BMJ 1993;307: 32-4.
42. Dawson C, Perkins M, Draper E, et al. Are outcome data regarding the survivors of neonatal care available from routine sources? Arch Dis Child Fetal Neonatal Ed 1997;77: F206-10.
43. Van Weel C. Validating long term morbidity recording. J Epidemiol Community Health 1995;49(Suppl. 1): 29-32.
44. Powell J, Fitton R, Fitton C. Sharing electronic health records: the patient view. Inform Prim Care 2006;14: 55-7.
45. Meara J, Bhowmick BK, Hobson P. Accuracy of diagnosis in patients with presumed Parkinson's disease. Age Ageing 1999;28: 99-102.
46. Vawdrey DK, Gardner RM, Evans RS, et al. Assessing data quality in manual entry of ventilator settings. J Am Med Inform Assoc 2007;14: 295-303.
47. Van Staa TP, Abenhaim L, Cooper C, et al. The use of a large pharmacoepidemiological database to study exposure to oral corticosteroids and risk of fractures: validation of study population and results. Pharmacoepidemiol Drug Saf 2000;9: 359-66.
48. Margulis AV, Garcia Rodriguez LA, Hernandez-Diaz S. Positive predictive value of computerized medical records for uncomplicated and complicated upper gastrointestinal ulcer. Pharmacoepidemiol Drug Saf 2009;18: 900-9.
49. Linder JA, Kaleba EO, Kmetik KS. Using electronic health records to measure physician performance for acute conditions in primary care: empirical evaluation of the community-acquired pneumonia clinical quality measure set. Med Care 2009;47: 208-16.
50. de Lusignan S, Chan T, Wood O, et al. Quality and variability of osteoporosis data in general practice computer records: implications for disease registers. Public Health 2005;119: 771-80.
51. de Lusignan S, Valentin T, Chan T, et al. Problems with primary care data quality: osteoporosis as an exemplar. Inform Prim Care 2004;12: 147-56.
52. Faulconer ER, de Lusignan S. An eight-step method for assessing diagnostic data quality in practice: chronic obstructive pulmonary disease as an exemplar. Inform Prim Care 2004;12: 243-54.
53. Hassey A, Gerrett D, Wilson A. A survey of validity and utility of electronic patient records in a general practice. BMJ 2001;322: 1401-5.
54. Tang PC, Ralston M, Arrigotti MF, et al. Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc 2007;14: 10-15.
55. Hohnloser JH, Puerner F, Soltanian H. Improving coded data entry by an electronic patient record system. Methods Inf Med 1996;35: 108-11.
56. Horsfield P. Trends in data recording by general practice teams: an analysis of data extracted from clinical computer systems by the PRIMIS project. Informat Prim Care 2002;10: 227-34.
57. Pringle M, Ward P, Chilvers C. Assessment of the completeness and accuracy of computer medical records in four practices committed to recording data on computer. Br J Gen Pract 1995;45: 537-41.
58. Thiru K, De Lusignan S, Hague N. Have the completeness and accuracy of computer medical records in general practice improved in the last five years? The report of a two-practice pilot study. Health Informatics Journal 1999;5: 224-32.
59. Botsis T, Hartvigsen G, Chen F, et al. Secondary use of EHR: data quality issues and informatics opportunities. AMIA Summits Transl Sci Proc 2010;2010: 1-5.
60. de Lusignan S, Khunti K, Belsey J, et al. A method of identifying and correcting miscoding, misclassification and misdiagnosis in diabetes: a pilot and validation study of routinely collected data. Diabet Med 2010;27: 203-9.
61. de Burgos-Lunar C, Salinero-Fort MA, Cardenas-Valladolid J, et al. Validation of diabetes mellitus and hypertension diagnosis in computerized medical records in primary health care. BMC Med Res Methodol 2011;11: 146.
62. Basden A, Clark EM. Data integrity in a general practice computer system (CLINICS). Int J Biomed Comput 1980;11: 511-19.
63. Hammond KW, Helbig ST, Benson CC, et al. Are electronic medical records trustworthy? Observations on copying, pasting and duplication. AMIA Annu Symp Proc 2003: 269-73.
64. Weir CR, Hurdle JF, Felgar MA, et al. Direct text entry in electronic progress notes. An evaluation of input errors. Methods Inf Med 2003;42: 61-7.
65. Hogan WR, Wagner MM. Free-text fields change the meaning of coded data. Proc AMIA Annu Fall Symp 1996: 517-21.
66. Brown PJ, Warmington V. Data quality probes-exploiting and improving the quality of electronic patient record data and patient care. Int J Med Inform 2002;68: 91-8.
67. Brown PJ, Warmington V. Info-tsunami: surviving the storm with data quality probes. Inform Prim Care 2003;11: 229-33; discussion 234-7.
68. Goulet JL, Erdos J, Kancir S, et al. Measuring performance directly using the veterans health administration electronic medical record: a comparison with external peer review. Med Care 2007;45: 73-9.
69. Jelovsek F, Hammond W. Formal error rate in a computerized obstetric medical record. Methods Inf Med 1978;17: 151-7.
70. Owen RR, Thrush CR, Cannon D, et al. Use of electronic medical record data for quality improvement in schizophrenia treatment. J Am Med Inform Assoc 2004;11: 351-7.
71. Stein HD, Nadkarni P, Erdos J, et al. Exploring the degree of concordance of coded and textual data in answering clinical queries from a clinical data repository. J Am Med Inform Assoc 2000;7: 42-54.
72. Terry AL, Chevendra V, Thind A, et al. Using your electronic medical record for research: a primer for avoiding pitfalls. Fam Pract 2010;27: 121-6.
73. Benson M, Junger A, Quinzio L, et al. Influence of the method of data collection on the documentation of blood-pressure readings with an Anesthesia Information Management System (AIMS). Methods Inf Med 2001;40: 190-5.
74. Einbinder JS, Rury C, Safran C. Outcomes research using the electronic patient record: beth Israel Hospital's experience with anticoagulation. Proceedings of the Annual Symposium on Computer Application in Medical Care. 1995: 819-23. This conference was held in New Orleans, Louisiana from October 28th to November 1st in 1995. http://www.ncbi.nlm.nih.gov/pmc/issues/173480/
75. Forster M, Bailey C, Brinkhof MW, et al. ART-LINC collaboration of International Epidemiological Databases to Evaluate AIDS. Electronic medical record systems, data quality and loss to follow-up: survey of antiretroviral therapy programmes in resource-limited settings. Bull WHO 2008;86: 939-47.
76. Hahn KA, Ohman-Strickland PA, Cohen DJ, et al. Electronic medical records are not associated with improved documentation in community primary care practices. Am J Med Qual 2011;26: 272-7.
77. Jones RB, Hedley AJ. A computer in the diabetic clinic. Completeness of data in a clinical information system for diabetes. Practical Diabetes International 1986;3: 295-6.
78. Porcheret M, Hughes R, Evans D, et al. Data quality of general practice electronic health records: the impact of a program of assessments, feedback, and training. J Am Med Inform Assoc 2004;11: 78-86.
79. Scobie S, Basnett I, McCartney P. Can general practice data be used for needs assessment and health care planning in an inner-London district? J Public Health Med 1995;17: 475-83.
80. Soto CM, Kleinman KP, Simon SR. Quality and correlates of medical record documentation in the ambulatory care setting. BMC Health Serv Res 2002;2: 22.
81. Tang PC, LaRosa MP, Gorden SM. Use of computer-based records, completeness of documentation, and appropriateness of documented clinical decisions. J Am Med Inform Assoc 1999;6: 245-51.
82. de Lusignan S, Hague N, Brown A, et al. An educational intervention to improve data recording in the management of ischaemic heart disease in primary care. J Public Health (Oxf) 2004;26: 34-7.
83. Jensen RE, Chan KS, Weiner JP, et al. Implementing electronic health recordbased quality measures for developmental screening. Pediatrics 2009;124: e648-54.
84. Williams JG. Measuring the completeness and currency of codified clinical information. Methods Inf Med 2003;42: 482-8.
85. Agnew-Blais JC, Coblyn JS, Katz JN, et al. Measuring quality of care for rheumatic diseases using an electronic medical record. Ann Rheum Dis 2009;68: 680-4.
86. Asche C, Said Q, Joish V, et al. Assessment of COPD-related outcomes via a national electronic medical record database. Int J Chron Obstruct Pulmon Dis 2008;3: 323-6.
87. Jick H, Terris BZ, Derby LE, et al. Further validation of information recorded on a general practitioner based computerized data resource in the United Kingdom. Pharmacoepidemiology and Drug Safety 1992;1: 347-9.
88. Neal RD, Heywood PL, Morley S. Real world data e retrieval and validation of consultation data from four general practices. Family Practice 1996; 13: 455-61.
89. Noel PH, Copeland LA, Perrin RA, et al. VHA Corporate Data Warehouse height and weight data: opportunities and challenges for health services research. J Rehabil Res Dev 2010;47: 739-50.
90. Lawrenson R, Todd JC, Leydon GM, et al. Validation of the diagnosis of venous thromboembolism in general practice database studies. Br J Clin Pharmacol 2000;49: 591-6.
91. Roos LL, Sharp SM, Wajda A. Assessing data quality: a computerized approach. Soc Sci Med 1989;28: 175-82.
92. Mikkelsen G, Aasly J. Concordance of information in parallel electronic and paper based patient records. Int J Med Inform 2001;63: 123-31.
93. Jick H, Jick SS, Derby LE. Validation of information recorded on general practitioner based computerised data resource in the United Kingdom. BMJ 1991;302: 766-8.
94. Stausberg J, Koch D, Ingenerf J, et al. Comparing paper-based with electronic patient records: lessons learned during a study on diagnosis and procedure codes. J Am Med Inform Assoc 2003;10: 470-7.
95. Pakhomov SV, Jacobsen SJ, Chute CG, et al. Agreement between patientreported symptoms and their documentation in the medical record. Am J Manag Care 2008;14: 530-9.
96. Conroy MB, Majchrzak NE, Silverman CB, et al. Measuring provider adherence to tobacco treatment guidelines: a comparison of electronic medical record review, patient survey, and provider survey. Nicotine Tob Res 2005;7(Suppl. 1): S35-43.
97. de Lusignan S, Chan T, Wells S, et al. Can patients with osteoporosis, who should benefit from implementation of the national service framework for older people, be identified from general practice computer records? A pilot study that illustrates the variability of computerized medical records and problems with searching them. Public Health 2003;117: 438-45.
98. Haynes K, Forde KA, Schinnar R, et al. Cancer incidence in The Health Improvement Network. Pharmacoepidemiol Drug Saf 2009;18: 730-6.
99. Iyen-Omofoman B, Hubbard RB, Smith CJ, et al. The distribution of lung cancer across sectors of society in the United Kingdom: a study using national primary care data. BMC Public Health 2011;11: 857.
100. Johnson N, Mant D, Jones L, et al. Use of computerised general practice data for population surveillance: comparative study of influenza data. BMJ 1991;302: 763-5.
101. Haynes K, Bilker WB, Tenhave TR, et al. Temporal and within practice variability in the health improvement network. Pharmacoepidemiol Drug Saf 2011;20: 948-55.
102. Kaye JA, Derby LE, del Mar Melero-Montes M, et al. The incidence of breast cancer in the General Practice Research Database compared with national cancer registration data. Br J Cancer 2000;83: 1556-8.
103. Hansell A, Hollowell J, Nichols T, et al. Use of the General Practice Research Database (GPRD) for respiratory epidemiology: a comparison with the 4th Morbidity Survey in General Practice (MSGP4). Thorax 1999;54(1): 413-9.
104. de Lusignan S, Belsey J, Hague N, et al. End-digit preference in blood pressure recordings of patients with ischaemic heart disease in primary care. J Hum Hypertens 2004;18: 261-5.
105. Haerian K, McKeeby J, Dipatrizio G, et al. Use of clinical alerting to improve the collection of clinical research data. AMIA Annu Symp Proc 2009;2009: 218-22.
106. Alsanjari ON, de Lusignan S, van Vlymen J, et al. Trends and transient change in end-digit preference in blood pressure recording: studies of sequential and longitudinal collected primary care data. Int J Clin Pract 2012;66: 37-43.
107. Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. J Manag Inform Syst 1996;12: 5-34.
108. Committee on Improving the Patient Record Institute of Medicine. The Computer-Based Patient Record: An Essential Technology for Health Care, Revised Edition. Institute of Medicine. The National Academies Press, 1997. http://iom.edu/Reports/1997/The-Computer-Based-Patient-Record-An-Essential-Technology-for-Health-Care-Revised-Edition.aspx
109. Pipino LL, Lee YW, Wang RY. Data quality assessment. Comm ACM 2002;45: 211-18.
110. de Lusignan S. The optimum granularity for coding diagnostic data in primary care: report of a workshop of the EFMI Primary Care Informatics Working Group at MIE 2005. Inform Prim Care 2006;14: 133-7.