A review of data quality assessment methods for public health information systems

International Journal of Environmental Research and Public Health

Volumn 11, Issue 5, 2014, Pages 5170-5207

  Chen, Hong ^a,b   Hailey, David ^a   Wang, Ning ^c   Yu, Ping ^a  

^a UNIVERSITY OF WOLLONGONG   (Australia)

^b Jiangxi Provincial Center for Disease Control and Prevention   (China)

^c CHINESE CENTER FOR DISEASE CONTROL AND PREVENTION   (China)

Author keywords

Assessment; Data collection process; Data quality; Data use; Evaluation; Information quality; Information systems; Population health; Public health

Indexed keywords

ASSESSMENT METHOD; DATA QUALITY; INFORMATION SYSTEM; PUBLIC HEALTH;

COMPLETION; CONCEPTUAL FRAMEWORK; DATA ANALYSIS; DATA BASE; DATA COLLECTION METHOD; DATA QUALITY; DESCRIPTIVE RESEARCH; DOCUMENTATION; HUMAN; INFORMATION PROCESSING; INFORMATION SYSTEM; INTERNAL CONSISTENCY; INTERVIEW; MEASUREMENT ACCURACY; OUTCOME ASSESSMENT; PUBLIC HEALTH; PUBLIC HEALTH INFORMATION SYSTEM; PUBLICATION; QUALITATIVE ANALYSIS; QUALITY CONTROL; QUANTITATIVE ANALYSIS; REVIEW; STATISTICAL PARAMETERS; SYSTEMATIC REVIEW; TIMELINESS; COMPARATIVE STUDY; HEALTH CARE QUALITY; MEDICAL INFORMATION SYSTEM; PROCEDURES; TRENDS;

HEALTH INFORMATION SYSTEMS; HUMANS; OUTCOME ASSESSMENT (HEALTH CARE); PUBLIC HEALTH; QUALITY ASSURANCE, HEALTH CARE;

EID: 84900790703     PISSN: 16617827     EISSN: 16604601     Source Type: Journal    
DOI: 10.3390/ijerph110505170     Document Type: Review

References (76)

1. Winslow, C.E. The untilled fields of public health. Science 1920, 51, 23-33.
2. Walker, R. Health information and public health. Health Inf. Manag. J. 2008, 37, 4-5.
3. Institute of Medicine. The Future of Public Health; National Academies Press: Washington, DC, USA, 1988.
4. Andresen, E.; Bouldin, E.D. Public Health Foundations: Concepts and Practices; Jossey-Bass: Hoboken, NJ, USA, 2010.
5. World Health Organization. Framework and Standards for Country Health Information Systems; World Health Organization: Geneva, Switzerland, 2008.
6. Australian Institute of Health and Welfare (AIHW); National Public Health Information Working Group. National Public Health Information Plan 2005; AIHW: Canberra, Australia, 2005.
7. O'Carroll, P.W.; Yasnoff, W.A.; Ward, M.E.; Ripp, L.H.; Martin, E.L. Public Health Informatics and Information Systems; Springer Science+Business Media Inc.: New York, NY, USA, 2003.
8. Lin, V.; Smith, J.; Fawkes, S.; Robinson, P.; Chaplin, S. Public Health Practice in Australia: The Organised Effort; Allen & Unwin: Crows Nest, NSW, Australia, 2007.
9. World Health Organization. An introduction to the World Health Organization. Available online: http://www.who.int/about/brochure_en.pdf (accessed on 30 August 2013).
10. European Commission. European Core Health Indicators. Available online: http://ec.europa.eu/ health/indicators/echi/index_en.htm (accessed on 30 August 2013).
11. U.S. Department of Health and Human Services. HealthyPeople. gov. Available online: http://www.healthypeople.gov/2020/about/default.aspx (accessed on 30 August 2013).
12. Preventative Health Taskforce. Australia: The Healthiest Country by 2020. Available online: http://www.preventativehealth.org.au/internet/preventativehealth/publishing.nsf/Content/nphsoverview-toc (accessed on 30 August 2013).
13. National Health and Family Planning Commission of the People's Republic of China. Healthy China 2020. Available online: http://www.moh.gov.cn/wsb/pzcjd/201208/55651.shtml (accessed on 30 August 2013).
14. Yasnoff, W.; O'Carroll, P.; Koo, D.; Linkins, R.; Kilbourne, E. Public health informatics: Improving and transforming public health in the information age. J. Public Health Manag. Pract. 2000, 6, 67-75.
15. CDC. Updated guidelines for evaluating surveillance systems: Recommendations from the guidelines working group. Morb. Mortal. Wkly. Rep. 2001, 50, 1-35.
16. Loonsk, J.W.; McGarvey, S.R.; Conn, L.A.; Johnson, J. The public health information network (PHIN) preparedness initiative. J. Am. Med. Inform. Assoc. 2006, 13, 1-4.
17. Thacker, S.B.; Stroup, D.F. Future directions for comprehensive public health surveillance and health information systems in the United States. Am. J. Epidemiol. 1994, 140, 383-397.
18. Araujo, J.; Pepper, C.; Richards, J.; Choi, M.; Xing, J.; Li, W. The profession of public health informatics: Still emerging? Int. J. Med. Inform. 2009, 78, 375-385.
19. World Health Organization. Monitoring the Building Blocks of Health System: A Handbook of Indicators and Their Measurement Strategies; World Health Organization: Geneva, Switzerland, 2010.
20. Wang, L.; Wang, Y.; Jin, S.; Wu, Z.; Chin, D.P.; Koplan, J.P.; Wilson, M.E. Emergence and control of infectious diseases in China. Lancet 2008, 372, 1598-1605.
21. Demlo, L.K.; Gentleman, J.F. Morbidity Data. In Public Health Informatics and Information Systems; O'Carroll, P.W., Yasnoff, W.A., Ward, M.E., Ripp, L.H., Martin, E.L., Eds.; Springer Science+Business Media Inc.: New York, NY, USA, 2003; pp. 286-315.
22. Venkatarao, E.; Patil, R.R.; Prasad, D.; Anasuya, A.; Samuel, R. Monitoring data quality in syndromic surveillance: Learnings from a resource limited setting. J. Glob. Infect. Dis. 2012, 4, 120-127.
23. Cheng, P.; Gilchrist, A.; Robinson, K.M.; Paul, L. The risk and consequences of clinical miscoding due to inadequate medical documentation: A case study of the impact on health services funding. Health Inf. Manag. J. 2009, 38, 35-46.
24. Chan, M.; Kazatchkine, M.; Lob-Levyt, J.; Obaid, T.; Schweizer, J.; Sidibe, M.; Veneman, A.; Yamada, T. Meeting the demand for results and accountability: a call for action on health data from eight global health agencies. PLoS Med. 2010, 7, doi:10.1371/journal.pmed.1000223.
25. Chen, H.; Yu, P.; Wang, N. Do we have the reliable data? An exploration of data quality for AIDS information system in China. Stud. Health Technol. Inform. 2012, 192, 1042-1042.
26. Wilkinson, D.L.; McCarthy, M. Use of comparative data for integrated cancer services. BMC Health Serv. Res. 2007, 7, 204.
27. Qazi, M.S.; Ali, M. Health management information system utilization in Pakistan: Challenges, pitfalls and the way forward. BioSci. Trends 2011, 5, 245-254.
28. Rugg, D.; Marais, H.; Carael, M.; de Lay, P.; Warner-Smith, M. Are we on course for reporting on the millennium development goals in 2015? J. Acquir. Immune Defic. Syndr. 2009, 52, S69-S76.
29. Chan, M. From new estimates to better data. Lancet 2012, 380, 2054.
30. Canadian Institute for Health Information. The CIHI Data Quality Framework; CIHI: Ottawa, ON, Canada, 2009.
31. Institute of Medicine. The Future of the Public's Health in the 21st Century; The National Academies Press: Washington, DC, USA,2003.
32. Snee, N.L.; McCormick, K.A. The case for integrating public health informatics networks. IEEE Eng. Med. Biol. Mag. 2004, 23, 81-88.
33. Wang, R.Y.; Strong, D.M. Beyond accuracy: What data quality means to data consumers. J. Manag. Inf. Syst. 1996, 12, 5-33.
34. MEASURE Evaluation. Data Quality Audit Tool Guidelines for Implementation. Available online: http://www.theglobalfund.org/en/me/documents/dataquality/ (accessed on 30 August 2013).
35. Karr, A.F.; Sanil, A.P.; Banks, D.L. Data quality: A statistical perspective. Stat. Methodol. 2006, 3, 137-173.
36. Redman, T.C. Measuring Data Accuracy A Framework and Review. In Information Quality; Wang, R.Y., Pierce, E.M., Madnick, S.E., Eds.; M.E. Sharpe, Inc.: Armonk, NY, USA, 2005; pp. 21-36.
37. Batini, C.; Cappiello, C.; Francalanci, C.; Maurino, A. Methodologies for data quality assessment and improvement. ACM Comput. Surv. 2009, 41, 1-52.
38. Pipino, L.; Wang, R.Y.; Kopcso, D.; Rybolt, W. Developing Measurement Scales for Data-quality Dimensions. In Information Quality; Wang, R.Y., Pierce, E.M., Madnick, S.E., Eds.; M.E. Sharpe, Inc.: Armonk, NY, USA, 2005; pp. 37-51.
39. MEASURE Evaluation. Routine Data Quality Assessment Tool (RDQA): Guidelines for Implementation for HIV, TB, & Malaria Programs. Available online: http://www.theglobalfund. org/en/me/documents/dataquality/ (accessed on 30 August 2013).
40. MEASURE Evaluation. PRISM: Performance of Routine Information System Management-PRISM Tools for Assessing, Monitoring, and Evaluating RHIS Performance: PRISM Tool Version 3.1 March 2010. Available online: http://www.cpc.unc.edu/measure/tools/monitoring-evaluationsystems/ prism (accessed on 30 August 2013).
41. Aqil, A.; Lippeveld, T.; Hozumi, D. PRISM framework: A paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan. 2009, 24, 217-228.
42. World Health Organization. The Immunization Data Quality Audit (DQA) Procedure; World Health Organization: Geneva, Switzerland, 2003.
43. World Health Organization. The Immunization Data Quality Self-assessment (DQS) Tool; World Health Organization: Geneva, Switzerland, 2005.
44. World Health Organization. Guide to the Health Facility Data Quality Report Card. Available online: http://www.who.int.ezproxy.uow.edu.au/healthinfo/topics_standards_tools_data_quality_ analysis/en/ (accessed on 30 August 2013).
45. World Health Organization. Assessing the National Health Information System: An Assessment Tool Version 4.00; World Health Organization: Geneva, Switzerland, 2008.
46. Van Hest, N.A.H.; Story, A.; Grant, A.D.; Antoine, D.; Crofts, J.P.; Watson, J.M. Record-linkage and capture-recapture analysis to estimate the incidence and completeness of reporting of tuberculosis in England 1999-2002. Epidemiol. Infect. 2008, 136, 1606-1616.
47. Savas, L.; del Junco, D.; Bastian, L.; Vernon, S. Mortality ascertainment of women veterans: A comparison of sources of vital status information, 1979-2002. Med. Care 2009, 47, 125-128.
48. Mate, K.S.; Bennett, B.; Mphatswe, W.; Barker, P.; Rollins, N. Challenges for routine health system data management in a large public programme to prevent mother-to-child HIV transmission in South Africa. PLoS One 2009, 4, e5483.
49. Makombe, S.D.; Hochgesang, M.; Jahn, A.; Tweya, H.; Hedt, B.; Chuka, S.; Yu, J.K.-L.; Aberle-Grasse, J.; Pasulani, O.; Bailey, C.; et al. Assessing the quality of data aggregated by antiretroviral treatment clinics in Malawi. Bull. World Health Organ. 2008, 86, 310-314.
50. Hahn, D.; Wanjala, P.; Marx, M. Where is information quality lost at clinical level? A mixed-method study on information systems and data quality in three urban Kenyan ANC clinics. Glob. Health Action 2013, 6, 1-10.
51. Frizzelle, B.G.; Evenson, K.R.; Rodriguez, D.A.; Laraia, B.A. The importance of accurate road data for spatial applications in public health: Customizing a road network. Int. J. Health Geogr. 2009, 8, doi:10.1186/1476-072X-8-24.
52. Freestone, D.; Williamson, D.; Wollersheim, D. Geocoding coronial data: Tools and techniques to improve data quality. Health Inf. Manag. J. 2012, 41, 4-12.
53. Ford, J.; Roberts, C.; Algert, C.; Bowen, J.; Bajuk, B.; Henderson-Smart, D. Using hospital discharge data for determining neonatal morbidity and mortality: A validation study. BMC Health Serv. Res. 2007, 20, doi:10.1186/1472-6963-7-188.
54. Dixon, B.E.; McGowan, J.J.; Grannis, S.J. Electronic laboratory data quality and the value of a health information exchange to support public health reporting processes. AMIA Annu. Symp. Proc. 2011, 2011, 322-330.
55. Corriols, M.; Marín, J.; Berroteran, J.; Lozano, L.; Lundberg, I.; Thörn, Å. The Nicaraguan Pesticide Poisoning Register: Constant Underreporting. Int. J. Health Serv. 2008, 38, 773-787.
56. Clayton, H.B.; Sappenfield, W.M.; Gulitz, E.; Mahan, C.S.; Petersen, D.J.; Stanley, K.M.; Salihu, H.M. The florida investigation of primary late preterm and cesarean delivery: The accuracy of the birth certificate and hospital discharge records. Matern. Child Health J. 2013, 17, 869-878.
57. Chiba, Y.; Oguttu, M.A.; Nakayama, T. Quantitative and qualitative verification of data quality in the childbirth registers of two rural district hospitals in Western Kenya. Midwifery 2012, 28, 329-339.
58. Bosch-Capblanch, X.; Ronveaux, O.; Doyle, V.; Remedios, V.; Bchir, A. Accuracy and quality of immunization information systems in forty-one low income countries. Trop. Med. Int. Health 2009, 14, 2-10.
59. Ancker, J.S.; Shih, S.; Singh, M.P.; Snyder, A.; Edwards, A.; Kaushal, R. Root causes underlying challenges to secondary use of data. AMIA Annu. Symp. Proc. 2011, 2011, 57-62.
60. Ronveaux, O.; Rickert, D.; Hadler, S.; Groom, H.; Lloyd, J.; Bchir, A.; Birmingham, M. The immunization data quality audit: Verifying the quality and consistency of immunization monitoring systems. Bull. World Health Organ. 2005, 83, 503-510.
61. Saeed, K.M.I.; Bano, R.; Asghar, R.J. Evaluation of the national tuberculosis surveillance system in Afghanistan. East. Mediterr. Health J. 2013, 19, 200-207.
62. Petter, S.; Fruhling, A. Evaluating the success of an emergency response medical information system. Int. J. Med. Inform. 2011, 80, 480-489.
63. Lowrance, D.; Filler, S.; Makombe, S.; Harries, A.; Aberle-Grasse, J.; Hochgesang, M.; Libamba, E. Assessment of a national monitoring and evaluation system for rapid expansion of antiretroviral treatment in Malawi. Trop. Med. Int. Health 2007, 12, 377-381.
64. Litow, F.; Krahl, P. Public health potential of a disability tracking system: Analysis of U.S. navy and marine corps physical evaluation boards 2005-2006. Mil. Med. 2007, 172, 1270-1274.
65. Lin, W.; Chen, S.; Seguy, N.; Chen, Z.; Sabin, K.; Callejaf, J.G.; Bulterysa, M. Is the HIV sentinel surveillance system adequate in China? Findings from an evaluation of the national HIV sentinel surveillance system. West. Pac. Surveill. Response J. 2012, 3, 78-85.
66. Harper, S.L.; Edge, V.L.; Schuster-Wallace, C.J.; Ar-Rushdi, M.; McEwen, S.A. Improving Aboriginal health data capture: Evidence from a health registry evaluation. Epidemiol. Infect. 2011, 139, 1774-1783.
67. Forster, M.; Bailey, C.; Brinkhof, M.W.G.; Graber, C.; Boulle, A.; Spohr, M.; Balestre, E.; May, M.; Keiser, O.; Jahnf, A.; et al. Electronic medical record systems, data quality and loss to follow-up: Survey of antiretroviral therapy programmes in resource-limited settings. Bull. World Health Organ. 2008, 86, 939-947.
68. Edmond, M.; Wong, C.; Chuang, S.K. Evaluation of sentinel surveillance system for monitoring hand, foot and mouth disease in Hong Kong. Public Health 2011, 125, 777-783.
69. Dai, L.; Zhu, J.; Liang, J.; Wang, Y.; Wang, H.; Mao, M. Birth defects surveillance in China. World J. Pediatr. 2011, 7, 302-310.
70. Iguiniz-Romero, R.; Palomino, N. Data do count! Collection and use of maternal mortality data in Peru, 1990-2005, and improvements since 2005. Reproduct. Health Matters 2012, 20, 174-184.
71. Matheson, A.I.; Baseman, J.G.; Wagner, S.H.; O'Malley, G.E.; Puttkammer, N.H.; Emmanuel, E.; Zamor, G.; Frédéric, R.; Coq, N.R.; Lober, W.B. Implementation and expansion of an electronic medical record for HIV care and treatment in Haiti: An assessment of system use and the impact of large-scale disruptions. Int. J. Med. Inform. 2012, 81, 244-256.
72. Pereira, J.A.; Quach, S.; Hamid, J.S.; Heidebrecht, C.L.; Quan, S.D.; Nassif, J.; Diniz, A.J.; van Exan, R.; Malawski, J.; Gentry, A.; et al. Exploring the feasibility of integrating barcode scanning technology into vaccine inventory recording in seasonal influenza vaccination clinics. Vaccine 2012, 30, 794-802.
73. Hills, R.A.; Revere, D.; Altamore, R.; Abernethy, N.F.; Lober, W.B. Timeliness and data element completeness of immunization data in Washington State in 2010: A comparison of data exchange methods. AMIA Annu. Symp. Proc. 2012, 2012, 340-349.
74. Lash, R.R.; Carroll, D.S.; Hughes, C.M.; Nakazawa, Y.; Karem, K.; Damon, I.K.; Peterson, A.T. Effects of georeferencing effort on mapping monkeypox case distributions and transmission risk. Int. J. Health Geogr. 2012, 11, 23.
75. Strong, D.M.; Lee, Y.W.; Wang, R.Y. Data quality in context. Commun. ACM 1997, 40, 103-110.
76. Pipino, L.L.; Lee, Y.W.; Wang, R.Y. Data quality assessment. Commun. ACM 2002, 45, 211-218.