Effectiveness of Practices to Support Appropriate Laboratory Test Utilization

American Journal of Clinical Pathology

Volumn 149, Issue 3, 2018, Pages 197-221

  Rubinstein, Matthew ^a   Hirsch, Robert ^b   Bandyopadhyay, Kakali ^c   Madison, Bereneice ^a   Taylor, Thomas ^a   Ranne, Anne ^a   Linville, Millie ^c   Donaldson, Keri ^d   Lacbawan, Felicitas ^e   Cornish, Nancy ^a  

^a CENTERS FOR DISEASE CONTROL AND PREVENTION   (United States)

^b Carter Consulting Inc   (United States)

^c Deloitte Consulting   (United States)

^d PENNSYLVANIA STATE UNIVERSITY   (United States)

^e QUEST DIAGNOSTICS NICHOLS INSTITUTE   (United States)

Author keywords

Clinical decision making; Clinical laboratory services; Laboratory diagnosis; Laboratory medicine; Laboratory test utilization; Meta analysis; Quality assurance; Quality improvement; Systematic review; Test ordering; Utilization management; Utilization review

Indexed keywords

CLINICAL DECISION MAKING; CLINICAL EFFECTIVENESS; CLINICAL OUTCOME; CLINICAL PRACTICE; COMPUTERIZED PROVIDER ORDER ENTRY; COST; EDUCATION; FEEDBACK SYSTEM; HEALTH CARE UTILIZATION; HOSPITAL READMISSION; HOSPITALIZATION; HUMAN; LABORATORY TEST; LENGTH OF STAY; MORBIDITY; MORTALITY; PRIORITY JOURNAL; PUBLIC HEALTH SERVICE; REVIEW; SYSTEMATIC REVIEW; TOTAL QUALITY MANAGEMENT; CLINICAL DECISION SUPPORT SYSTEM; HEALTH CARE QUALITY; LABORATORY TECHNIQUE; META ANALYSIS; PHYSICIAN ORDER ENTRY SYSTEM; PRACTICE GUIDELINE; PROCEDURES; STANDARDS; UTILIZATION;

CLINICAL LABORATORY TECHNIQUES; DECISION SUPPORT SYSTEMS, CLINICAL; HUMANS; MEDICAL ORDER ENTRY SYSTEMS; PRACTICE GUIDELINES AS TOPIC; QUALITY ASSURANCE, HEALTH CARE; QUALITY IMPROVEMENT;

EID: 85042530707     PISSN: 00029173     EISSN: 19437722     Source Type: Journal    
DOI: 10.1093/ajcp/aqx147     Document Type: Review

References (138)

1. Balogh E, Miller BT, Ball J; Institute of Medicine (U. S. ). Committee on Diagnostic Error in Health Care: Improving Diagnosis in Health Care. Washington, DC: The National Academies Press; 2015. xxvii, 444.
2. Hallworth MJ, Epner PL, Ebert C, et al. IFCC Task Force on the Impact of Laboratory Medicine on Clinical Management and Outcomes. Current evidence and future perspectives on the effective practice of patient-centered laboratory medicine. Clin Chem. 2015;61:589-599.
3. Plebani M. Towards a new paradigm in laboratory medicine: the five rights. Clin Chem Lab Med. 2016;54:1881-1891.
4. Rohr UP, Binder C, Dieterle T, et al. The value of in vitro diagnostic testing in medical practice: a status report. Plos One. 2016;11:e0149856.
5. Zhi M, Ding EL, Theisen-Toupal J, et al. The landscape of inappropriate laboratory testing: a 15-year meta-analysis. Plos One. 2013;8:e78962.
6. Plebani M, Panteghini M. Promoting clinical and laboratory interaction by harmonization. Clin Chim Acta. 2014;432:15-21.
7. Baird G. The laboratory test utilization management toolbox. Biochem Med (Zagreb). 2014;24:223-234.
8. Janssens PM. Managing the demand for laboratory testing: options and opportunities. Clin Chim Acta. 2010;411:1596-1602.
9. Huck A, Lewandrowski K. Utilization management in the clinical laboratory: an introduction and overview of the literature. Clin Chim Acta. 2014;427:111-117.
10. Plebani M, Zaninotto M, Faggian D. Utilization management: a European perspective. Clin Chim Acta. 2014;427:137-141.
11. Yeh DD. A clinician's perspective on laboratory utilization management. Clin Chim Acta. 2014;427:145-50.
12. Solomon DH, Hashimoto H, Daltroy L, et al. Techniques to improve physicians' use of diagnostic tests: a new conceptual framework. JAMA. 1998;280:2020-2027.
13. Fryer AA, Smellie WS. Managing demand for laboratory tests: a laboratory toolkit. J Clin Pathol. 2013;66:62-72.
14. Smellie WS. Demand management and test request rationalization. Ann Clin Biochem. 2012;49:323-336.
15. Barth JH. Clinical quality indicators in laboratory medicine. Ann Clin Biochem. 2012;49 (pt 1):9-16.
16. Ferraro S, Panteghini M. The role of laboratory in ensuring appropriate test requests. Clin Biochem. 2017;50:555-561.
17. Plebani M. Exploring the iceberg of errors in laboratory medicine. Clin Chim Acta. 2009;404:16-23.
18. Barr JT, Silver S. The total testing process and its implications for laboratory administration and education. Clin Lab Manage Rev. 1994;8:526-542.
19. Christenson RH, Snyder SR, Shaw CS, et al. Laboratory medicine best practices: systematic evidence review and evaluation methods for quality improvement. Clin Chem. 2011;57:816-825.
20. Hauser RG, Shirts BH. Do we now know what inappropriate laboratory utilization is? An expanded systematic review of laboratory clinical audits. Am J Clin Pathol. 2014;141:774-783.
21. Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549.
22. Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557-560.
23. Borenstein M. Introduction to Meta-Analysis. Chichester, UK: John Wiley & Sons; 2009:xxviii, 421.
24. Bansal P, Aronsky D, Aronsky D, et al. A computer based intervention on the appropriate use of arterial blood gas. Proc AMIA Symp. 2001:32-6.
25. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999;106:144-150.
26. Bates DW, Kuperman GJ, Jha A, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med. 1997;157:2501-2508.
27. Bridges SA, Papa L, Norris AE, et al. Duplicated laboratory tests: evaluation of a computerized alert intervention abstract. J Healthc Qual. 2014;36:46-53.
28. Fang DZ, Sran G, Gessner D, et al. Cost and turn-around time display decreases inpatient ordering of reference laboratory tests: a time series. BMJ Qual Saf. 2014;23:994-1000.
29. Feldman LS, Shihab HM, Thiemann D, et al. Impact of providing fee data on laboratory test ordering: a controlled clinical trial. Jama Intern Med. 2013;173:903-908.
30. Georgiou A, Lang S, Rosenfeld D, et al. The use of computerized provider order entry to improve the effectiveness and efficiency of coagulation testing. Arch Pathol Lab Med. 2011;135:495-498.
31. Horn DM, Koplan KE, Senese MD, et al. The impact of cost displays on primary care physician laboratory test ordering. J Gen Intern Med. 2014;29:708-14.
32. Hwang JI, Park HA, Bakken S. Impact of a physician's order entry (POE) system on physicians' ordering patterns and patient length of stay. Int J Med Inform. 2002;65:213-223.
33. Kahan NR, Waitman DA, Vardy DA. Curtailing laboratory test ordering in a managed care setting through redesign of a computerized order form. Am J Manag Care. 2009;15:173-6.
34. Le RD, Kosowsky JM, Landman AB, et al. Clinical and financial impact of removing creatine kinase-MB from the routine testing menu in the emergency setting. Am J Emerg Med. 2015;33:72-75.
35. Li L, Georgiou A, Vecellio E, et al. What is the effect of electronic pathology ordering on test re-ordering patterns for paediatric patients? Stud Health Technol Inform. 2014;204:74-79.
36. Lippi G, Brambilla M, Bonelli P, et al. Effectiveness of a computerized alert system based on re-testing intervals for limiting the inappropriateness of laboratory test requests. Clin Biochem. 2015;48:1174-1176.
37. Love SA, McKinney ZJ, Sandoval Y, et al. Electronic medical record-based performance improvement project to document and reduce excessive cardiac troponin testing. Clin Chem. 2015;61:498-504.
38. May TA, Clancy M, Critchfield J, et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol. 2006;126:200-206.
39. Olson J, Hollenbeak C, Donaldson K, et al. Default settings of computerized physician order entry system order sets drive ordering habits. J Pathol Inform. 2015;6:16.
40. Pageler NM, Franzon D, Longhurst CA, et al. Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization. Pediatr Crit Care Med. 2013;14:413-419.
41. Probst CA, Shaffer VA, Chan YR. The effect of defaults in an electronic health record on laboratory test ordering practices for pediatric patients. Health Psychol. 2013;32:995-1002.
42. Procop GW, Keating C, Stagno P, et al. Reducing duplicate testing: a comparison of two clinical decision support tools. Am J Clin Pathol. 2015;143:623-626.
43. Shalev V, Chodick G, Heymann AD. Format change of a laboratory test order form affects physician behavior. Int J Med Inform. 2009;78:639-644.
44. Solis DLE, Rabrich J, Van Leer P. Improving physician order practices and cost savings by changing the electronic medical records. Ann Emerg Med. 2015;66 (4 suppl 1):S34.
45. Vardy DA, Simon T, Limoni Y, et al. The impact of structured laboratory routines in computerized medical records in a primary care service setting. J Med Syst. 2005;29:619-626.
46. Waldron JL, Ford C, Dobie D, et al. An automated minimum retest interval rejection rule reduces repeat CRP workload and expenditure, and influences clinician-requesting behaviour. J Clin Pathol. 2014;67:731-733.
47. Westbrook JI, Georgiou A, Dimos A, et al. Computerised pathology test order entry reduces laboratory turnaround times and influences tests ordered by hospital clinicians: a controlled before and after study. J Clin Pathol. 2006;59:533-536.
48. Bindels R, Hasman A, Kester AD, et al. The efficacy of an automated feedback system for general practitioners. Inform Prim Care. 2003;11:69-74.
49. Collins RA, Triulzi DJ, Waters JH, et al. Evaluation of realtime clinical decision support systems for platelet and cryoprecipitate orders. Am J Clin Pathol. 2014;141:78-84.
50. Howell LP, MacDonald S, Jones J, et al. Can automated alerts within computerized physician order entry improve compliance with laboratory practice guidelines for ordering pap tests? J Pathol Inform. 2014;5:37.
51. McKinney ZJ, Peters JM, Gorlin JB, et al. Improving red blood cell orders, utilization, and management with point-of-care clinical decision support. Transfusion. 2015;55:2086-2094.
52. Nightingale PG, Peters M, Mutimer D, et al. Effects of a computerised protocol management system on ordering of clinical tests. Qual Health Care. 1994;3:23-28.
53. Poley MJ, Edelenbos K, Mosseveld M, et al. Cost consequences of implementing an electronic decision support system for ordering laboratory tests in primary care: evidence from a controlled prospective study in the Netherlands. Clin Chem. 2007;53:213-9.
54. Roukema J, Steyerberg EW, van der Lei J, et al. Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008;15:107-113.
55. van Wijk MA, van der Lei J, Mosseveld M, et al. Assessment of decision support for blood test ordering in primary care: a randomized trial. Ann Intern Med. 2001;134:274-281.
56. Abio-Calvete MDLO, Botón Contreras ME, et al. Demand management of analytical request in erythropathogy section of a tertiary hospital. Haematologica. 2015;100:581-582.
57. Baral N, Koner BC, Lamsal M, et al. Thyroid function testing in eastern Nepal and the impact of CME on subsequent requests. Trop Doct. 2001;31:155-157.
58. Chonfhaola ACM, O'Riain M, Murray M. Demand management in a university laboratory in the west of Ireland: the financial benefit of distribution of local guidelines for vitamin B12 and folate testing. Haematologica. 2013;98:216.
59. Dawes L, Miller L, Subramoney M, et al. The impact of education and audit on compliance with a clinical guideline for fetal fibronectin testing and the management of threatened preterm labour at National Women's Health, Auckland City Hospital. BJOG Int J Obstet Gynaecol. 2015;122:237-8.
60. DellaVolpe JD, Chakraborti C, Cerreta K, et al. Effects of implementing a protocol for arterial blood gas use on ordering practices and diagnostic yield. Healthc (Amst). 2014;2:130-135.
61. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ. 1977;52:578-581.
62. Gardezi SA. Troponin: think before you request one. BMJ Qual Improv Rep. 2015;4.
63. Kotecha N, Cardasis J, Narayanswami G, et al. Reducing unnecessary lab tests in the MICU by incorporating a guideline in daily ICU team rounds. Am J Respir Crit Care Med. 2015;191:no pagination.
64. Thakkar RN, Kim D, Knight AM, et al. Impact of an educational intervention on the frequency of daily blood test orders for hospitalized patients. Am J Clin Pathol. 2015;143:393-397.
65. Baker R, Falconer Smith J, Lambert PC. Randomised controlled trial of the effectiveness of feedback in improving test ordering in general practice. Scand J Prim Health Care. 2003;21:219-223.
66. Bunting PS, Van Walraven C. Effect of a controlled feedback intervention on laboratory test ordering by community physicians. Clin Chem. 2004;50:321-326.
67. Gama R, Nightingale PG, Broughton PM, et al. Modifying the request behaviour of clinicians. J Clin Pathol. 1992;45:248-249.
68. Miyakis S, Karamanof G, Liontos M, et al. Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy. Postgrad Med J. 2006;82:823-829.
69. Verstappen WH, van der Weijden T, Dubois WI, et al. Improving test ordering in primary care: the added value of a small-group quality improvement strategy compared with classic feedback only. Ann Fam Med. 2004;2:569-575.
70. Verstappen WH, van der Weijden T, ter Riet G, et al. Block design allowed for control of the hawthorne effect in a randomized controlled trial of test ordering. J Clin Epidemiol. 2004;57:1119-1123.
71. Winkens RA, Pop P, Grol RP, et al. Effect of feedback on test ordering behaviour of general practitioners. BMJ. 1992;304:1093-1096.
72. Baird GS, Rainey PM, Wener M, et al. Reducing routine ionized calcium measurement. Clin Chem. 2009;55:533-540.
73. Bonaguri C, Melegari A, Ballabio A, et al. Italian multicentre study for application of a diagnostic algorithm in autoantibody testing for autoimmune rheumatic disease: conclusive results. Autoimmun Rev. 2011;11:1-5.
74. Cernich JT, Hamilton M, Mitre N. Increased use of lab algorithm to reduce cost of screening for hypothyroidism in pediatric type 1 diabetes. Diabetes. 2014;63:A322-A323.
75. Froom P, Barak M. Cessation of dipstick urinalysis reflex testing and physician ordering behavior. Am J Clin Pathol. 2012;137:486-489.
76. Tampoia M, Brescia V, Fontana A, et al. Application of a combined protocol for rational request and utilization of antibody assays improves clinical diagnostic efficacy in autoimmune rheumatic disease. Arch Pathol Lab Med. 2007;131:112-116.
77. Van Walraven C, Goel V. The effect of a hepatitis serology testing algorithm on laboratory utilization. J Eval Clin Pract. 2002;8:327-32.
78. Wu AH, Ghani F, Prigent F, et al. Reflex testing II: evaluation of an algorithm for use of cardiac markers in the assessment of emergency department patients with chest pain. Clin Chim Acta. 1999;288:97-109.
79. Aesif SW, Parenti DM, Lesky L, et al. A cost-effective interdisciplinary approach to microbiologic send-out test use. Arch Pathol Lab Med. 2015;139:194-198.
80. Barazzoni F, Grilli R, Amicosante AM, et al. Impact of end user involvement in implementing guidelines on routine pre-operative tests. Int J Qual Health Care. 2002;14:321-327.
81. Chu KH, Wagholikar AS, Greenslade JH, et al. Sustained reductions in emergency department laboratory test orders: impact of a simple intervention. Postgrad Med J. 2013;89:566-571.
82. Dickerson JA, Cole B, Conta JH, et al. Improving the value of costly genetic reference laboratory testing with active utilization management. Arch Pathol Lab Med. 2014;138:110-113.
83. Dolezal A, Krasowski MD. Paraneoplastic autoantibody panel ordering as a model for laboratory test utilization analysis and intervention. Lab Invest. 2015;95:498A.
84. Miller CE, Krautscheid P, Baldwin EE, et al. Genetic counselor review of genetic test orders in a reference laboratory reduces unnecessary testing. Am J Med Genet A. 2014;164A:1094-1101.
85. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. BMJ. 1990;301:1305-1307.
86. Baricchi R, Zini M, Nibali MG, et al. Using pathology-specific laboratory profiles in clinical pathology to reduce inappropriate test requesting: two completed audit cycles. BMC Health Serv Res. 2012;12:187.
87. Calderon-Margalit R, Mor-Yosef S, Mayer M, et al. An administrative intervention to improve the utilization of laboratory tests within a university hospital. Int J Qual Health Care. 2005;17:243-248.
88. Dowling PT, Alfonsi G, Brown MI, et al. An education program to reduce unnecessary laboratory tests by residents. Acad Med. 1989;64:410-412.
89. Gilmour J, Weisman A, Orlov S, Vecchiarelli J, et al. Reducing unnecessary free thyroid hormone testing at an academic ambulatory hospital: a quality improvement (QI) initiative. Thyroid. 2015;25:A219-A20.
90. Iosfina I, Merkeley H, Cessford T, Geller G, et al. Implementation of an on-demand strategy for routine blood testing in ICU patients. Am J Respir Crit Care Med. 2013;187.
91. Hutton HD, Drummond HS, Fryer AA. The rise and fall of C-reactive protein: managing demand within clinical biochemistry. Ann Clin Biochem. 2009;46:155-158.
92. Janssens PM, Staring W, Winkelman K, et al. Active intervention in hospital test request panels pays. Clin Chem Lab Med. 2015;53:731-742.
93. Kroenke K, Hanley JF, Copley JB, et al. Improving house staff ordering of three common laboratory tests: reductions in test ordering need not result in underutilization. Med Care. 1987;25:928-935.
94. Larochelle MR, Knight AM, Pantle H, et al. Reducing excess cardiac biomarker testing at an academic medical center. J Gen Intern Med. 2014;29:1468-1474.
95. Lum G. Evaluation of a protocol to control utilization of B-type natriuretic peptide testing. Am J Clin Pathol. 2006;126:190-194.
96. MacPherson RD, Reeve SA, Stewart TV, et al. Effective strategy to guide pathology test ordering in surgical patients. Anz J Surg. 2005;75:138-143.
97. McNicoll L, Butterfield K, Caliendo A, et al. Vampire medicine: do we need all those blood tests? J Am Geriatr Soc. 2015;63:S213.
98. Minerowicz C, Abel N, Hunter K, et al. Impact of weekly feedback on test ordering patterns. Am J Manag Care. 2015;21:763-768.
99. Newman DB, Siontis KC, Chandrasekaran K, et al. Intervention to reduce inappropriate ionized calcium ordering practices: a quality-improvement project. Perm J. 2015;19:49-51.
100. Riley JD, Procop GW, Kottke-Marchant K, et al. Improving molecular genetic test utilization through order restriction, test review, and guidance. J Mol Diagn. 2015;17:225-229.
101. Roggeman N, Aronson C, Netols C. Review of red blood cell (RBC) irradiation requests results in decreased utilization. Transfusion. 2014;54:246A-247A.
102. Rosenbloom ST, Chiu KW, Byrne DW, et al. Interventions to regulate ordering of serum magnesium levels: report of an unintended consequence of decision support. J Am Med Inform Assoc. 2005;12:546-553.
103. Samuelson BT, Glynn E, Holmes M, et al. Use of a computer-based provider order entry (CPOE) intervention to optimize laboratory testing in patients with suspected heparin-induced thrombocytopenia. Thromb Res. 2015;136:928-931.
104. Spiegel JS, Shapiro MF, Berman B, et al. Changing physician test ordering in a university hospital: an intervention of physician participation, explicit criteria, and feedback. Arch Intern Med. 1989;149:549-553.
105. Thomas RE, Croal B, Ramsay C. Reducing inappropriate laboratory testing through provider education and feedback. J Clin Outcomes Manage. 2006;13:472-477.
106. Tomlin A, Dovey S, Gauld R, et al. Better use of primary care laboratory services following interventions to "market" clinical guidelines in New Zealand: a controlled before-and-after study. BMJ Qual Saf. 2011;20:282-290.
107. Vegting IL, van Beneden M, Kramer MH, et al. How to save costs by reducing unnecessary testing: lean thinking in clinical practice. Eur J Intern Med. 2012;23:70-75.
108. Vidyarthi AR, Hamill T, Green AL, et al. Changing resident test ordering behavior: a multilevel intervention to decrease laboratory utilization at an academic medical center. Am J Med Qual. 2015;30:81-87.
109. Wang TJ, Mort EA, Nordberg P, et al. A utilization management intervention to reduce unnecessary testing in the coronary care unit. Arch Intern Med. 2002;162:1885-1890.
110. Warren JS. Laboratory test utilization program: structure and impact in a large academic medical center. Am J Clin Pathol. 2013;139:289-97.
111. White P, Kenton K. Use of electronic medical record-based tools to improve compliance with cervical cancer screening guidelines: effect of an educational intervention on physicians' practice patterns. J Low Genit Tract Dis. 2013;17:175-181.
112. Alonso-Coello P, Schunemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices, 1: introduction. BMJ. 2016;353:i2016.
113. Bosse HM, Mohr J, Buss B, et al. The benefit of repetitive skills training and frequency of expert feedback in the early acquisition of procedural skills. BMC Med Educ. 2015;15:22.
114. Moulaert V, Verwijnen MG, Rikers R, et al. The effects of deliberate practice in undergraduate medical education. Med Educ. 2004;38:1044-1052.
115. Crawford JM, Shotorbani K, Sharma G, et al. Improving American healthcare through "clinical lab 2. 0": a Project Santa Fe report. Acad Pathol. 2017;4:2374289517701067.
116. Brunetti M, Pregno S, Schünemann H, et al. Economic evidence in decision-making process in laboratory medicine. Clin Chem Lab Med. 2011;49:617-621.
117. Husereau D, Drummond M, Petrou S, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)-explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health. 2013;16:231-250.
118. Bruns DE. Laboratory-related outcomes in healthcare. Clin Chem. 2001;47:1547-1552.
119. Shirts BH, Jackson BR, Baird GS, et al. Clinical laboratory analytics: challenges and promise for an emerging discipline. J Pathol Inform. 2015;6:9.
120. Thornbury JR, Fryback DG. Technology assessment: an American view. Eur J Radiol. 1992;14:147-156.
121. Staub LP, Lord SJ, Simes RJ, et al. Using patient management as a surrogate for patient health outcomes in diagnostic test evaluation. bmc Med Res Methodol. 2012;12:12.
122. Staub LP, Dyer S, Lord SJ, et al. Linking the evidence: intermediate outcomes in medical test assessments. Int J Technol Assess Health Care. 2012;28:52-58.
123. Pearl WS. A hierarchical outcomes approach to test assessment. Ann Emerg Med. 1999;33:77-84.
124. Epner PL, Gans JE, Graber ML. When diagnostic testing leads to harm: a new outcomes-based approach for laboratory medicine. BMJ Qual Saf. 2013;22 (suppl 2):ii6-ii10.
125. Ferrante di Ruffano L, Hyde CJ, McCaffery KJ, et al. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ. 2012;344:e686.
126. Horvath AR, Pewsner D. Systematic reviews in laboratory medicine: principles, processes and practical considerations. Clin Chim Acta. 2004;342:23-39.
127. Barth JH. Selecting clinical quality indicators for laboratory medicine. Ann Clin Biochem. 2012;49:257-261.
128. Cappelletti P. Appropriateness of diagnostics tests. Int J Lab Hematol. 2016;38 (suppl 1):91-99.
129. Alonso-Cerezo MC, Martín JS, García Montes MA, et al. Appropriate utilization of clinical laboratory tests. Clin Chem Lab Med. 2009;47:1461-1465.
130. Graber ML, Trowbridge R, Myers JS, et al. The next organizational challenge: finding and addressing diagnostic error. Jt Comm J Qual Patient Saf. 2014;40:102-110.
131. Melanson SE. Establishing benchmarks and metrics for utilization management. Clin Chim Acta. 2014;427:127-130.
132. van Walraven C, Naylor CD. Do we know what inappropriate laboratory utilization is? A systematic review of laboratory clinical audits. JAMA. 1998;280:550-558.
133. Price CP. Application of the principles of evidence-based medicine to laboratory medicine. Clin Chim Acta. 2003;333:147-154.
134. Sciacovelli L, Panteghini M, Lippi G, et al. Defining a roadmap for harmonizing quality indicators in laboratory medicine: a consensus statement on behalf of the IFCC working group "laboratory error and patient safety" and EFLM task and finish group "performance specifications for the extra-analytical phases". Clin Chem Lab Med. 2017;55:1478-1488.
135. Shahangian S, Snyder SR. Laboratory medicine quality indicators: a review of the literature. Am J Clin Pathol. 2009;131:418-431.
136. Plebani M. Harmonization in laboratory medicine: the complete picture. Clin Chem Lab Med. 2013;51:741-751.
137. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008;17 (suppl 1):i13-32.
138. Harris AD, McGregor JC, Perencevich EN, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc. 2006;13:16-23.