Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review

Annals of Internal Medicine

Volumn 140, Issue 3, 2004, Pages 189-202

  Whiting, Penny ^a   Rutjes, Anne W S ^b   Reitsma, Johannes B ^b   Glas, Afina S ^b   Bossuyt, Patrick M M ^b   Kleijnen, Jos ^b  

^a UNIVERSITY OF YORK   (United Kingdom)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

BIOSIS; COCHRANE LIBRARY; DEMOGRAPHY; DIAGNOSTIC ACCURACY; DIAGNOSTIC TEST; DISEASE SEVERITY; EMBASE; EVIDENCE BASED MEDICINE; MEDLINE; PREVALENCE; PRIORITY JOURNAL; REVIEW;

EID: 0742318857     PISSN: 00034819     EISSN: None     Source Type: Journal    
DOI: 10.7326/0003-4819-140-3-200402030-00010     Document Type: Review

References (61)

1. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem. 2003;49:7-18. [PMID: 12507954]
2. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia: WB Saunders Co; 1985.
3. Whiting PJ, Rutjes AWS, Dinnes J, Reitsma JB, Bossuyt PMM, Kleijnen J. The development and validation of methods for assessing the quality and reporting of diagnostic studies. Health Technol Assess [In press].
4. Curtin F, Morabia A, Pichard C, Slosman DO. Body mass index compared to dual-energy x-ray absorptiometry: evidence for a spectrum bias. J Clin Epidemiol. 1997;50:837-43. [PMID: 9253396]
5. Detrano R, Janosi A, Lyons KP, Marcondes G, Abbassi N, Froelicher VF. Factors affecting sensitivity and specificity of a diagnostic test: the exercise thallium scintigram. Am J Med. 1988;84:699-710. [PMID: 3041808]
6. Detrano R, Lyons KP, Marcondes G, Abbassi N, Froelicher VF, Janosi A. Methodologic problems in exercise testing research. Are we solving them? Arch Intern Med. 1988:148:1289-95. [PMID: 3288157]
7. Detrano R, Gianrossi R, Mulvihill D, Lehmann K, Dubach P, Colombo A, et al. Exercise-induced ST segment depression in the diagnosis of multivessel coronary disease: a meta analysis. J Am Coll Cardiol. 1989;14:1501-8. [PMID: 2809010]
8. Egglin TK, Feinstein AR. Context bias. A problem in diagnostic radiology. JAMA. 1996;276:1752-5. [PMID: 8940325]
9. Hlatky MA, Pryor DB, Harrell FE Jr, Califf RM, Mark DB, Rosati RA. Factors affecting sensitivity and specificity of exercise electrocardiography. Multivariable analysis. Am J Med. 1984;77:64-71. [PMID: 6741986]
10. Lachs MS, Nachamkin I, Edelstein PH, Goldman J, Feinstein AR, Schwartz JS. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection. Ann Intern Med. 1992;117:135-40. [PMID: 1605428]
11. Levy D, Labib SB, Anderson KM, Christiansen JC, Kannel WB, Castelli WP. Determinants of sensitivity and specificity of electrocardiographic criteria for left ventricular hypertrophy. Circulation. 1990;81:815-20. [PMID: 2137733]
12. Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282:1061-6. [PMID: 10493205]
13. Melbye H, Straume B. The spectrum of patients strongly influences the usefulness of diagnostic tests for pneumonia. Scand J Prim Health Care. 1993; 11:241-6. [PMID: 8146507]
14. Moons KG, van Es GA, Deckers JW, Habbema JD, Grobbee DE. Limitations of sensitivity, specificity, likelihood ratio, and Bayes' theorem in assessing diagnostic probabilities: a clinical example. Epidemiology. 1997;8:12-7. [PMID: 9116087]
15. Morise AP, Diamond GA. Does sex discrimination explain the differences in test accuracy among men and women referred for exercise electrocardiography? Circulation. 1994;90(Pt 2):1-273.
16. Morise AP, Diamond GA. Comparison of the sensitivity and specificity of exercise electrocardiography in biased and unbiased populations of men and women. Am Heart J. 1995;130:741-7. [PMID: 7572581]
17. O'Connor PW, Tansay CM, Detsky AS, Mushlin AI, Kucharczyk W. The effect of spectrum bias on the utility of magnetic resonance imaging and evoked potentials in the diagnosis of suspected multiple sclerosis. Neurology. 1996;47: 140-4. [PMID: 8710067]
18. Philbrick JT, Horwitz RI, Feinstein AR, Langou RA, Chandler JP. The limited spectrum of patients studied in exercise test research. Analyzing the tip of the iceberg. JAMA. 1982;248:2467-70. [PMID: 7131702]
19. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299:926-30. [PMID: 692598]
20. Roger VL, Pellikka PA, Bell MR, Chow CW, Bailey KR, Seward JB. Sex and test verification bias. Impact on the diagnostic value of exercise echocardiography. Circulation. 1997;95:405-10. [PMID: 9008457]
21. Rozanski A, Diamond GA, Berman D, Forrester JS, Morris D, Swan HJ. The declining specificity of exercise radionuclide ventriculography. N Engl J Med. 1983;309:518-22. [PMID: 6877322]
22. Santana-Boado C, Candell-Riera J, Castell-Conesa J, Aguada-Bruix S, Garcia-Burillo A, Canela T, et al. Diagnostic accuracy of technetium-99m-MIBI myocardial SPECT in women and men. J Nucl Med. 1998;39:751-5. [PMID: 9591568]
23. Stein PD, Gottschalk A, Henry JW, Shivkumar K. Stratification of patients according to prior cardiopulmonary disease and probability assessment based on the number of mismatched segmental equivalent perfusion defects. Approaches to strengthen the diagnostic value of ventilation/perfusion lung scans in acute pulmonary embolism. Chest. 1993;104:1461-7. [PMID: 8222807]
24. Steinbauer JR, Cantor SB, Holzer CE 3rd, Volk RJ. Ethnic and sex bias in primary care screening tests for alcohol use disorders. Ann Intern Med. 1998;129: 353-62. [PMID: 9735062]
25. Taube A, Tholander B. Over- and underestimation of the sensitivity of a diagnostic malignancy test due to various selections of the study population. Acta Oncol. 1990;29:971-6. [PMID: 2278729]
26. van der Schouw YT, Van Dijk R, Verbeek AL. Problems in selecting the adequate patient population from existing data files for assessment studies of new diagnostic tests. J Clin Epidemiol. 1995;48:417-22. [PMID: 7897462]
27. van Rijkom HM, Verdonschot EH. Factors involved in validity measurements of diagnostic tests for approximal caries-a meta-analysis. Caries Res. 1995;29:364-70. [PMID: 8521438]
28. Froelicher VF, Lehmann KG, Thomas R, Goldman S, Morrison D, Edson R, et al. The electrocardiographic exercise test in a population with reduced workup bias: diagnostic performance, computerized interpretation, and multivariable prediction. Veterans Affairs Cooperative Study in Health Services #016 (QUEXTA) Study Group. Quantitative Exercise Testing and Angiography. Ann Intern Med. 1998;128:965-74. [PMID: 9625682]
29. Arana GW, Zarzar MN, Baker E. The effect of diagnostic methodology on the sensitivity of the TRH stimulation test for depression: a literature review. Biol psychiatry. 1990;28:733-7. [PMID: 2122917]
30. Bowler JV, Munoz DG, Merskey H, Hachinski V. Fallacies in the pathological confirmation of the diagnosis of Alzheimer's disease. J Neurol Neurosurg psychiatry. 1998;64:18-24. [PMID: 9436722]
31. Boyko EJ, Alderman BW, Baron AE. Reference test errors bias the evaluation ofdiagnostic tests for ischemic heart disease. J Gen Intern Med. 1988;3:476-81. [PMID: 3049969]
32. Cecil MP, Kosinski AS, Jones MT, Taylor A, Alazraki NP, Pettigrew RI, et al. The importance of work-up (verification) bias correction in assessing the accuracy of SPECT thallium-201 testing for the diagnosis of coronary artery disease. J Clin Epidemiol. 1996;49:735-42. [PMID: 8691222]
33. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-15. [PMID: 6871349]
34. De Neef P. Evaluating rapid tests for streptococcal pharyngitis: the apparent accuracy of a diagnostic test when there are errors in the standard of comparison. Med Decis Making. 1987;7:92-6. [PMID: 3553828]
35. Diamond GA. Affirmative actions: can the discriminant accuracy of a test be determined in the face of selection bias? Med Decis Making. 1991;11:48-56. [PMID: 2034075]
36. Diamond GA. Off Bayes: effect of verification bias on posterior probabilities calculated using Bayes' theorem. Med Decis Making. 1992;12:22-31. [PMID: 1538629]
37. Lijmer JG, Hunink MG, van den Dungen JJ, Loonstra J, Smit AJ. ROC analysis of noninvasive tests for peripheral arterial disease. Ultrasound Med Biol. 1996;22:391-8. [PMID: 8795165]
38. Miller TD, Hodge DO, Christian TF, Milavetz JJ, Bailey KR, Gibbons RJ. The impact of adjusting for post-test referral bias on apparent sensitivity and specificity of SPECT myocardial perfusion imaging in men and women [Abstract]. J Am Coll Cardiol. 1998;31(2 Suppl A):167A.
39. Mol BW, Lijmer JG, van der Meulen J, Pajkrt E, Bilardo CM, Bossuyt PM. Effect of study design on the association between nuchal translucency measurement and Down syndrome. Obstet Gynecol. 1999;94:864-9. [PMID: 10546775]
40. Panzer RJ, Suchman AL, Griner PF. Workup bias in prediction research. Med Decis Making. 1987;7:115-9. [PMID: 3574021]
41. Phelps CE, Hutson A. Estimating diagnostic test accuracy using a "fuzzy gold standard." Med Decis Making. 1995;15:44-57. [PMID: 7898298]
42. Ransohoff DF, Muir WA. Diagnostic workup bias in the evaluation of a test. Serum ferritin and hereditary hemochromatosis. Med Decis Making. 1982;2: 139-45. [PMID: 7167042]
43. Thibodeau L. Evaluating diagnostic tests. Biometrics. 1981:801-4.
44. Zhou XH. Effect of verification bias on positive and negative predictive values. Stat Med. 1994;13:1737-45. [PMID: 7997707]
45. Detrano R, Gianrossi R, Froelicher V. The diagnostic accuracy of the exercise electrocardiogram: a meta-analysis of 22 years of research. Prog Cardiovasc Dis. 1989;32:173-206. [PMID: 2530605]
46. Berbaum KS, el-Khoury GY, Franken EA Jr, Kathol M, Montgomery WJ, Hesson W. Impact of clinical history on fracture detection with radiography. Radiology. 1988;168:507-11. [PMID: 3393672]
47. Berbaum KS, Franken EA Jr, el-Khoury GY. Impact of clinical history on radiographic detection of fractures: a comparison of radiologists and orthopedists. AJR Am J Roentgenol. 1989;153:1221-4. [PMID: 2816635]
48. Ciccone G, Vineis P, Frigerio A, Segnan N. Inter-observer and intraobserver variability of mammogram interpretation: a field study. Eur J Cancer. 1992;28A:1054-8. [PMID: 1627374]
49. Cohen MB, Rodgers RP, Hales MS, Gonzales JM, Ljung BM, Beckstead JH, et al. Influence of training and experience in fine-needle aspiration biopsy of breast. Receiver operating characteristics curve analysis. Arch Pathol Lab Med. 1987;111:518-20. [PMID: 3579506]
50. Corley DE, Kirtland SH, Winterbauer RH, Hammar SP, Dail DH, Bauermeister DE, et al. Reproducibility of the histologic diagnosis of pneumonia among a panel of four pathologists: analysis of a gold standard. Chest. 1997;112: 458-65. [PMID: 9266884]
51. Cuaron A, Acero AP, Cardenas M, Huerta D, Rodriguez A, de Garay R. Interobserver variability in the interpretation of myocardial images with Tc-99m-labeled diphosphonate and pyrophosphate. J Nucl Med. 1980;21:1-9. [PMID: 6243350]
52. Doubilet P, Herman PG. Interpretation of radiographs: effect of clinical history. AJR Am J Roentgenol. 1981;137:1055-8. [PMID: 6975000]
53. Eldevik OP, Dugstad G, Orrison WW, Haughton VM. The effect of clinical bias on the interpretation of myelography and spinal computed tomography. Radiology. 1982;145:85-9. [PMID: 7122902]
54. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists' interpretations of mammograms. N Engl J Med. 1994;331:1493-9. [PMID: 7969300]
55. Elmore JG, Wells CK, Howard DH, Feinstein AR. The impact of clinical history on mammographic interpretations. JAMA. 1997;277:49-52. [PMID: 898O210]
56. Good BC, Cooperstein LA, DeMarino GB, Miketic LM, Gennari RC, Rockette HE, et al. Does knowledge of the clinical history affect the accuracy of chest radiograph interpretation? AJR Am J Roentgenol. 1990;154:709-12. [PMID: 2107662]
57. Potchen E, Gard J, Lazar P, Lahaie P, Andary M. The effect of clinical history data on chest film interpretation: direction or distraction [Abstract]. Invest Radiol. 1979;14:404.
58. Raab SS, Thomas PA, Lenel JC, Bottles K, Fitzsimmons KM, Zaleski MS, et al. Pathology and probability. Likelihood ratios and receiver operating characteristic curves in the interpretation of bronchial brush specimens. Am J Clin Pathol. 1995;103:588-93. [PMID: 7741104]
59. Raab SS, Oweity T, Hughes JH, Salomao DR, Kelley CM, Flynn CM, et al. Effect of clinical history on diagnostic accuracy in the cytologic interpretation of bronchial brush specimens. Am J Clin Pathol. 2000;114:78-83. [PMID: 10884802]
60. Ronco G, Montanari G, Aimone V, Parisio F, Segnan N, Valle A, et al. Estimating the sensitivity of cervical cytology: errors of interpretation and test limitations. Cytopathology. 1996;7:151-8. [PMID: 8782987]
61. Schreiber M. The clinical history as a factor in roentgenogram interpretation. JAMA. 1963;185:137-9.