Breast cancer risk estimation with artificial neural networks revisited: Discrimination and calibration

Cancer

Volumn 116, Issue 14, 2010, Pages 3310-3321

  Ayer, Turgay ^a,b   Alagoz, Oguzhan ^a   Chhatwal, Jagpreet ^c   Shavlik, Jude W ^d   Kahn Jr , Charles E ^e   Burnside, Elizabeth S ^a,b  

^a University of Wisconsin Madison   (United States)

^b UNIVERSITY OF WISCONSIN SCHOOL OF MEDICINE AND PUBLIC HEALTH   (United States)

^c MERCK RESEARCH LABORATORIES   (United States)

^d University of Wisconsin Madison   (United States)

^e MEDICAL COLLEGE OF WISCONSIN   (United States)

Author keywords

Breast cancer; Calibration; Computer assisted decisions; Computer assisted diagnosis; Computer assisted radiographic image interpretation; Discrimination; Neural networks; Risk assessment

Indexed keywords

ACCURACY; ADULT; AGED; AREA UNDER THE CURVE; ARTICLE; ARTIFICIAL NEURAL NETWORK; BENIGN TUMOR; BREAST CANCER; CALIBRATION; CANCER RISK; COMPUTER ASSISTED DIAGNOSIS; COMPUTER MODEL; DIAGNOSTIC ACCURACY; EVALUATION; FEMALE; HUMAN; IMAGE ANALYSIS; MAJOR CLINICAL STUDY; MALE; MALIGNANT NEOPLASTIC DISEASE; MAMMOGRAPHY; PREDICTION; PRIORITY JOURNAL; RADIOLOGIST; RECEIVER OPERATING CHARACTERISTIC; RISK ASSESSMENT; SENSITIVITY AND SPECIFICITY;

AGED; BREAST NEOPLASMS; CALIBRATION; DECISION MAKING, COMPUTER-ASSISTED; DIAGNOSIS, COMPUTER-ASSISTED; DISCRIMINATION (PSYCHOLOGY); FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAMMOGRAPHY; MIDDLE AGED; NEURAL NETWORKS (COMPUTER); RISK ASSESSMENT; SENSITIVITY AND SPECIFICITY;

EID: 77954928071     PISSN: 0008543X     EISSN: 10970142     Source Type: Journal    
DOI: 10.1002/cncr.25081     Document Type: Article

References (76)

1. Giger ML. Computer-aided diagnosis in radiology. Acad Radiol. 2002;9:18-25.
2. Kahneman D, Slovic P, Tversky A. Judgment Under Uncertainty: Heuristics and Biases. Cambridge, UK: Cambridge University; 2001.
3. Smith-Bindman R, Chu PW, Miglioretti DL, et al. Comparison of screening mammography in the United States and the United Kingdom. JAMA. 2003;290:2129-2137. (Pubitemid 37432973)
4. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variability in radiologists' interpretations of mammograms. N Engl J Med. 1994;331:1493-1499.
5. Chan EC. Promoting an ethical approach to unproven screening imaging tests. J Am Coll Radiol. 2005;2:311-320.
6. Hillman BJ. Informed and shared decision making: an alternative to the debate over unproven screening tests. J Am Coll Radiol. 2005;2:297-298.
7. Picano E. Informed consent and communication of risk from radiological and nuclear medicine examinations: how to escape from a communication inferno. BMJ. 2004;329: 849-851. (Pubitemid 39388415)
8. Harrell FE Jr. Regression Modeling Strategies. With Applications to Linear Models, Logistic Regression and Survival Analysis. 1st ed. New York, NY: Springer; 2001.
9. Ikeda M, Ishigaki T, Yamauchi K. Relationship between Brier score and area under the binormal ROC curve. Comput Methods Programs Biomed. 2002;67:187-194.
10. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928-935.
11. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008; 54:17-23.
12. Birdwell R, Bandodkar P, Ikeda D. Computer-aided detection with screening mammography in a university hospital setting. Radiology. 2005;236:451-457.
13. Cupples TE, Cunningham JE, Reynolds JC. Impact of computer-aided detection in a regional screening mammography program. AJR Am J Roentgenol. 2005;185:944-950.
14. Dean JC, Ilvento CC. Improved cancer detection using computer-aided detection with diagnostic and screening mammography: prospective study of 104 cancers. AJR Am J Roentgenol. 2006;187:20-28.
15. Fenton JJ, Taplin SH, Carney PA, et al. Influence of computer-aided detection on performance of screening mammography. N Engl J Med. 2007;356:1399-1409.
16. Hadjiiski L, Sahiner B, Helvie MA, et al. Breast masses: computer-aided diagnosis with serial mammograms. Radiology. 2006;240:343-356.
17. Chan HP, Sahiner B, Helvie MA, et al. Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study. Radiology. 1999;212:817-827.
18. Huo Z, Giger ML, Vyborny CJ, Metz CE. Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms. Radiology. 2002;224:560-568. (Pubitemid 34809142)
19. Kallergi M. Computer-aided diagnosis of mammographic microcalcification clusters. Med Phys. 2004;31:314-326.
20. Jiang Y, Nishikawa RM, Schmidt RA, Metz CE. Comparison of independent double readings and computer-aided diagnosis (CAD) for the diagnosis of breast calcifications. Acad Radiol. 2006;13:84-94.
21. Gail MH, Brinton LA, Byar DP, Corle DK, Green SB, Schairer C, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually. J Natl Cancer Inst. 1989;81:1879-1886. (Pubitemid 20026073)
22. Claus EB, Risch N, Thompson WD. Genetic analysis of breast cancer in the cancer and steroid hormone study. Am J Hum Genet. 1991;48:232-242. (Pubitemid 21891587)
23. Claus EB, Risch N, Thompson WD. Autosomal dominant inheritance of early-onset breast cancer. Implications for risk prediction. Cancer. 1994;73:643-651.
24. Barlow WE, White E, Ballard-Barbash R, et al. Prospective breast cancer risk prediction model for women undergoing screening mammography. J Natl Cancer Inst. 2006;98:1204-1214.
25. Freedman AN, Seminara D, Gail MH, et al. Cancer risk prediction models: a workshop on development, evaluation, and application. J Natl Cancer Inst. 2005;97:715-723.
26. Freer T, Ulissey M. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology. 2001;220:781-786. (Pubitemid 32799990)
27. Morton MJ, Whaley DH, Brandt KR, Amrami KK. Screening mammograms: interpretation with computer-aided detection - prospective evaluation. Radiology. 2006;239:375-383.
28. Gur D, Wallace LP, Klym AH, et al. Trends in recall, biopsy, and positive biopsy rates for screening mammography in an academic practice. Radiology. 2005;235:396-401.
29. Vyborny CJ, Giger ML, Nishikawa RM. Computer-aided detection and diagnosis of breast cancer. Radiol Clin North Am. 2000;38:725-740. (Pubitemid 30640062)
30. Baker J, Kornguth P, Lo J, Floyd C Jr. Artificial neural network: improving the quality of breast biopsy recommendations. Radiology. 1996;198:131-135. (Pubitemid 26008254)
31. Baker J, Kornguth P, Lo J, Williford M, Floyd C Jr. Breast cancer: prediction with artificial neural network based on BI-RADS standardized lexicon. Radiology. 1995;196:817-822.
32. Burnside ES. Bayesian networks: computer-assisted diagnosis support in radiology. Acad Radiol. 2005;12:422-430.
33. Burnside ES, Davis J, Chhatwal J, et al. A probabilistic computer model developed from clinical data in the national mammography database format to classify mammography findings. Radiology. 2009;251:663-672.
34. Chhatwal J, Alagoz O, Lindstrom MJ, Kahn CE, Shaffer KA, Burnside ES. A logistic regression model based on the national mammography database format to aid breast cancer diagnosis. AJR Am J Roentgenol. 2009;192:1117-1127.
35. Floyd C Jr, Lo J, Yun A, Sullivan D, Kornguth P. Prediction of breast cancer malignancy using an artificial neural network. Cancer. 1994;74:2944-2948.
36. Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K. Improving breast cancer diagnosis with computer-aided diagnosis. Acad Radiol. 1999;6:22-33.
37. Lo JY, Baker JA, Kornguth PJ, Floyd CE Jr. Computeraided diagnosis of breast cancer: artificial neural network approach for optimized merging of mammographic features. Acad Radiol. 1995;2:841-850.
38. Orr RK. Use of an artificial neural network to quantitate risk of malignancy for abnormal mammograms. Surgery. 2001;129:459-466.
39. Wu Y, Giger M, Doi K, Vyborny C, Schmidt R, Metz C. Artificial neural networks in mammography: application to decision making in the diagnosis of breast cancer. Radiology. 1993;187:81-87. (Pubitemid 23097863)
40. Giger ML, Huo Z, Kupinski MA, Vyborny CJ. Computer-aided diagnosis in mammography. In: Fitzpatrick JM, Milan S. Handbook of Medical Imaging. Vol. 2. Bellingham WA: SPIE; 2000:917-986.
41. Gurney JW. Neural networks at the crossroads: caution ahead. Radiology. 1994;193:28-30.
42. Burnside ES, Rubin DL, Fine JP, Shachter RD, Sisney GA, Leung WK. Bayesian network to predict breast cancer risk of mammographic microcalcifications and reduce number of benign biopsy results: initial experience. Radiology. 2006; 240:666-673.
43. Sickles EA. Periodic mammographic follow-up of probably benign lesions: results in 3,184 consecutive cases. Radiology. 1991;179:463-468.
44. Osuch J, Anthony M, Bassett L, et al. A proposal for a national mammography database: content, purpose, and value. AJR Am J Roentgenol. 1995;164:1329-1334.
45. Breast Imaging Reporting And Data System (BI-RADS), 4th ed. Reston, VA: American College of Radiology; 2004.
46. Rumelhart DE, Hinton EE, Williams RJ. Learning representations by back-propogating errors. Nature. 1986;323:533.
47. Weigend A. On overfitting and the effective number of hidden units. In: Mozer A, Smolensky P, Touretzky DS, Elman JL, Weigend AS, eds. Proceedings of the 1993 Connectionist Models Summer School. Hillsdale, NJ: Lawrence Erlbaum; 1994:335-342.
48. Caruana R, Lawrence S, Giles CL. Overfitting in neural networks: backpropagation, conjugate gradient, and early stopping. In: Leen TK, Dietterich TG, Tresp V, eds. Advances in Neural Information Processing Systems 13. Cambridge: Mass: MIT Press; 2001.
49. Lawrence S, Giles CL, Tsoi AC. Lessons in neural network training: overfitting may be harder than expected. In: American Association for Artificial Intelligence. Proceedings of the Fourteenth National Conference on Artificial Intelligence. Menlo Park, Calif: AAAI Press; 1997:540-545.
50. Bishop CM. Neural Networks For Pattern Recognition. New York, NY: Oxford; 1995.
51. Mitchell TM. Machine Learning. Burr Ridge, Ill: McGraw Hill; 1997.
52. Haykin S. Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice Hall; 1998.
53. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837-845.
54. Bassett LW, Hendrick RE, Bassford TL, Butler PF, Carter D, DeBor M; Agency for Health Care Policy and Research. Quality Determinants of Mammography. Clinical Practice Guideline No. 13. Washington, DC: Public Health Service, US Department of Health and Human Services; 1994.
55. Rosner B. Two-sample test for binomial proportions for matched-pair data (McNemar's Test). In: Rosner B, ed. Fundamentals of Biostatistics. Pacific Grove, Calif: Duxbury; 2000:376-384.
56. Agresti A, Coull BA. Approximate is better than exact for interval estimation of binomial proportions. Am Stat. 1998; 52:119-126. (Pubitemid 128487452)
57. Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley; 2000.
58. Poses RM, Cebul RD, Centor RM. Evaluating physicians' probabilistic judgments. Med Decis Making. 1988;8:233-240.
59. Diamond G. What price perfection? Calibration and discrimination of clinical prediction models. J Clin Epidemiol. 1992;45:85-89.
60. Ohno-Machado L. Medical Applications of Artificial Neural Networks: Connectionist Models of Survival [dissertation]. Stanford, Calif: Stanford University; 1996.
61. Harrell FE Jr, Lee KL, Matchar DB, Reichert TA. Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep. 1985;69:1071-1077.
62. Miller RA. Why the standard view is standard: people, not machines, understand patients' problems. J Med Philos. 1990;15:581-591.
63. Hickam DH, Shortliffe EH, Bischoff MB, Scott AC, Jacobs CD. The treatment advice of a computer-based cancer chemotherapy protocol advisor. Ann Intern Med. 1985; 103(6 pt 1):928-936.
64. Masarie FE Jr, Miller RA, Myers JD. INTERNIST-I properties: representing common sense and good medical practice in a computerized medical knowledge base. Comput Biomed Res. 1985;18:458-479.
65. Miller RA, Pople HE Jr, Myers JD. Internist-I, an experimental computer-based diagnostic consultant for general internal medicine. N Engl J Med. 1982;307:468-476.
66. Shortliffe EH. Medical expert systems-knowledge tools for physicians. West J Med. 1986;145:830-839.
67. Shortliffe EH, Davis R, Axline SG, Buchanan BG, Green CC, Cohen SN. Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. Comput Biomed Res. 1975;8:303-320.
68. Yu VL, Buchanan BG, Shortliffe EH, et al. Evaluating the performance of a computer-based consultant. Comput Programs Biomed. 1979;9:95-102.
69. Craven MW, Shavlik JW. Extracting tree-structured representations of trained networks. In: Touretzky DS, Mozer MC, Hasselmo ME, eds. Advances in Neural Information Processing Systems 8. Proceedings of the 1995 Conference. Cambridge, Mass: MIT; 1996:24-30.
70. Nauck D, Klawonn F, Kruse R. Foundations of Neuro-Fuzzy Systems. New York, NY: John Wiley; 1997.
71. Setiono R, Liu H. Symbolic representation of neural networks. Computer. 1996;29:71-77.
72. Vinterbo S. Predictive models in medicine: some methods for construction and adaptation. Trondheim, Norway: Norwegian University of Science and Technology, 1999.
73. Taplin S, Abraham L, Barlow WE, et al. Mammography facility characteristics associated with interpretive accuracy of screening mammography. J Natl Cancer Inst. 2008;100:876-887.
74. Sohlich RE, Sickles EA, Burnside ES, Dee KE. Interpreting data from audits when screening and diagnostic mammography outcomes are combined. AJR Am J Roentgenol. 2002; 178:681-686.
75. Bickell NA, Chassin MR. Determining the quality of breast cancer care: do tumor registries measure up? Ann Intern Med. 2000;132:705-710.
76. Malin JL, Kahn KL, Adams J, Kwan L, Laouri M, Ganz PA. Validity of cancer registry data for measuring the quality of breast cancer care. J Natl Cancer Inst. 2002;94:835-844. (Pubitemid 34705055)