The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans

Medical Physics

Volumn 38, Issue 2, 2011, Pages 915-931

  Armato III, Samuel G ^a   McLennan, Geoffrey ^b   Bidaut, Luc ^c,z   McNitt Gray, Michael F ^d   Meyer, Charles R ^e   Reeves, Anthony P ^f   Zhao, Binsheng ^g,aa   Aberle, Denise R ^d   Henschke, Claudia I ^h,ab   Hoffman, Eric A ^b   Kazerooni, Ella A ⁱ   MacMahon, Heber ^a   Van Beek, Edwin J R ^b,ac   Yankelevitz, David ^h,ab   Biancardi, Alberto M ^f   Bland, Peyton H ^e   Brown, Matthew S ^d   Engelmann, Roger M ^a   Laderach, Gary E ^e   Max, Daniel ^h,ab   Pais, Richard C ^d   Qing, David P Y ^c,d   Roberts, Rachael Y ^a,ad   Smith, Amanda R ^b   Starkey, Adam ^a   Batra, Poonam ^d,ae   Caligiuri, Philip ^a,af   Farooqi, Ali ^h,ad   Gladish, Gregory W ^j   Jude, C Matilda ^d   Munden, Reginald F ^j   Petkovska, Iva ^d,ag   Quint, Leslie E ⁱ   Schwartz, Lawrence H ^g,aa   Sundaram, Baskaran ⁱ   Dodd, Lori E ^k,ah   Fenimore, Charles ^l   Gur, David ^m   Petrick, Nicholas ⁿ   Freymann, John ^o   Kirby, Justin ^o   Hughes, Brian ^p   Vande Casteele, Alessi ^q   Gupte, Sangeeta ^r   Sallam, Maha ^s,ai   Heath, Michael D ^t   Kuhn, Michael H ^u   Dharaiya, Ekta ^u   Burns, Richard ^v   Fryd, David S ^v   Salganicoff, Marcos ^w   Anand, Vikram ^w   Shreter, Uri ^x,aj   Vastagh, Stephen ^y   Croft, Barbara Y ^k   Clarke, Laurence P ^k   more..

^a UNIVERSITY OF CHICAGO   (United States)

^b UNIVERSITY OF IOWA   (United States)

^c UNIVERSITY OF TEXAS   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

^f Cornell University   (United States)

^g MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

^h MOUNT SINAI SCHOOL OF MEDICINE   (United States)

ⁱ UNIVERSITY OF MICHIGAN HEALTH SYSTEM   (United States)

^j UNIVERSITY OF TEXAS MD ANDERSON CANCER CENTER   (United States)

^k NATIONAL CANCER INSTITUTE   (United States)

^l NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^m UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

ⁿ CENTER FOR DRUG EVALUATION AND RESEARCH   (United States)

^o SAIC FREDERICK INC   (United States)

^p TerpSys   (United States)

^q AGFA GEVAERT N V   (Belgium)

^r FUJIFILM Medical Systems USA Inc   (United States)

^s ICAD INC   (United States)

^t CARESTREAM HEALTH INC   (United States)

^u PHILIPS HEALTHCARE   (United States)

^v Riverain Medical Group   (United States)

^w SIEMENS CORPORATE RESEARCH   (United States)

^x GE HEALTHCARE   (United States)

^y Medical Imaging and Technology Alliance (MITA)   (United States)

^z UNIVERSITY OF DUNDEE   (United Kingdom)

^aa NewYork Presbyterian Hospital   (United States)

^ab WEILL CORNELL MEDICAL COLLEGE   (United States)

^ac UNIVERSITY OF EDINBURGH   (United Kingdom)

^ad University Medical CenterLSU   (United States)

^ae VETERANS AFFAIRS MEDICAL CENTER   (United States)

^af UNIVERSITY OF UTAH SCHOOL OF MEDICINE   (United States)

^ag Sterile Product Compounding Unit   (United States)

^ah NATIONAL INSTITUTE OF ALLERGY AND INFECTIOUS DISEASES   (United States)

^ai VuEssence Inc   (United States)

^aj ActiViews Inc   (United States)

more..

Author keywords

computed tomography (CT); computer aided diagnosis (CAD); interobserver variability; lung nodule; thoracic imaging

Indexed keywords

BIOLOGICAL ORGANS; CLINICAL RESEARCH; COMPUTER AIDED DIAGNOSIS; DATABASE SYSTEMS; IMAGING SYSTEMS; MEDICAL IMAGING; PUBLIC ADMINISTRATION;

COMPUTER AIDED DIAGNOSIS(CAD); COMPUTER AIDED DIAGNOSTICS; FOOD AND DRUG ADMINISTRATION; INTEROBSERVER VARIABILITY; LUNG NODULE; NATIONAL INSTITUTES OF HEALTH; PUBLIC PRIVATE PARTNERSHIPS; THORACIC IMAGING;

COMPUTERIZED TOMOGRAPHY;

EID: 79551672468     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.3528204     Document Type: Article

References (45)

1. R. M. Nishikawa, " Design of a common database for research in mammogram image analysis.," Proc. SPIE PSISDG 0277-786X 1905, 548-549 (1993). 10.1117/12.148620
2. M. Kallergi, R. A. Clark, and L. P. Clarke, " Medical image databases for CAD applications in digital mammography: Design issues.," Stud. Health Technol. Inform. ZZZZZZ 0926-9630 43, 601-605 (1997). (Pubitemid 128145230)
3. R. M. Nishikawa and L. M. Yarusso, " Variations in measured performance of CAD schemes due to database composition and scoring protocol.," Proc. SPIE PSISDG 0277-786X 3338, 840-844 (1998). 10.1117/12.310894
4. R. M. Nishikawa, " Effect of case selection on the performance of computer-aided detection schemes.," Med. Phys. MPHYA6 0094-2405 21, 265-269 (1999). 10.1118/1.597287
5. K. Bowyer, in " The digital database for screening mammography.," Digital Mammography '96: Proceedings of the Third International Workshop on Digital Mammography, edited by, K. Doi, M. L. Giger, R. M. Nishikawa, and, R. A. Schmidt, (Elsevier, Amsterdam, 1996), pp. 431-434.
6. M. Heath, in " Current status of the digital database for screening mammography.," Digital Mammography '98: Proceedings of the 4th International Workshop on Digital Mammography, edited by, N. Karssemeijer, M. Thijssen, J. Hendriks, and, L. van Erning, (Kluwer, Dordrecht, 1998), pp. 431-434.
7. M. Heath, K. Bowyer, D. Kopans, R. Moore, and W. P. Kegelmeyer, in " The digital database for screening mammography.," Digital Mammography 2000: Proceedings of the Fifth International Workshop on Digital Mammography, edited by, M. J. Yaffe, (Medical Physics, Madison, 2000), pp. 431-434.
8. M. L. Giger, H. -P. Chan, and J. Boone, " Anniversary paper: History and status of CAD and quantitative image analysis: The role of Medical Physics and AAPM.," Med. Phys. MPHYA6 0094-2405 35, 5799-5820 (2008). 10.1118/1.3013555
9. J. Shiraishi, " Development of a digital image database for chest radiographs with and without a lung nodule: Receiver operating characteristic analysis of radiologists' detection of pulmonary nodules.," AJR, Am. J. Roentgenol. AAJRDX 0361-803X 174, 71-74 (2000). (Pubitemid 30030212)
10. A. P. Reeves, " A public image database to support research in computer aided diagnosis.," 31st Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009, pp. 3715-3718.
11. C. R. Meyer, " Quantitative imaging to assess tumor response to therapy: Common themes of measurement, truth data and error sources.," Translational Oncology 2, 198-210 (2009).
12. S. G. Armato III, " The Reference Image Database to Evaluate Response to therapy in lung cancer (RIDER) project: A resource for the development of change analysis software.," Clin. Pharmacol. Ther. ZZZZZZ 0009-9236 84, 448-456 (2008). 10.1038/clpt.2008.161
13. K. W. Clark, " Creation of a CT image library for the Lung Screening Study of the National Lung Screening Trial.," J. Digit Imaging ZZZZZZ 0897-1889 20, 23-31 (2007). 10.1007/s10278-006-0589-5 (Pubitemid 46555359)
14. D. D. Cody, " Normalized CT dose index of the CT scanners used in the National Lung Screening Trial.," AJR, Am. J. Roentgenol. AAJRDX 0361-803X 194, 1539-1546 (2010). 10.2214/AJR.09.3268
15. C. A. van Iersel, " Risk-based selection from the general population in a screening trial: Selection criteria, recruitment and power for the Dutch-Belgian randomised lung cancer multi-slice CT screening trial (NELSON).," Int. J. Cancer IJCNAW 0020-7136 120, 868-874 (2007). 10.1002/ijc.22134 (Pubitemid 46175408)
16. D. M. Xu, " Nodule management protocol of the NELSON randomised lung cancer screening trial.," Lung Cancer ZZZZZZ 0169-5002 54, 177-184 (2006). 10.1016/j.lungcan.2006.08.006 (Pubitemid 44802721)
17. H. A. Gietema, " Pulmonary nodules detected at lung cancer screening: Interobserver variability of semiautomated volume measurements.," Radiology RADLAX 0033-8419 241, 251-257 (2006). 10.1148/radiol.2411050860 (Pubitemid 44433358)
18. D. M. Xu, " Limited value of shape, margin and CT density in the discrimination between benign and malignant screen detected solid pulmonary nodules of the NELSON trial.," Eur. J. Radiol. EJRADR 0720-048X 68, 347-352 (2008). 10.1016/j.ejrad.2007.08.027
19. D. M. Xu, " Role of baseline nodule density and changes in density and nodule features in the discrimination between benign and malignant solid indeterminate pulmonary nodules.," Eur. J. Radiol. EJRADR 0720-048X 70, 492-498 (2009). 10.1016/j.ejrad.2008.02.022
20. K. Murphy, B. van Ginneken, A. M. R. Schilham, B. J. de Hoop, H. A. Gietema, and M. Prokop, " A large-scale evaluation of automatic pulmonary nodule detection in chest CT using local image features and k-nearest-neighbour classification.," Med. Image Anal. MIAECY 1361-8415 13, 757-770 (2009). 10.1016/j.media.2009.07.001
21. B. van Ginneken, " Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 study.," Med. Image Anal. MIAECY 1361-8415 14, 707-722 (2010). 10.1016/j.media.2010.05.005
22. E. M. van Rikxoort, B. de Hoop, M. A. Viergever, M. Prokop, and B. van Ginneken, " Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection.," Med. Phys. MPHYA6 0094-2405 36, 2934-2947 (2009). 10.1118/1.3147146
23. T. Doshi, D. Rusinak, R. A. Halvorsen, D. C. Rockey, K. Suzuki, and A. H. Dachman, " CT colonography: False-negative interpretations.," Radiology RADLAX 0033-8419 244, 165-173 (2007). 10.1148/radiol.2441061122 (Pubitemid 46981971)
24. D. C. Rockey, " Analysis of air contrast barium enema, computed tomographic colonography, and colonoscopy: Prospective comparison.," Lancet LANCAO 0140-6736 365, 305-311 (2005). (Pubitemid 40139107)
25. L. P. Clarke, B. Y. Croft, E. Staab, H. Baker, and D. C. Sullivan, " National Cancer Institute initiative: Lung image database resource for imaging research.," Acad. Radiol. ZZZZZZ 1076-6332 8, 447-450 (2001). 10.1016/S1076-6332(03)80555-X (Pubitemid 32674656)
26. L. E. Dodd, " Assessment methodologies and statistical issues for computer-aided diagnosis of lung nodules in computed tomography: Contemporary research topics relevant to the Lung Image Database Consortium.," Acad. Radiol. ZZZZZZ 1076-6332 11, 462-475 (2004). 10.1016/S1076-6332(03)00814-6 (Pubitemid 38405374)
27. S. G. Armato III, " Lung Image Database Consortium: Developing a resource for the medical imaging research community.," Radiology RADLAX 0033-8419 232, 739-748 (2004). 10.1148/radiol.2323032035 (Pubitemid 39100258)
28. M. C. Carrillo, C. A. Sanders, and R. G. Katz, " Maximizing the Alzheimer's Disease Neuroimaging Initiative II.," Alzheimers and Dementia 1552-5260 5, 271-275 (2009). 10.1016/j.jalz.2009.02.005
29. M. F. McNitt-Gray, " The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation.," Acad. Radiol. ZZZZZZ 1076-6332 14, 1464-1474 (2007). 10.1016/j.acra.2007.07.021 (Pubitemid 350116641)
30. Department of Health and Human Services, " Standards for privacy of individually identifiable health information: Final rules.," Fed. Regist. FEREAC 0097-6326 67, 53182-53272 (2002).
31. J. H. M. Austin, " Glossary of terms for CT of the lungs: Recommendations of the Nomenclature Committee of the Fleischner Society.," Radiology RADLAX 0033-8419 200, 327-331 (1996).
32. S. G. Armato III, " The Lung Image Database Consortium (LIDC): An evaluation of radiologist variability in the identification of lung nodules on CT scans.," Acad. Radiol. ZZZZZZ 1076-6332 14, 1409-1421 (2007). 10.1016/j.acra.2007.07.008 (Pubitemid 47615519)
33. C. R. Meyer, " Evaluation of lung MDCT nodule annotations across radiologists and methods.," Acad. Radiol. ZZZZZZ 1076-6332 13, 1254-1265 (2006). 10.1016/j.acra.2006.07.012 (Pubitemid 44345260)
34. I. Petkovska, " The effect of lung volume on nodule size on CT.," Acad. Radiol. ZZZZZZ 1076-6332 14, 476-485 (2007). 10.1016/j.acra.2007.01.008 (Pubitemid 46401415)
35. N. Otsu, " A threshold selection method from grey level histograms.," IEEE Trans. Syst. Man Cybern. ISYMAW 0018-9472 SMC-9, 62-66 (1979). (Pubitemid 9413341)
36. S. G. Armato III, " The Lung Image Database Consortium (LIDC): Ensuring the integrity of expert-defined "truth"," Acad. Radiol. ZZZZZZ 1076-6332 14, 1455-1463 (2007). 10.1016/j.acra.2007.08.006 (Pubitemid 350116642)
37. A. P. Reeves, " The Lung Image Database Consortium (LIDC): A comparison of different size metrics for pulmonary nodule measurements.," Acad. Radiol. ZZZZZZ 1076-6332 14, 1475-1485 (2007). 10.1016/j.acra.2007.09.005 (Pubitemid 350116645)
38. R. Ochs, H. J. Kim, E. Angel, C. Panknin, M. McNitt-Gray, and M. Brown, " Forming a reference standard from LIDC data: Impact of reader agreement on reported CAD performance.," Proc. SPIE PSISDG 0277-786X 6514, 65142A-1-65142A-6 (2007). 10.1117/12.707916 (Pubitemid 47567603)
39. W. F. Sensakovic, A. Starkey, R. Y. Roberts, and S. G. Armato III, " Discrete space vs. continuous space lesion boundary and area definition.," Med. Phys. MPHYA6 0094-2405 35, 4070-4078 (2008). 10.1118/1.2963989
40. A. M. Biancardi, A. C. Jirapatnakul, and A. P. Reeves, " A comparison of ground truth estimation methods.," International Journal of Computer Assisted Radiology and Surgery 1861-6410 5, 295-305 (2010). 10.1007/s11548-009-0401-3
41. W. H. Horsthemke, D. S. Raicu, and J. D. Furst, " Evaluation challenges for bridging semantic gap: Shape disagreements on pulmonary nodules in the Lung Image Database Consortium.," International Journal of Healthcare Information Systems and Informatics 1555-3396 4, 17-33 (2008).
42. D. Zinovev, D. S. Raicu, J. D. Furst, and S. G. Armato III, " Predicting radiological panel opinions using a panel of machine learning classifiers.," J. Algorithms JOALDV 0196-6774 2, 1473-1502 (2009). 10.3390/a2041473
43. S. G. Armato III, " Assessment of radiologist performance in the detection of lung nodules: Dependence on the definition of "truth", " Acad. Radiol. ZZZZZZ 1076-6332 16, 28-38 (2009). 10.1016/j.acra.2008.05. 022
44. D. L. Rubin, P. Mongkolwat, and D. S. Channin, " A semantic image annotation model to enable integrative translational research.," AMIA Summit on Translational Bioinformatics, San Francisco, CA, 2009.
45. A. J. Buckler, " Volumetric CT in lung cancer: An example for the qualification of imaging as a biomarker.," Acad. Radiol. ZZZZZZ 1076-6332 17, 107-115 (2010). 10.1016/j.acra.2009.06.019