Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 study

Medical Image Analysis

Volumn 14, Issue 6, 2010, Pages 707-722

  van Ginneken, Bram ^a,b   Armato, Samuel G ^c   de Hoop, Bartjan ^d   van Amelsvoort van de Vorst, Saskia ^d   Duindam, Thomas ^a   Niemeijer, Meindert ^a   Murphy, Keelin ^a   Schilham, Arnold ^a   Retico, Alessandra ^e   Fantacci, Maria Evelina ^e,f   Camarlinghi, Niccol ^e,f   Bagagli, Francesco ^e,f   Gori, Ilaria ^e,g   Hara, Takeshi ^h   Fujita, Hiroshi ^h   Gargano, Gianfranco ^i,j   Bellotti, Roberto ^i,j   Tangaro, Sabina ^j   Bolaos, Lourdes ^k,l   Carlo, Francesco De ^j   Cerello, Piergiorgio ^k   Cristian Cheran, Sorin ^k   Lopez Torres, Ernesto ^l   Prokop, Mathias ^b,d   more..

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^b RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

^c UNIVERSITY OF CHICAGO   (United States)

^d UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^e INFN SEZIONE DI PISA   (Italy)

^f UNIVERSITY OF PISA   (Italy)

^g Bracco Imaging SpA   (Italy)

^h GIFU UNIVERSITY GRADUATE SCHOOL OF MEDICINE   (Japan)

ⁱ Dipartimento Interateneo M Merlin dellUniverisit degli Studi di Bari   (Italy)

^j INFN SEZIONE DI BARI   (Italy)

^k SEZIONE DI TORINO   (Italy)

^l NONE

more..

Author keywords

Computed tomography; Computer aided detection; Lung cancer; Lung nodules

Indexed keywords

AUTOMATIC DETECTION; COMMERCIAL SYSTEMS; COMPARATIVE STUDIES; COMPUTED TOMOGRAPHY; COMPUTED TOMOGRAPHY SCAN; COMPUTER-AIDED DETECTION; DATA SETS; LUNG CANCER; LUNG CANCER SCREENING; LUNG NODULE; MULTIPLE SYSTEMS; NODULE DETECTION; OBJECTIVE EVALUATION; PERFORMANCE IMPROVEMENTS; PULMONARY NODULES; WEB-BASED FRAMEWORK;

BIOLOGICAL ORGANS; COMPUTERIZED TOMOGRAPHY;

ALGORITHMS;

ARTICLE; CANCER DIAGNOSIS; CANCER SCREENING; COMPUTER ASSISTED TOMOGRAPHY; LUNG CANCER; LUNG NODULE; PRIORITY JOURNAL;

AGED; ALGORITHMS; FEMALE; HUMANS; LUNG NEOPLASMS; MALE; MIDDLE AGED; RADIOGRAPHIC IMAGE INTERPRETATION, COMPUTER-ASSISTED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SOFTWARE; SOFTWARE VALIDATION; SOLITARY PULMONARY NODULE; TOMOGRAPHY, X-RAY COMPUTED;

EID: 77954657504     PISSN: 13618415     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.media.2010.05.005     Document Type: Article

References (70)

1. Arimura H., Katsuragawa S., Suzuki K., Li F., Shiraishi J., Sone S., Doi K. Computerized scheme for automated detection of lung nodules in low-dose computed tomography images for lung cancer screening. Academic Radiology 2004, 11(6):617-629.
2. Armato S.G., Giger M.L., MacMahon H. Automated detection of lung nodules in CT scans: preliminary results. Medical Physics 2001, 28(8):1552-1561.
3. Armato S.G., Li F., Giger M.L., MacMahon H., Sone S., Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 2002, 225(3):685-692.
4. Armato S.G., McLennan G., McNitt-Gray M.F., Meyer C.R., Yankelevitz D., Aberle D.R., Henschke C.I., Hoffman E.A., Kazerooni E.A., MacMahon H., Reeves A.P., Croft B.Y., Clarke L.P. Lung image database consortium: developing a resource for the medical imaging research community. Radiology 2004, 232(3):739-748.
5. Armato S.G., McNitt-Gray M.F., Reeves A.P., Meyer C.R., McLennan G., Aberle D.R., Kazerooni E.A., MacMahon H., van Beek E.J.R., Yankelevitz D., Hoffman E.A., Henschke C.I., Roberts R.Y., Brown M.S., Engelmann R.M., Pais R.C., Piker C.W., Qing D., Kocherginsky M., Croft B.Y., Clarke L.P. The lung image database consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans. Academic Radiology 2007, 14(11):1409-1421.
6. Armato S.G., Roberts R.Y., Kocherginsky M., Aberle D.R., Kazerooni E.A., Macmahon H., van Beek E.J.R., Yankelevitz D., McLennan G., McNitt-Gray M.F., Meyer C.R., Reeves A.P., Caligiuri P., Quint L.E., Sundaram B., Croft B.Y., Clarke L.P. Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of " truth" Academic Radiology 2009, 16(1):28-38.
7. Austin J.H., Mller N.L., Friedman P.J., Hansell D.M., Naidich D.P., Remy-Jardin M., Webb W.R., Zerhouni E.A. Glossary of terms for CT of the lungs: recommendations of the nomenclature committee of the Fleischner society. Radiology 1996, 200(2):327-331.
8. Bae K.T., Kim J.S., Na Y.H., Kim K.G., Kim J.H. Pulmonary nodules: automated detection on CT images with morphologic matching algorithm - preliminary results. Radiology 2005, 236:286-294.
9. Bellotti R., De Carlo F., Gargano G., Tangaro S., Cascio D., Catanzariti E., Cerello P., Cheran S.C., Delogu P., De Mitri I., Fulcheri C., Grosso D., Retico A., Squarcia S., Tommasi E., Golosio Bruno A CAD system for nodule detection in low-dose lung CTs based on region growing and a new active contour model. Medical Physics 2007, 34(12):4901-4910.
10. Bellotti R., Cerello P., Tangaro S., Bevilacqua V., Castellano M., Mastronardi G., De Carlo F., Bagnasco S., Bottigli U., Cataldo R., Catanzariti E., Cheran S.C., Delogu P., De Mitri I., De Nunzio G., Fantacci M.E., Fauci F., Gargano G., Golosio B., Indovina P.L., Lauria A., Lopez Torres E., Magro R., Masala G.L., Massafra R., Oliva P., Preite Martinez A., Quarta M., Raso G., Retico A., Sitta M., Stumbo S., Tata A., Squarcia S., Schenone A., Molinari E., Canesi B. Distributed medical images analysis on a grid infrastructure. Future Generation Computer Systems 2007, 23(3):475-484.
11. Brown M.S., Goldin J.G., Suh R.D., McNitt-Gray M.F., Sayre J.W., Aberle D.R. Lung micronodules: automated method for detection at thin-section CT-initial experience. Radiology 2003, 226:256-262.
12. Buscema, P.M., 2004. A method for encoding image pixels, a method for processing images and a method for processing images aimed at qualitative recognition of the object reproduced by one or more image pixels. Patent EP1483721.
13. Cerello, P., Cheran, S.C., Bagagli, F., Bagnasco, S., Bellotti, R., Bolanos, L., Catanzariti, E., De Nunzio, G., Fiorina, E., Gargano, G., Gemme, G., Lopez Torres, E., Masala, G., Peroni, C., Santoro, M., 2008. The channeler ant model: object segmentation with virtual ant colonies. In: IEEE Nuclear Science Symposium, pp. 3147-3152.
14. Das M., Mhlenbruch G., Mahnken A.H., Flohr T.G., Gndel L., Stanzel S., Kraus T., Gnther R.W., Wildberger J.E. Small pulmonary nodules: effect of two computer-aided detection systems on radiologist performance. Radiology 2006, 241(2):564-571.
15. Dehmeshki J., Ye X., Lin X., Valdivieso M., Amin H. Automated detection of lung nodules in CT images using shape-based genetic algorithm. Computerized Medical Imaging and Graphics 2007, 31(6):408-417.
16. de Hoop B., Gietema H., van Ginneken B., Zanen P., Groenewegen G., Prokop M. A comparison of six software packages for evaluation of solid lung nodules using semi-automated volumetry: what is the minimum increase in size to detect growth in repeated CT examinations. European Radiology 2009, 19(4):800-808.
17. Dolejs, M., Kybic, J., 2009. The lung TIME annotated lung nodule dataset and nodule detection framework. In: Proceedings of the SPIE, vol. 7260, pp. 72601U1-72601U8.
18. Enquobahrie A.A., Reeves A.P., Yankelevitz D.F., Henschke C.I. Automated detection of small pulmonary nodules in whole lung CT scans. Academic Radiology 2007, 14:579-593.
19. Farag, A., El-Baz, A., Gimelfarb, G.G., Falk, R., Hushek, S.G., 2004. Automatic detection and recognition of lung abnormalities in helical CT images using deformable templates. In: Medical Image Computing and Computer-Assisted Intervention. Lecture Notes in Computer Science, vol. 3217, pp. 856-864.
20. Fotin, S.V., Reeves, A.P., Yankelevitz, D.F., Henschke, C.I., 2008. The impact of pulmonary nodule size estimation accuracy on the measured performance of automated nodule detection systems. In: Proceedings of the SPIE, vol. 6915, pp. 6915G1-6915G8.
21. Fotin, S.V., Reeves, A.P., Biancardi, A.M., Yankelevitz, D.F., Henschke, C.I., 2009. A multiscale Laplacian of Gaussian filtering approach to automated pulmonary nodule detection from whole-lung low-dose CT scans. In: Proceedings of the SPIE, vol. 7260, pp. 72601Q1-72601Q8.
22. Ge Z., Sahiner B., Chan H., Hadjiiski L.M., Cascade P.N., Bogot N., Kazerooni E.A., Wei J., Zhou C. Computer-aided detection of lung nodules: false positive reduction using a 3D gradient field method and 3D ellipsoid fitting. Medical Physics 2005, 32(8):2443-2454.
23. Godoy M.C.B., Cooperberg P.L., Maizlin Z.V., Yuan R., McWilliams A., Lam S., Mayo J.R. Detection sensitivity of a commercial lung nodule CAD system in a series of pathologically proven lung cancers. Journal of Thoracic Imaging 2008, 23(1):1-6.
24. Gohagan J., Marcus P., Fagerstrom R., Pinsky P., Kramer B., Prorok P. Baseline findings of a randomized feasibility trial of lung cancer screening with spiral CT scan vs. chest radiograph: the lung screening study of the national cancer institute. Chest 2004, 126(1):114-121.
25. Gori, I., Mattiuzzi, M., 2008. Method for coding pixels or voxels of a digital image and a method for processing digital images. Patent EP1880364.
26. Gori I., Bagagli F., Fantacci M.E., Preite Martinez A., Retico A., De Mitri I., Donadio S., Fulcheri C., Gargano G., Magro R., Santoro M., Stumbo S. Multi-scale analysis of lung computed tomography images. Journal of Instrumentation 2007, 2(09):P09007.
27. Gori, I., Fantacci, M.E., Preite Martinez, A., Retico, A., 2007b. An automated system for lung nodule detection in low-dose computed tomography. In: Proceedings of the SPIE, vol. 6514, pp. 65143R1-65143R9.
28. Gori, I., Bagagli, F., Camarlinghi, N., Fantacci, M.E., Retico, A., Barattini, M., Bolanos, L., Falaschi, F., Gargano, G., Massafra, A., Spinelli, C., 2009. Methodology for automated detection of parenchymal and juxtapleural lung nodules in computed tomography images. In: Proceedings of CARS.
29. Gruden J.F., Ouanounou S., Tigges S., Norris S.D., Klausner T.S. Incremental benefit of maximum-intensity-projection images on observer detection of small pulmonary nodules revealed by multidetector CT. American Journal of Roentgenology 2002, 179(1):149-157.
30. Heimann T., van Ginneken B., Styner M., Arzhaeva Y., Aurich V., Bauer C., Beck A., Becker C., Beichel R., Bekes G., Bello F., Binnig G., Bischof H., Bornik A., Cashman P., Chi Y., Cordova A., Dawant B., Fidrich M., Furst J., Furukawa D., Grenacher L., Hornegger J., Kainmuller D., Kitney R., Kobatake H., Lamecker H., Lange T., Lee J., Lennon B., Li R., Li S., Meinzer H-P., Nemeth G., Raicu D., Rau A-M., van Rikxoort E., Rousson M., Rusko L., Saddi K., Schmidt G., Seghers D., Shimizu A., Slagmolen P., Sorantin E., Soza G., Susomboon R., Waite J., Wimmer A., Wolf I. Comparison and evaluation of methods for liver segmentation from CT datasets. IEEE Transactions on Medical Imaging 2009, 28(8):1251-1265.
31. Henschke, C.I., the International Early Lung Cancer Action Program Investigators, 2007. Survival of patients with clinical stage I lung cancer diagnosed by computed tomography screening for lung cancer. Clinical Cancer Research 13 (17), 4949-4950.
32. Henschke C.I., Yankelevitz D.F., Naidich D.P., McCauley D.I., McGuinness G., Libby D.M., Smith J.P., Pasmantier M.W., Miettinen O.S. CT screening for lung cancer: suspiciousness of nodules according to size on baseline scans. Radiology 2004, 231(1):164-168.
33. Infante M., Cavuto S., Romano Lutman F., Brambilla G., Chiesa G., Ceresoli G., Passera E., Angeli E., Chiarenza M., Aranzulla G., Cariboni U., Errico V., Inzirillo F., Bottoni E., Voulaz E., Alloisio M., Destro A., Roncalli M., Santoro A., Ravasi A randomized study of lung cancer screening with spiral CT (the Dante trial): three-year results. American Journal of Respiratory and Critical Care Medicine 2009, (June 11. Epub ahead of print).
34. Kittler J., Hatef M., Duin R.P.W., Matas J. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20(3):226-239.
35. Ko J.P., Betke M. Chest CT: automated nodule detection and assessment of change over time-preliminary experience. Radiology 2001, 218(1):267-273.
36. Kostis W.J., Reeves A.P., Yankelevitz D.F., Henschke C.I. Three-dimensional segmentation and growth rate estimation of small pulmonary nodules in helical CT images. IEEE Transactions on Medical Imaging 2003, 22(10):1259-1274.
37. Kung J.W., Matsumoto S., Hasegawa I., Nguyen B., Toto L.C., Kundel H., Hatabu H. Mixture distribution analysis of a computer assisted diagnostic method for the evaluation of pulmonary nodules on computed tomography scan. Academic Radiology 2004, 11(3):281-285.
38. Lee Y., Hara T., Fujita H., Itoh S., Ishigaki T. Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique. IEEE Transactions on Medical Imaging 2001, 20(7):595-604.
39. Lee I.J., Gamsu G., Czum J., Wu N., Johnson R., Chakrapani S. Lung nodule detection on chest CT: evaluation of a computer-aided detection (CAD) system. Korean Journal of Radiology 2005, 6(2):89-93.
40. Li Q. Recent progress in computer-aided diagnosis of lung nodules on thin-section CT. Computerized Medical Imaging and Graphics 2007, 31(4-5):248-257.
41. Li Q., Sone S., Doi K. Selective enhancement filters for nodules, vessels, and airway walls in two- and three-dimensional CT scans. Medical Physics 2003, 30(8):2040-2051.
42. Li Q., Li F., Doi K. Computerized detection of lung nodules in thin-section CT images by use of selective enhancement filters and an automated rule-based classifier. Academic Radiology 2008, 15(2):165-175.
43. MacMahon H., Austin J.H.M., Gamsu G., Herold C.J., Jett J.R., Naidich D.P., Patz E.F., Swensen S.J. Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner society. Radiology 2005, 237(2):395-400. the Fleischner Society.
44. Matsumoto S., Kundel H.L., Gee J.C., Gefter W.B., Hatabu H. Pulmonary nodule detection in CT images with quantized convergence index filter. Medical Image Analysis 2006, 10(3):343-352.
45. McCulloch C.C., Kaucic R.A., Mendona P.R.S., Walter D.J., Avila R.S. Model-based detection of lung nodules in computed tomography exams. Academic Radiology 2004, 11(3):258-266.
46. Mendona, P.R.S., Bhotika, R., Zhao, F., Miller, J.V., 2007. Lung nodule detection via Bayesian voxel labeling. In: Information Processing in Medical Imaging. Lecture Notes in Computer Science, vol. 4584, pp. 134-146.
47. Murphy, K., Schilham, A.M.R., Gietema, H., Prokop, M., van Ginneken, B., 2007. Automated detection of pulmonary nodules from low-dose computed tomography scans using a two-stage classification system based on local image features. In: Proceedings of the SPIE, vol. 6514, pp. 651410-1-651410-12.
48. Murphy K., van Ginneken B., Schilham A.M.R., de Hoop B.J., Gietema H.A., Prokop M. A large scale evaluation of automatic pulmonary nodule detection in chest CT using local image features and k-nearest-neighbour classification. Medical Image Analysis 2009, 13:757-770.
49. Novak, C.L., Shen, H., Odry, B.L., Ko, J.P., Naidich, D.P., 2004. A system for automatic detection of lung nodules exhibiting growth. In: Proceedings of the SPIE, vol. 5370, pp. 10-21.
50. Ochs, R., Kimb, H.J., Angel, E., Panknin, C., McNitt-Gray, M., Brown, M., 2007. Forming a reference standard from LIDC data: impact of reader agreement on reported CAD performance. In: Proceedings of the SPIE, vol. 6514, pp. 65142A-1-65142A-6.
51. Receiver Operating Characteristic Analysis in Medical Imaging. Journal of the ICRU 8.
52. Osman O., Ozekes S., Ucan O.N. Lung nodule diagnosis using 3D template matching. Computers in Biology and Medicine 2007, 37(8):1167-1172.
53. Paik D.S., Beaulieu C.F., Rubin G.D., Acar B., Jeffrey R.B., Yee J., Dey J., Napel S. Surface normal overlap: a computer-aided detection algorithm with application to colonic polyps and lung nodules in helical CT. IEEE Transactions on Medical Imaging 2004, 23(6):661-675.
54. Pegna A.L., Picozzi G., Mascalchi M., Carozzi F.M., Carrozzi L., Comin C., Spinelli C., Falaschi F., Grazzini M., Innocenti F., Ronchi C., Paci E. Design, recruitment and baseline results of the ITALUNG trial for lung cancer screening with low-dose CT. Lung Cancer 2009, 64(1):34-40.
55. Peloschek P., Sailer J., Weber M., Herold C.J., Prokop M., Schaefer-Prokop C.M. Pulmonary nodules: sensitivity of maximum intensity projection versus that of volume rendering of 3D multidetector CT data. Radiology 2007, 243(2):561-569.
56. Retico A., Delogu P., Fantacci M.E., Gori I., Preite Martinez A. Lung nodule detection in low-dose and thin-slice computed tomography. Computers in Biology and Medicine 2008, 38(4):525-534.
57. Retico, A., Bagagli, F., Camarlinghi, N., Carpentieri, C., Fantacci, M.E., Gori, I., 2009. A voxel-based neural approach (VBNA) to identify lung nodules in the ANODE09 study. In: Medical Imaging 2009: Computer-Aided Diagnosis, vol. 7260. SPIE, Lake Buena Vista, FL, USA, pp. 72601S-8.
58. Schneider, C., Amjadi, A., Richter, A., Fiebich, M., 2009. Automated lung nodule detection and segmentation. In: Proceedings of the SPIE, vol. 7260, pp. 72601T1-72601T8.
59. Sluimer I.C., Schilham A.M.R., Prokop M., van Ginneken B. Computer analysis of computed tomography scans of the lung: a survey. IEEE Transactions on Medical Imaging 2006, 25(4):385-405.
60. Suzuki K., Armato S.G., Li F., Sone S., Doi K. Massive training artificial neural network (MTANN) for reduction of false positives in computerized detection of lung nodules in low-dose computed tomography. Medical Physics 2003, 30(7):1602-1617.
61. van den Bergh K.A.M., Essink-Bot M.-L., Bunge E.M., Scholten E.Th., Prokop M., van Iersel C.A., van Klaveren R.J., de Koning H.J. Impact of computed tomography screening for lung cancer on participants in a randomized controlled trial (NELSON trial). Cancer 2008, 113(2):396-404.
62. van Iersel C.A., de Koning H.J., Draaisma G., Mali W.P.T.M., Scholten E.Th., Nackaerts K., Prokop M., Habbema J.D.F., Oudkerk M., van Klaveren R.J. Risk-based selection from the general population in a screening trial: selection criteria, recruitment and power for the Dutch-Belgian randomised lung cancer multi-slice CT screening trial (NELSON). International Journal of Cancer 2006, 120(4):868-874.
63. van Rikxoort E.M., de Hoop B., Viergever M.A., Prokop M., van Ginneken B. Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection. Medical Physics 2009, 36(7):2934-2947.
64. Wei, G.-Q., Fan, L., Qian, J., 2002. Automatic detection of nodules attached to vessels in lung CT by volume projection analysis. In: Medical Image Computing and Computer-Assisted Intervention. Lecture Notes in Computer Science, vol. 2488, pp. 746-752.
65. Wiemker, R., Rogalla, P., Zwartkruis, A., Blaffert, T., 2002. Computer aided lung nodule detection on high resolution CT data. In: Proceedings of the SPIE, vol. 4684, pp. 677-688.
66. Wiemker, R., Rogalla, P., Blaffert, T., Sifri, D., Hay, O., Shah, E., Truyen, R., Fleiter, T., 2005. Aspects of computer-aided detection (CAD) and volumetry of pulmonary nodules using multislice CT. British Journal of Radiology 78 (1), S46-S56.
67. Xu D.M., Gietema H., de Koning H., Vernhout R., Nackaerts K., Prokop M., Weenink C., Lammers J., Groen H., Oudkerk M., van Klaveren R. Nodule management protocol of the NELSON randomised lung cancer screening trial. Lung Cancer 2006, 54(2):177-184.
68. Ye, X., Lin, X., Beddoe, G., Dehmeshki, J., 2007. Efficient computer-aided detection of ground-glass opacity nodules in thoracic CT images. In: Proceedings of the 29th Annual International Conference of the IEEE EMBS, vol. 1, pp. 4449-4452.
69. Zhang, X., Stockel, J., Wolf, M., Cathier, P., McLennan, G., Hoffman, E.A., Sonka, M., 2007. A new method for spherical object detection and its application to computer aided detection of pulmonary nodules in CT images. In: Medical Image Computing and Computer-Assisted Intervention. Lecture Notes in Computer Science, vol. 4791, pp. 842-849.
70. Zhao B., Gamsu G., Ginsberg M.S., Jiang L., Schwartz L.H. Automatic detection of small lung nodules on CT utilizing a local density maximum algorithm. Journal of Applied Clinical Medical Physics 2003, 4(3):248-260.