Cloud-Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks

IEEE Transactions on Medical Imaging

Volumn 35, Issue 11, 2016, Pages 2459-2475

  Jimenez Del Toro, Oscar ^a,b   Muller, Henning ^a,b   Krenn, Markus ^d   Gruenberg, Katharina ^e   Taha, Abdel Aziz ^f   Winterstein, Marianne ^e   Eggel, Ivan ^a   Foncubierta Rodriguez, Antonio ^g   Goksel, Orcun ^g   Jakab, Andras ^d   Kontokotsios, Georgios ^f   Langs, Georg ^d   Menze, Bjoern H ^g   Salas Fernandez, Tomas ^g,h   Schaer, Roger ^a   Walleyo, Anna ^e   Weber, Marc Andre ^e   Dicente Cid, Yashin ^a,b   Gass, Tobias ^g   Heinrich, Mattias ⁱ   Jia, Fucang ^j   Kahl, Fredrik ^k   Kechichian, Razmig ^l   Mai, Dominic ^m   Spanier, Assaf B ⁿ   Vincent, Graham ^o   Wang, Chunliang ^p   Wyeth, Daniel ^c   Hanbury, Allan ^f   more..

^a UNIVERSITY OF APPLIED SCIENCES WESTERN SWITZERLAND   (Switzerland)

^b UNIVERSITY OF GENEVA   (Switzerland)

^c Toshiba Medical Visualization Systems Europe Ltd   (United Kingdom)

^d MEDICAL UNIVERSITY OF VIENNA   (Austria)

^e UNIVERSITY HOSPITAL HEIDELBERG   (Germany)

^f VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

^g ETH ZURICH   (Switzerland)

^h Agency for Health Quality and Assessment of Catalonia   (Spain)

ⁱ UNIVERSITY OF LÜBECK   (Germany)

^j INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^k CHALMERS UNIVERSITY OF TECHNOLOGY   (Sweden)

^l UNIVERSITÉ DE LYON   (France)

^m UNIVERSITY OF FREIBURG   (Germany)

ⁿ HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^o IMORPHICS LTD   (United Kingdom)

^p ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

more..

Author keywords

Evaluation framework; landmark detection; organ segmentation

Indexed keywords

BENCHMARKING; COMPUTERIZED TOMOGRAPHY; CONTOUR MEASUREMENT; MAGNETIC LEVITATION VEHICLES; MAGNETIC RESONANCE;

ANATOMICAL STRUCTURE SEGMENTATION; ANATOMICAL STRUCTURES; EVALUATION FRAMEWORK; LANDMARK DETECTION; LANDMARK LOCALIZATION; MEDICAL IMAGING ALGORITHMS; ORGAN SEGMENTATION; PERFORMANCE ANALYSIS;

MEDICAL IMAGING;

ALGORITHM; ANATOMIC LANDMARK; ARTICLE; AUTOMATION; CLOUD COMPUTING; COMPUTER ASSISTED TOMOGRAPHY; CONCEPTUAL FRAMEWORK; EVALUATION STUDY; HUMAN; NUCLEAR MAGNETIC RESONANCE IMAGING; AGED; ANATOMY; DIAGNOSTIC IMAGING; FEMALE; IMAGE PROCESSING; MALE; MIDDLE AGED; PROCEDURES; X-RAY COMPUTED TOMOGRAPHY;

AGED; ALGORITHMS; ANATOMIC LANDMARKS; ANATOMY; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MIDDLE AGED; TOMOGRAPHY, X-RAY COMPUTED;

EID: 84994495874     PISSN: 02780062     EISSN: 1558254X     Source Type: Journal    
DOI: 10.1109/TMI.2016.2578680     Document Type: Article

References (54)

1. K. Marten et al., "Inadequacy of manual measurements compared to automated CT volumetry in assessment of treatment response of pulmonary metastases using RECIST criteria, " Eur. Radiol., vol. 16, no. 4, pp. 781-790, 2006.
2. N. Sharma and L. M. Aggarwal, "Automated medical image segmentation techniques, " J. Med. Phys./Assoc. Med. Phys. India, vol. 35, no. 1, pp. 3-14, 2010.
3. K. Doi, "Current status and future potential of computer-aided diagnosis in medical imaging, " Br. J. Radiol., vol. 78, pp. 3-19, 2005.
4. H. Kobatake, "Future CAD in multi-dimensional medical images: Project on multi-organ, multi-disease CAD system, " Comput. Med. Imag. Graph., vol. 31, nos. 4-5, pp. 258-266, 2007.
5. H. Yoshida and A. H. Dachman, "CAD techniques, challenges, controversies in computed tomographic colonography, " Abdominal Imag., vol. 30, no. 1, pp. 26-41, 2004.
6. S. Seifert et al., "Hierarchical parsing and semantic navigation of full body CT data, " Proc. SPIE, vol. 7259, p. 725902, Mar. 2009.
7. T. Heimann and H.-P. Meinzer, "Statistical shape models for 3D medical image segmentation: A review, " Med. Image Anal., vol. 13, no. 4, pp. 543-563, 2009.
8. A. Criminisi et al., "Regression forests for efficient anatomy detection and localization in computed tomography scans, " Med. Image Anal., vol. 17, no. 8, pp. 1293-1303, 2013.
9. S. K. Warfield, K. H. Zou, W. M. Wells, "Simultaneous truth and performance level estimation (STAPLE): An algorithm for the validation of image segmentation, " IEEE Trans. Med. Imag., vol. 23, no. 7, pp. 903-921, Jul. 2004.
10. A. Hanbury et al., "Bringing the algorithms to the data: Cloud-based benchmarking for medical image analysis, " in Proc. CLEF Conf., 2012, pp. 24-29.
11. G. Langs, A. Hanbury, B. Menze, H. Müller, "VISCERAL: Towards large data in medical imaging-Challenges and directions, " in Medical Content-Based Retrieval for Clinical Decision Support (Lecture Notes in Computer Science), H. Greenspan, H. Müller and T. Syeda-Mahmood, Eds. Berlin, Germany: Springer, 2012, pp. 92-98.
12. T. G. Armstrong, A. Moffat, W. Webber, J. Zobel, "Improvements that don't add up: Ad-hoc retrieval results since 1998, " in Proc. 18th ACM Conf. Inf. Knowl. Manage. (CIKM), 2009, pp. 601-610.
13. D. J. Hand, "Classifier technology and the illusion of progress, " Statist. Sci., vol. 21, no. 1, pp. 1-14, 2006.
14. D. W. Shattuck, G. Prasad, M. Mirza, K. L. Narr, A. W. Toga, "Online resource for validation of brain segmentation methods, " NeuroImage, vol. 45, no. 2, pp. 431-439, 2009.
15. B. H. Menze et al., "The multimodal brain tumor image segmentation benchmark (BRATS), " IEEE Trans. Med. Imag., vol. 34, no. 10, pp. 1993-2024, Oct. 2015.
16. O. Camara, T. Mansi, M. Pop, K. Rhode, M. Sermesant, A. Young, Statistical Atlases and Computational Models of the Heart. Imaging and Modelling Challenges, vol. 7746. Nice, France: Springer, 2013.
17. P. Lo et al., "Extraction of airways from CT (EXACT'09), " IEEE Trans. Med. Imag., vol. 31, no. 11, pp. 2093-2107, Nov. 2012.
18. R. D. Rudyanto et al., "Comparing algorithms for automated vessel segmentation in computed tomography scans of the lung: The VESSEL12 study, " Med. Image Anal., vol. 18, no. 7, pp. 1217-1232, 2014.
19. B. van Ginneken et al., "Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The ANODE09 study, " Med. Image Anal., vol. 14, no. 6, pp. 707-722, 2010.
20. G. Litjens et al., "Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge, " Med. Image Anal., vol. 18, no. 2, pp. 359-373, 2014.
21. T. Heimann et al., "Comparison and evaluation of methods for liver segmentation from CT datasets, " IEEE Trans. Med. Imag., vol. 28, no. 8, pp. 1251-1265, Aug. 2009.
22. K. Murphy et al., "Evaluation of registration methods on thoracic CT: The EMPIRE10 challenge, " IEEE Trans. Med. Imag., vol. 30, no. 11, pp. 1901-1920, Nov. 2011.
23. M. G. Linguraru, J. A. Pura, A. S. Chowdhury, R. M. Summers, "Multi-organ segmentation from multi-phase abdominal CT via 4D graphs using enhancement, shape and location optimization, " in Medical Image Computing and Computer-Assisted Intervention (Lecture Notes in Computer Science), vol. 6363, T. Jiang, N. Navab, J. P. W. Pluim, M. A. Viergever, Eds. Berlin, Germany: Springer, 2010, pp. 89-96.
24. T. Okada et al., "Abdominal multi-organ segmentation of CT images based on hierarchical spatial modeling of organ interrelations, " in Abdominal Imaging. Computational and Clinical Applications. New York, Springer, 2011, pp. 173-180.
25. A. A. Taha and A. Hanbury, "Metrics for evaluating 3D medical image segmentation: Analysis, selection, tool, " BMC Med. Imag., vol. 15, no. 1, p. 29, 2015.
26. L. R. Dice, "Measures of the amount of ecologic association between species, " Ecology, vol. 26, no. 3, pp. 297-302, 1945.
27. A. Fenster and B. Chiu, "Evaluation of segmentation algorithms for medical imaging, " in Proc. IEEE Eng. Med. Biol. Soc. Conf. (EMBC), vol. 7. 2005, pp. 7186-7189.
28. O. A. Jimenez del Toro et al., "VISCERAL-VISual concept extraction challenge in RAdioLogy: ISBI 2014 Challenge Organization, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1194. Beijing, China, May 2014, pp. 6-15.
29. O. Goksel et al., "Overview of the VISCERAL challenge at ISBI 2015, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1390. Apr. 2015, pp. 6-11.
30. Y. Dicente Cid, O. A. Jimenez del Toro, A. Depeursinge, H. Müller, "Efficient and fully automatic segmentation of the lungs in CT volumes, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1390, Apr. 2015.
31. A. B. Spanier and L. Joskowicz, "Rule-based ventral cavity multi-organ automatic segmentation in CT scans, " in Proc. Int. Workshop MCV, vol. 8848, 2014, pp. 163-170.
32. X. Li, C. Huang, F. Jia, Z. Li, C. Fang, Y. Fan, "Automatic liver segmentation using statistical prior models and free-form deformation, " in Proc. Int. Workshop MCV, vol. 8848. 2014, pp. 181-188.
33. B. He et al., "Fully automatic multi-organ segmentation based on multi-boost learning and statistical shape model search, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1390. Apr. 2015, pp. 18-21.
34. G. Vincent, G. Guillard, M. Bowes, "Fully automatic segmentation of the prostate using active appearance models, " in Proc. MICCAI Workshop: Prostate Cancer Imag., PROMISE12 Prostate Segmentation Challenge, 2012.
35. C. Wang and Ö. Smedby, "Automatic multi-organ segmentation using fast model based level set method and hierarchical shape priors, " in Proc. CEUR Workshop VISCERAL Challenge ISBI vol. 1194. Beijing, China, May 2014, pp. 25-31.
36. C. Wang and Ö. Smedby, "Multi-organ segmentation using shape model guided local phase analysis, " in Medical Image Computing and Computer-Assisted Intervention (Lecture Notes in Computer Science), vol. 9351, N. Navab, J. Hornegger, W. M. Wells and A. F. Frangi, Eds. Springer, 2015, pp. 149-156. •
37. O. Goksel, T. Gass, G. Szekely, "Segmentation and landmark localization based on multiple atlases, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1194. Beijing, China, May 2014, pp. 37-43.
38. T. Gass, G. Szekely, O. Goksel, "Multi-atlas segmentation and landmark localization in images with large field of view, " in Proc. Int. Workshop MCV, vol. 8848. 2014, pp. 171-180.
39. M. P. Heinrich et al., "Multi-modal multi-atlas segmentation using discrete optimisation and self-similarities, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1390, Apr. 2015.
40. O. A. Jimenez del Toro and H. Müller, "Hierarchic multi-atlas based segmentation for anatomical structures: Evaluation in the VISCERAL anatomy benchmarks, " in Proc. Int. Workshop MCV, vol. 8848. 2014, pp. 189-200.
41. O. A. Jimenez del Toro, Y. Dicente Cid, A. Depeursinge, H. Müller, "Hierarchic anatomical structure segmentation guided by spatial correlations (AnatSeg-Gspac): VISCERAL anatomy3, " in Proc. CEUR Workshop VISCERAL Challenge ISBI vol. 1390. Apr. 2015, pp. 22-26.
42. F. Kahl et al., "Good features for reliable registration in multi-atlas segmentation, " in Proc. CEUR Workshop VISCERAL Challenge ISBI, vol. 1390. Apr. 2015, pp. 12-17.
43. R. Kéchichian, S. Valette, M. Sdika, M. Desvignes, "Automatic 3D multiorgan segmentation via clustering and graph cut using spatial relations and hierarchically-registered atlases, " in Proc. Int. Workshop MCV, vol. 8848. 2014, pp. 201-210.
44. D. Mai et al., "Discriminative detection and alignment in volumetric data, " in Pattern Recognition, vol. 8142. Springer, 2013, pp. 205-214.
45. M. A. Dabbah et al., "Detection and location of 127 anatomical landmarks in diverse CT datasets, " Proc. SPIE, vol. 9034, p. 903415, Mar. 2014.
46. B. S. Elger et al., "Strategies for health data exchange for secondary, cross-institutional clinical research, " Comput. Methods Programs Biomed., vol. 99, pp. 230-251, Sep. 2010.
47. A. Shimizu, T. Kimoto, H. Kobatake, S. Nawano, K. Shinozaki, "Automated pancreas segmentation from three-dimensional contrastenhanced computed tomography, " Int. J. Comput. Assist. Radiol. Surg., vol. 5, no. 1, pp. 85-98, 2010.
48. P. Kohlmann et al., "Automatic lung segmentation method for MRIbased lung perfusion studies of patients with chronic obstructive pulmonary disease, " Int. J. Comput. Assist. Radiol. Surg., vol. 10, no. 4, pp. 403-417, 2014.
49. H. T. Huynh, I. Karademir, A. Oto, K. Suzuki, "Computerized liver volumetry on MRI by using 3D geodesic active contour segmentation, " Am. J. Roentgenol., vol. 202, no. 1, p. 152, 2014.
50. A. Alkan, S. A. Tuncer, M. Gunay, "Comparative MR image analysis for thyroid nodule detection and quantification, " Measurement, vol. 47, pp. 861-868, Jan. 2014.
51. V. Pekar, T. R. McNutt, M. R. Kaus, "Automated model-based organ delineation for radiotherapy planning in prostatic region, " Int. J. Radiat. Oncol. Biol. Phys., vol. 60, no. 3, pp. 973-980, 2004.
52. C. W. Jeong et al., "Comparison of prostate volume measured by transrectal ultrasonography and MRI with the actual prostate volume measured after radical prostatectomy, " Urologia Internationalis, vol. 81, no. 2, pp. 179-185, 2008.
53. T. R. Langerak et al., "Label fusion in atlas-based segmentation using a selective and iterative method for performance level estimation (SIMPLE), " IEEE Trans. Med. Imag., vol. 29, no. 12, pp. 2000-2008, Dec. 2010.
54. M. Krenn et al., "Creating a large-scale silver corpus from multiple algorithmic segmentations, " in Proc. MICCAI Workshop Med. Comput. Vis., Algorithms Big Data MCV, vol. 8848. 2014, pp. 163-170.