Evaluation of registration methods on thoracic CT: The EMPIRE10 challenge

IEEE Transactions on Medical Imaging

Volumn 30, Issue 11, 2011, Pages 1901-1920

  Murphy, Keelin ^a   Van Ginneken, Bram ^a,b   Reinhardt, Joseph M ^c   Kabus, Sven ^d   Ding, Kai ^c   Deng, Xiang ^e   Cao, Kunlin ^c   Du, Kaifang ^c   Christensen, Gary E ^c   Garcia, Vincent ^f   Vercauteren, Tom ^g   Ayache, Nicholas ^f   Commowick, Olivier ^h   Malandain, Grgoire ^f   Glocker, Ben ⁱ   Paragios, Nikos ^j,k   Navab, Nassir ⁱ   Gorbunova, Vladlena ^l   Sporring, Jon ^l   De Bruijne, Marleen ^l,m   Han, Xiao ⁿ   Heinrich, Mattias P ^o,p   Schnabel, Julia A ^o   Jenkinson, Mark ^p   Lorenz, Cristian ^d   Modat, Marc ^q   McClelland, Jamie R ^q   Ourselin, Sebastien ^q   Muenzing, Sascha E A ^a   Viergever, Max A ^a   De Nigris, Dante ^r   Collins, D Louis ^r   Arbel, Tal ^r   Peroni, Marta ^s,t,u   Li, Rui ^s   Sharp, Gregory C ^s   Schmidt Richberg, Alexander ^v   Ehrhardt, Jan ^v   Werner, René ^v   Smeets, Dirk ^w   Loeckx, Dirk ^w   Song, Gang ^x   Tustison, Nicholas ^a   Avants, Brian ^x   Gee, James C ^x   Staring, Marius ^y   Klein, Stefan ^m   Stoel, Berend C ^y   Urschler, Martin ^z,aa   Werlberger, Manuel ^z   Vandemeulebroucke, Jef ^ab   Rit, Simon ^ab   Sarrut, David ^ab   Pluim, Josien P W ^a   more..

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^b RADBOUD UNIVERSITY MEDICAL CENTER   (Netherlands)

^c University of Iowa   (United States)

^d PHILIPS RESEARCH LABORATORIES   (Germany)

^e Philips Research Laboratories ^*  (China)

^f INRIA   (France)

^g Mauna Kea Technologies   (France)

^h INRIA   (France)

ⁱ TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^j ECOLE CENTRALE PARIS   (France)

^k INRIA SACLAY   (France)

^l UNIVERSITY OF COPENHAGEN   (Denmark)

^m ERASMUS MEDICAL CENTER   (Netherlands)

ⁿ Elekta AB   (Sweden)

^o UNIVERSITY OF OXFORD   (United Kingdom)

^p UNIVERSITY OF OXFORD   (United Kingdom)

^q UNIVERSITY COLLEGE LONDON   (United Kingdom)

^r MCGILL UNIVERSITY   (Canada)

^s MASSACHUSETTS GENERAL HOSPITAL   (United States)

^t POLITECNICO DI MILANO   (Italy)

^u MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^v UNIVERSITY OF LÜBECK   (Germany)

^w UNIVERSITY OF LEUVEN   (Belgium)

^x UNIVERSITY OF PENNSYLVANIA   (United States)

^y UNIVERSITY OF VIRGINIA   (United States)

^z GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^aa LUDWIG BOLTZMANN INSTITUTE FOR LUNG VASCULAR RESEARCH   (Austria)

^ab UNIVERSITÉ DE LYON   (France)

more..

Author keywords

Chest; computed tomography; evaluation; registration

Indexed keywords

INDUSTRIAL RESEARCH;

CHEST; COMPREHENSIVE EVALUATION; EVALUATION; EVALUATION OF METHODS; NONRIGID REGISTRATION; REGISTRATION; REGISTRATION ALGORITHMS; REGISTRATION METHODS;

COMPUTERIZED TOMOGRAPHY;

ALGORITHM; ANIMAL; ARTICLE; COMPARATIVE STUDY; COMPUTER ASSISTED DIAGNOSIS; COMPUTER ASSISTED TOMOGRAPHY; COMPUTER PROGRAM; EVALUATION; FACTUAL DATABASE; IMAGE QUALITY; LUNG; METHODOLOGY; OBSERVER VARIATION; RADIOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; SHEEP; STANDARD; THORAX; THORAX RADIOGRAPHY;

ALGORITHMS; ANIMALS; DATABASES, FACTUAL; LUNG; OBSERVER VARIATION; RADIOGRAPHIC IMAGE ENHANCEMENT; RADIOGRAPHIC IMAGE INTERPRETATION, COMPUTER-ASSISTED; RADIOGRAPHY, THORACIC; REFERENCE STANDARDS; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SHEEP; SOFTWARE VALIDATION; THORAX; TOMOGRAPHY, X-RAY COMPUTED;

EID: 80455145309     PISSN: 02780062     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMI.2011.2158349     Document Type: Article

References (63)

1. L. G. Brown, "A survey of image registration techniques," ACM Comput. Surv., vol. 24, pp. 325-376, 1992.
2. J. B. A. Maintz and M. A. Viergever, "A survey of medical image registration," Med. Image Anal., vol. 2, pp. 1-36, 1998. (Pubitemid 128500719)
3. H. Lester and S. R. Arridge, "A survey of hierarchical non-linear medical image registration," Pattern Recognit., vol. 32, pp. 129-149, 1999.
4. D. L. Hill, P. G. Batchelor, M. Holden, and D. J. Hawkes, "Medical image registration," Phys. Med. Biol., vol. 46, pp. R1-45, 2001. (Pubitemid 32218849)
5. B. Zitov́ and J. Flusser, "Image registration methods: A survey," Image Vis. Comput., vol. 21, pp. 977-1000, 2003.
6. J. P. W. Pluim, J. B. A. Maintz, and M. A. Viergever, "Mutual-information-based registration of medical images: A survey," IEEE Trans. Med. Imag., vol. 22, no. 8, pp. 986-1004, Aug. 2003.
7. N. J. Tustison, T. S. Cook, G. Song, and J. C. Gee, "Pulmonary kinematics from image data: A review," Acad. Radiol., vol. 18, no. 4, pp. 402-417, 2011.
8. A. Klein, J. Andersson, B. A. Ardekani, J. Ashburner, B. Avants, M. Chiang, G. E. Christensen, D. L. Collins, J. Gee, P. Hellier, J. Song, M. Jenkinson, C. Lepage, D. Rueckert, P. Thompson, T. Vercauteren, R. P. Woods, J. J. Mann, and R. V. Parsey, "Evaluation of 14 nonlinear deformation algorithms applied tohuman brain MRI registration,"Neu-roimage, vol. 46, no. 3, pp. 786-802, 2009.
9. X. Gu, H. Pan, Y. Liang, R. Castillo, D. Yang, D. Choi, E. Castillo, A. Majumdar, T. Guerrero, and S. B. Jiang, "Implementation and evaluation of various demons deformable image registration algorithms on a GPU," Phys. Med. Biol., vol. 55, no. 1, pp. 207-219, Jan. 2010.
10. S. Kabus, T. Klinder, K. Murphy, B. van Ginneken, C. Lorenz, and J. P. W. Pluim, "Evaluation of 4-D-CT lung registration," Proc. MICCAI, vol. 12, pp. 747-754, 2009.
11. M. A. Yassa and C. E. L. Stark, "A quantitative evaluation of cross-participant registration techniques for MRI studies of the medial temporal lobe," Neuroimage, vol. 44, no. 2, pp. 319-327, Jan. 2009.
12. D. Sarrut, B. Delhay, P. Villard, V. Boldea, M. Beuve, and P. Clarysse, "A comparison framework for breathing motion estimation methods from 4-D imaging," IEEE Trans. Med. Imag., vol. 26, no. 12, pp. 1636-1648, Dec. 2007. (Pubitemid 350183910)
13. R. Castillo, E. Castillo, R. Guerra, V. E. Johnson, T. McPhail, A. K. Garg, and T. Guerrero, "A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets," Phys. Med. Biol., vol. 54, no. 7, pp. 1849-1870, 2009.
14. J. West, J. M. Fitzpatrick, M. Y. Wang, B. M. Dawant, C. R. Maurer, Jr., R. M. Kessler, R. J. Maciunas, C. Barillot, D. Lemoine, A. Col-lignon, F. Maes, P. Suetens, D. Vandermeulen, P. A. van den Elsen, S. Napel, T. Sumanaweera, B. Harkness, P. F. Hemler, D. L. G. Hill, D. J. Hawkes, C. Studholme, J. B. A. Maintz, M. A. Viergever, G. Ma-landain, X. Pennec, M. E. Noz, G. Q. Maguire, Jr., M. Pollack, C. A. Pelizzari, R. A. Robb, D. Hanson, and R. P. Woods, "Comparison and evaluation of retrospective intermodality image registration techniques," J. Comput. Assist. Tomogr., vol. 21, pp. 554-566, 1997. (Pubitemid 27432693)
15. G. Christensen, X. Geng, J. Kuhl, J. Bruss, T. Grabowski, I. Pirwani, M. Vannier, J. Allen, and H. Damasio, "Introduction to the non-rigid image registration evaluation project (NIREP)," in Third International Workshop on Biomedical Image Registration. New York: Springer, 2006, vol. 4057, Lecture Notes in Computer Science, pp. 128-135. (Pubitemid 44113208)
16. P. Hellier, C. Barillot, I. Corouge, B. Gibaud, G. L. Goualher, D. L. Collins, A. Evans, G. Malandain, N. Ayache, G. E. Christensen, and H. J. Johnson, "Retrospective evaluation of intersubject brain registration," IEEE Trans. Med. Imag., vol. 22, no. 9, pp. 1120-1130, Sep. 2003.
17. K. K. Brock, Deformable Registration Accuracy Consortium, "Results of a multi-institution deformable registration accuracy study (MIDRAS)," Int. J. Radiat. Oncol. Biol. Phys., vol. 76, no. 2, pp. 583-596, 2010.
18. R. Kashani, M. Hub, J. M. Balter, M. L. Kessler, L. Dong, L. Zhang, L. Xing, Y. Xie, D. Hawkes, J. A. Schnabel, J. R. McClelland, S. Joshi, Q. Chen, and W. Lu, "Objective assessment of deformable image registration in radiotherapy: A multi-institution study," Med. Phys., vol. 35, no. 12, pp. 5944-5953, 2008.
19. MICCAI Conf. Beijing, China, 2010 [Online]. Available: http://www.grand-challenge.org/index.php/MICCAI-2010-Work-shop
20. E. M. van Rikxoort, B. J. de Hoop, M. A. Viergever, M. Prokop, and B. van Ginneken, "Automatic lung segmentation from thoracic computed tomography scans using a hybrid approach with error detection," Med. Phys., vol. 36, no. 7, pp. 2934-2947, 2009.
21. D. M. Xu, H. Gietema, H. de Koning, R. Vernhout, K. Nackaerts, M. Prokop, C. Weenink, J. Lammers, H. Groen, M. Oudkerk, and R. van Klaveren, "Nodule management protocol of the NELSON randomised lung cancer screening trial," Lung Cancer, vol. 54, no. 2, pp. 177-184, 2006. (Pubitemid 44802721)
22. J. Vandemeulebroucke, D. Sarrut, and P. Clarysse, "The POPI model, a point-validated pixel-based breathing thorax model," in XVth ICCR, Toronto, Canada, 2007, vol. 2, pp. 195-199.
23. J. A. Schnabel, C. Tanner, A. D. Castellano-Smith, A. Degenhard, M. O. Leach, D. R. Hose, D. L. G. Hill, and D. J. Hawkes, "Validation of nonrigid image registration using finite-element methods: Application to breast MR images," IEEE Trans. Med. Imag., vol. 22, no. 2, pp. 238-247, Feb. 2003.
24. M. Urschler, S. Kluckner, and H. Bischof, "A framework for comparison and evaluation of nonlinear intra-subject image registration algorithms," presented at the MICCAI Open Science Workshop, Brisbane, Australia, 2007.
25. H. Wang, L. Dong, J. O'Daniel, R. Mohan, A. S. Garden, K. K. Ang, D. A. Kuban, M. Bonnen, J. Y. Chang, and R. Cheung, "Validation of an accelerated "demons" algorithm for deformable image registration in radiation therapy," Phys. Med. Biol., vol. 50, no. 12, pp. 2887-2905, 2005. (Pubitemid 40771018)
26. K. Murphy, B. van Ginneken, J. P. W. Pluim, S. Klein, and M. Staring, "Semi-automatic reference standard construction for quantitative evaluation of lung CT registration," Proc. MICCAI, vol. 11, pp. 1006-1013, 2008.
27. K.Murphy, B. van Ginneken, S. Klein, M. Staring, B. J. de Hoop, M. A. Viergever, and J. P. W. Pluim, "Semi-automatic construction of reference standards for evaluation of image registration," Med. Image Anal., vol. 15, pp. 71-84, 2011.
28. E. M. van Rikxoort, B. van Ginneken, M. Klik, and M. Prokop, "Supervised enhancement filters: Application to fissure detection in chest CT scans," IEEE Trans. Med. Imag., vol. 27, no. 1, pp. 1-10, Jan. 2008.
29. V. Boldea, G. C. Sharp, S. B. Jiang, and D. Sarrut, "4-D-CT lung motion estimation with deformable registration: Quantification of motion nonlinearity and hysteresis," Med. Phys., vol. 35, no. 3, pp. 1008-1018, 2008. (Pubitemid 351322647)
30. I. D. Grachev, D. Berdichevsky, S. L. Rauch, S. Heckers, D. N. Kennedy, V. S. Caviness, and N. M. Alpert, "A method for assessing the accuracy of intersubject registration of the human brain using anatomic landmarks," Neuroimage, vol. 9, no. 2, pp. 250-268, 1999. (Pubitemid 29369881)
31. E. Heath, D. L. Collins, P. J. Keall, L. Dong, and J. Seuntjens, "Quantification of accuracy of the automated nonlinear image matching and anatomical labeling (ANIMAL) nonlinear registration algorithm for 4-D CT images of lung," Med. Phys., vol. 34, no. 11, pp. 4409-4421, 2007. (Pubitemid 350029662)
32. A. Pevsner, B. Davis, S. Joshi, A. Hertanto, J. Mechalakos, E. Yorke, K. Rosenzweig, S. Nehmeh, Y. E. Erdi, J. L. Humm, S. Larson, C. C. Ling, and G. S. Mageras, "Evaluation of an automated deformable image matching method for quantifying lung motion in respiration-correlated CT images," Med. Phys., vol. 33, no. 2, pp. 369-376, 2006. (Pubitemid 43183979)
33. T. Vik, S. Kabus, J. von Berg, K. Ens, S. Dries, T. Klinder, and C. Lorenz, "Validation and comparison of registration methods for free-breathing 4-D lung CT," in Proc. SPIE, 2008, vol. 6914.
34. Z. Wu, E. Rietzel, V. Boldea, D. Sarrut, and G. C. Sharp, "Evaluation of deformable registration of patient lung 4DCT with subanatomical region segmentations," Med. Phys., vol. 35, no. 2, pp. 775-781, 2008. (Pubitemid 351190897)
35. D. Rey, G. Subsol, H. Delingette, and N. Ayache, "Automatic detection and segmentation of evolving processes in 3D medical images: Application to multiple sclerosis," Med. Image Anal., vol. 6, no. 2, pp. 163-179, 2002. (Pubitemid 35282824)
36. V. Garcia, T. Vercauteren, G. Malandain, and N. Ayache, "Diffeomor-phic demons and the EMPIRE10 challenge," Med. Image Anal. Clinic: A Grand Challenge, pp. 91-98, 2010.
37. T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache, "Dif-feomorphic demons: Efficient non-parametric image registration," NeuroImage, vol. 45, no. 1, pp. S61-S72, 2009.
38. V. Garcia, O. Commowick, and G. Malandain, "A robust and efficient block-matching framework for non linear registration of thoracic CT images," Med. Image Anal. Clinic: A Grand Challenge, pp. 137-146, 2010.
39. X. Han, "Feature-constrained nonlinear registration of lung CT images," Med. Image Anal. Clinic: A Grand Challenge, pp. 63-72, 2010.
40. V. Gorbunova, P. Lo, H. Ashraf, A. Dirksen, M. Nielsen, and M. de Bruijne, "Weight preserving image registration for monitoring disease progression in lung CT," in Medical Image Computing and Computer-Assisted Intervention. New York: Springer, vol. 5242, Lecture Notes in Computer Science, pp. 863-870.
41. V. Gorbunova,J.Sporring, P. Lo, A. Dirksen, and M.deBruijne, "Mass preserving image registration: Results of evaluationof methods for pulmonary image registration 2010 challenge," Med. Image Anal. Clinic: A Grand Challenge, pp. 155-164, 2010.
42. B. Glocker, N. Komodakis, N. Paragios, and N. Navab, "Non-rigid registration using discrete MRFs: Application to thoracic CT images," Med. Image Anal. Clinic: A Grand Challenge, pp. 147-154, 2010.
43. B. Glocker, N. Komodakis, G. Tziritas, N. Navab, and N. Paragios, "Dense image registration through MRFs and efficient linear programming," Med. Image Anal., vol. 12, no. 6, pp. 731-741, 2008.
44. M. Staring, S. Klein, J. H. C. Reiber, W. J. Niessen, and B. Stoel, "Pulmonary image registration with elastix using a standard intensity-based algorithm," Med. Image Anal. Clinic: A Grand Challenge, pp. 73-79, 2010.
45. S. Klein, M. Staring, K. Murphy, M. A. Viergever, and J. P. W. Pluim, "elastix: A toolbox for intensity-based medical image registration," IEEE Trans. Med. Imag., vol. 29, no. 1, pp. 196-205, Jan. 2010.
46. M. Urschler, M. Werlberger, E. Scheurer, and H. Bischof, "Robust optical flow based deformable registration of thoracic CT images," Med. Image Anal. Clinic: A Grand Challenge, pp. 195-204, 2010.
47. T. Pock, M. Urschler, C. Zach, R. Beichel, and H. Bischof, "A duality based algorithm for TV-L1-optical-flow image registration," in Medical Image Computer Computer-Assisted Intervention. New York: Springer, 2007, vol. 4792, Lecture Notes in Computer Science, pp. 511-518.
48. A. Schmidt-Richberg, J. Ehrhardt, R. Werner, and H. Handels, "Diffeo-morphic diffusion registration of lung CT images," Med. Image Anal. Clinic: A Grand Challenge, pp. 55-62, 2010.
49. J. Ehrhardt, R. Werner, A. Schmidt-Richberg, and H. Handels, "Statistical modeling of 4-D respiratory lung motion using diffeomorphic image registration," IEEE Trans. Med. Imag., vol. 30, no. 2, pp. 251-265, Feb. 2011.
50. K. Cao, K. Du, K. Ding, J. M. Reinhardt, and G. E. Christensen, "Regularized nonrigid registration of lung CT images by preserving tissue volume and vesselness measure," Med. Image Anal. Clinic: A Grand Challenge, pp. 43-54, 2010.
51. K. Cao, K. Ding, G. E. Christensen, M. L. Raghavan, R. E. Amelon, and J. M. Reinhardt, "Unifying vascular information in intensity-based nonrigid lung CT registration," in Biomedical Image Registration: 4th International Workshop, WBIR2010. New York: Springer, 2010, vol. 6204, Lecture Notes in Computer Science.
52. S. E. A. Muenzing, B. van Ginneken, and J. P. W. Pluim, "Knowledge driven regularization of the deformation field for PDE based non-rigid registration algorithms," Med. Image Anal. Clinic: A Grand Challenge, pp. 127-136, 2010.
53. J. Vandemeulebroucke, S. Rit, J. Schaerer, and D. Sarrut, "Deformable image registration with automated motion-mask extraction," Med. Image Anal. Clin.: A Grand Challenge, pp. 119-125, 2010.
54. G. C. Sharp, M. Peroni, R. Li, J. Shackleford, and N. Kandasamy, "Evaluation of plastimatch B-Spline registration on the EMPIRE10 data set," Med. Image Anal. Clinic: A Grand Challenge, pp. 99-108, 2010.
55. J. A. Shackleford, N. Kandasamy, and G. C. Sharp, "On developing B-spline registration algorithms for multi-core processors," Phys. Med. Biol., vol. 55, no. 21, pp. 6329-6351, 2010.
56. M. Modat, J. R. McClelland, and S. Ourselin, "Lung registration using the NiftyReg package," Med. Image Anal. Clinic: A Grand Challenge, pp. 33-42, 2010.
57. M. Modat, G. R. Ridgway, Z. A. Taylor, M. Lehmann, J. Barnes, D. J. Hawkes, N. C. Fox, and S. Ourselin, "Fast free-form deformation using graphics processing units," Comput. Methods Programs Biomed., vol. 98, no. 3, pp. 278-284, 2010.
58. M. P. Heinrich, M. Jenkinson, M. Brady, and J. Schnabel, "Discontinuity preserving regularisation for variational optical-flow registration using the modified Lp norm," Med. Image Anal. Clinic: A Grand Challenge, pp. 185-194, 2010.
59. S. Kabus and C. Lorenz, "Fast elastic image registration," Med. Image Anal. Clinic: A Grand Challenge, pp. 81-89, 2010.
60. G. Song, N. Tustison, B. Avants, and J. C. Gee, "Lung CT image registration using diffeomorphic transformation models," Med. Image Anal. Clinic: A Grand Challenge, pp. 23-32, 2010.
61. B. B. Avants, N. J. Tustison, G. Song, P. A. Cook, A. Klein, and J. C. Gee, "A reproducible evaluation of ANTs similarity metric performance in brain image registration," Neuroimage, vol. 54, no. 3, pp. 2033-2044, 2011.
62. D. De Nigris, D. L. Collins, and T. Arbel, "Deformable registration of chest CT scans with adaptive local mutual information," Med. Image Anal. Clinic: A Grand Challenge, pp. 175-184, 2010.
63. D. Loeckx, D. Smeets, J. Keustermans, J. Hermans, F. Maes, D. Van-dermeulen, and P. Suetens, "3D lung registration using splineMIRIT and robust tree registration (RTR)," Med. Image Anal. Clinic: A Grand Challenge, pp. 109-117, 2010.