Quicksilver: Fast predictive image registration – A deep learning approach

NeuroImage

Volumn 158, Issue , 2017, Pages 378-396

  Yang, Xiao ^a   Kwitt, Roland ^c   Styner, Martin ^a,b   Niethammer, Marc ^a,b  

^a UNIVERSITY OF NORTH CAROLINA   (United States)

^b UNIVERSITY OF NORTH CAROLINA SCHOOL OF MEDICINE   (United States)

^c UNIVERSITY OF SALZBURG   (Austria)

Author keywords

Brain imaging; Deep learning; Image registration

Indexed keywords

ACCURACY; ART; ARTICLE; COMPUTER NETWORK; CORRECTION NETWORK; DEEP LEARNING MODEL; DETERMINISTIC NETWORK; DIFFEOMORPHIC MAPPING; FAST PREDICTIVE IMAGE REGISTRATION; IMAGING SOFTWARE; LABEL OVERLAP; LARGE DEFORMATION DIFFEOMORPHIC METRIC MAPPING MODEL; MATHEMATICAL ANALYSIS; PARAMETERIZATION; PATCH PRUNING; PATCH WISE PREDICTION; PREDICTION; PRIORITY JOURNAL; PROBABILISTIC NETWORK; PROCESS OPTIMIZATION; VALIDATION PROCESS; ALGORITHM; ANATOMY AND HISTOLOGY; AUTOMATED PATTERN RECOGNITION; BRAIN; BRAIN MAPPING; HUMAN; IMAGE PROCESSING; MACHINE LEARNING; PHYSIOLOGY; PROCEDURES; THREE DIMENSIONAL IMAGING;

ALGORITHMS; BRAIN; BRAIN MAPPING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MACHINE LEARNING; PATTERN RECOGNITION, AUTOMATED;

EID: 85024476040     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2017.07.008     Document Type: Article

References (71)

1. Ardekani, B.A., Guckemus, S., Bachman, A., Hoptman, M.J., Wojtaszek, M., Nierenberg, J., Quantitative comparison of algorithms for inter-subject registration of 3D volumetric brain MRI scans. J. Neurosci. Methods 142:1 (2005), 67–76.
2. Ashburner, J., A fast diffeomorphic image registration algorithm. NeuroImage 38:1 (2007), 95–113.
3. Ashburner, J., Friston, K.J., Diffeomorphic registration using geodesic shooting and Gauss–Newton optimisation. NeuroImage 55:3 (2011), 954–967.
4. Avants, B., Epstein, C., Grossman, M., Gee, J., Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. MedIA 12:1 (2008), 26–41 special Issue on The Third International Workshop on Biomedical Image Registration WBIR 2006.
5. Beg, M.F., Miller, M., Trouvé, A., Younes, L., Computing large deformation metric mappings via geodesic flows of diffeomorphisms. IJCV 61:2 (2005), 139–157.
6. Biobank website: www.ukbiobank.ac.uk.
7. Borzi, A., Ito, K., Kunisch, K., Optimal control formulation for determining optical flow. SIAM J. Sci. Comput. 24:3 (2003), 818–847.
8. Bruveris, M., Risser, L., Vialard, F.-X., Mixture of kernels and iterated semidirect product of diffeomorphisms groups. Multiscale Model. Simul. 10:4 (2012), 1344–1368.
9. Cao, T., Zach, C., Modla, S., Powell, D., Czymmek, K., Niethammer, M., Multi-modal registration for correlative microscopy using image analogies. MedIA 18:6 (2014), 914–926.
10. Cao, T., Singh, N., Jojic, V., Niethammer, M., Semi-coupled Dictionary Learning for Deformation Prediction. 2015, ISBI, 691–694.
11. Cheng, X., Zheng, L.Z.Y., Deep similarity learning for multimodal medical images. Comput. Methods Biomech. Biomed. Eng. Imaging Vis., 2015, 1–5.
12. Chou, C.-R., Frederick, B., Mageras, G., Chang, S., Pizer, S., 2D/3D image registration using regression learning. CVIU 117:9 (2013), 1095–1106.
13. Chung, K., Deisseroth, K., CLARITY for mapping the nervous system. Nat. Methods 10:6 (2013), 508–513.
14. Dosovitskiy, A., Fischery, P., Ilg, E., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., Brox, T., Flownet: Learning Optical Flow with Convolutional Networks. 2015, ICCV, 2758–2766.
15. Dupuis, P., Grenander, U., Miller, M.I., Variational problems on flows of diffeomorphisms for image matching. Q. Appl. Math., 1998, 587–600.
16. Gal, Y., Ghahramani, Z., Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. arXiv:1506.02158, 2015.
17. Gal, Y., Ghahramani, Z., Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. 2016, ICML.
18. Gao, T., Jojic, V., Degrees of Freedom in Deep Neural Networks. 2016, UAI, 232–241.
19. Grabner, G., Janke, A.L., Budge, M.M., Smith, D., Pruessner, J., Collins, D.L., Symmetric Atlasing and Model Based Segmentation: an Application to the hippocampus in Older Adults. Larsen, R., Nielsen, M., Sporring, J., (eds.), 2006, MICCAI, 58–66.
20. Griewank, A., Walther, A., Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. 2008, SIAM.
21. Guetter, C., Xu, C., Sauer, F., Hornegger, J., Learning Based Non-rigid Multi-modal Image Registration Using Kullback-leibler Divergence. 2005, MICCAI, 255–262.
22. Gutierrez-Becker, B., Mateus, D., Peter, L., Navab, N., Guiding multimodal registration with learned optimization updates. MedIA, 2017.
23. Gutiérrez-Becker, B., Mateus, D., Peter, L., Navab, N., Learning Optimization Updates for Multimodal Registration. 2016, MICCAI, 19–27.
24. Haber, E., Modersitzki, J., Image registration with guaranteed displacement regularity. Int. J. Comput. Vis. 71:3 (2007), 361–372.
25. Hart, G.L., Zach, C., Niethammer, M., An optimal control approach for deformable registration. Computer Vision and Pattern Recognition Workshops, 2009. CVPR Workshops 2009, 2009, IEEE Computer Society Conference on, IEEE, 9–16.
26. Hazlett, H.C., Gu, H., Munsell, B.C., Kim, S.H., Styner, M., Wolff, J.J., Elison, J.T., Swanson, M.R., Zhu, H., Botteron, K.N., et al. Early brain development in infants at high risk for autism spectrum disorder. Nature 542:7641 (2017), 348–351.
27. He, K., Zhang, X., Ren, S., Sun, J., Delving Deep into Rectifiers: Surpassing Human-level Performance on ImageNet Classification, CoRR abs/1502.01852. 2015.
28. Hermosillo, G., Chefd'Hotel, C., Faugeras, O., Variational methods for multimodal image matching. IJCV 50:3 (2002), 329–343.
29. Hill, D.L., Batchelor, P.G., Holden, M., Hawkes, D.J., Medical image registration. Phys. Med. Biol., 46(3), 2001, R1.
30. Hong, Y., Shi, Y., Styner, M., Sanchez, M., Niethammer, M., Simple Geodesic Regression for Image Time-series. 2012, WBIR, 11–20.
31. Horn, B.K., Schunck, B.G., Determining optical flow. Artif. Intell. 17:1–3 (1981), 185–203.
32. Joshi, S., Davis, B., Jomier, M., Gerig, G., Unbiased diffeomorphic atlas construction for computational anatomy. NeuroImage 23 (2004), 151–160.
33. Kingma, D.P., Ba, J., Adam: a Method for Stochastic Optimization. 2014, ICLR.
34. Klein, A., Andersson, J., Ardekani, B.A., Ashburner, J., Avants, B., Chiang, M.-C., Christensen, G.E., Collins, D.L., Gee, J., Hellier, P., Song, J.H., Jenkinson, M., Lepage, C., Rueckert, D., Thompson, P., Vercauteren, T., Woods, R.P., Mann, J.J., Parsey, R.V., Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. NeuroImage 46:3 (2009), 786–802.
35. LeCun, Y. A theoretical framework for back-propagation. In: Proceedings of the 1988 Connectionist Models Summer School, 1988.
36. Lee, D., Hofmann, M., Steinke, F., Altun, Y., Cahill, N., Schölkopf, B., Learning Similarity Measure for Multi-modal 3D Image Registration. 2009, CVPR, 186–193.
37. Long, J., Shelhamer, E., Darrell, T., Fully Convolutional Networks for Semantic Segmentation. 2015, CVPR, 3431–3440.
38. Lorenzen, P., Prastawa, M., Davis, B., Gerig, G., Bullitt, E., Joshi, S., Multi-modal image set registration and atlas formation. MedIA 10:3 (2006), 440–451.
39. Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L., Open access series of imaging studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J. Cogn. Neurosci. 19:9 (2007), 1498–1507.
40. Meyer, C., Boes, J., Kim, B., Bland, P., Evaluation of Control Point Selection in Automatic, Mutual Information Driven, 3D Warping. 1998, MICCAI, 944–951.
41. Miao, S., Wang, Z.J., Liao, R., A CNN regression approach for real-time 2D/3D registration. IEEE TMI 35:5 (2016), 1352–1363.
42. Michel, F., Bronstein, M., Bronstein, A.M., Paragios, N., Boosted Metric Learning for 3D Multi-modal Deformable Registration. 2011, ISBI, 1209–1214.
43. Modersitzki, J., Numerical Methods for Image Registration. 2004, Oxford University Press on Demand.
44. Nair, V., Hinton, G.E., Rectified linear units improve restricted Boltzmann machines. Frnkranz, J., Joachims, T., (eds.) Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, Omnipress, 807–814.
45. Niethammer, M., Huang, Y., Vialard, F.-X., Geodesic Regression for Image Time-series. 2011, MICCAI, 655–662.
46. Nocedal, J., Wright, S.J., Numerical Optimization. second ed., 2006, Springer.
47. Oliveira, F.P., Tavares, J.M.R., Medical image registration: a review. Comput. Methods Biomech. Biomed. Eng. 17:2 (2014), 73–93.
48. Reuter, M., Schmansky, N.J., Rosas, H.D., Fischl, B., Within-subject template estimation for unbiased longitudinal image analysis. NeuroImage 61:4 (2012), 1402–1418.
49. Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L., Leach, M.O., Hawkes, D.J., Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18:8 (1999), 712–721.
50. Shams, R., Sadeghi, P., Kennedy, R.A., Hartley, R.I., A survey of medical image registration on multicore and the GPU. IEEE Signal Process. Mag. 27:2 (2010), 50–60.
51. Simonovsky, M., Gutiérrez-Becker, B., Mateus, D., Navab, N., Komodakis, N., A Deep Metric for Multimodal Registration. 2016, MICCAI, 10–18.
52. Simpson, I., Woolrich, M., Groves, A., Schnabel, J., Longitudinal Brain MRI Analysis with Uncertain Registration. 2011, MICCAI, 647–654.
53. Singh, N., Hinkle, J., Joshi, S., Fletcher, P., A Vector Momenta Formulation of Diffeomorphisms for Improved Geodesic Regression and Atlas Construction. 2013, ISBI, 1219–1222.
54. Singh, N., Hinkle, J., Joshi, S., Fletcher, P., A Hierarchical Geodesic Model for Diffeomorphic Longitudinal Shape Analysis. 2013, IPMI, 560–571.
55. Sotiras, A., Davatzikos, C., Paragios, N., Deformable medical image registration: a survey. IEEE Trans. Med. Imaging 32:7 (2013), 1153–1190.
56. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.A., Striving for Simplicity: the All Convolutional Net, CoRR abs/1412.6806. 2014.
57. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: a simple way to prevent neural networks from overfitting. JMLR 15 (2014), 1929–1958.
58. Vaillant, M., Miller, M., Younes, L., Trouvé, A., Statistics on diffeomorphisms via tangent space representations. NeuroImage 23:Suppl. 1 (2004), 161–169.
59. Van Essen, D.C., Smith, S.M., Barch, D.M., Behrens, T.E., Yacoub, E., Ugurbil, K., WU-Minn HCP Consortium, the WU-Minn human connectome project: an overview. NeuroImage 80 (2013), 62–79.
60. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N., Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45:1 (2009), S61–S72.
61. Vialard, F.-X., Risser, L., Rueckert, D., Cotter, C.J., Diffeomorphic 3D image registration via geodesic shooting using an efficient adjoint calculation. IJCV 97:2 (2012), 229–241.
62. Viola, P., W.M.W. III, Alignment by maximization of mutual information. IJCV 24:2 (1997), 137–154.
63. Wang, J., Vachet, C., Rumple, A., Gouttard, S., Ouziel, C., Perrot, E., Du, G., Huang, X., Gerig, G., Styner, M., Multi-atlas segmentation of subcortical brain structures via the autoseg software pipeline. Front. Neuroinformatics, 8, 2014, 7.
64. Wang, Q., Kim, M., Shi, Y., Wu, G., Shen, D., Predict brain MR image registration via sparse learning of appearance & transformation. MedIA 20:1 (2015), 61–75.
65. Wein, W., Brunke, S., Khamene, A., Callstrom, M.R., Navab, N., Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. MedIA 12:5 (2008), 577–585.
66. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C., DeepFlow: Large Displacement Optical Flow with Deep Matching. 2013, ICCV, 1385–1392.
67. Wellek, S., Testing Statistical Hypotheses of Equivalence. 2010, CRC Press.
68. Yang, X., Kwitt, R., Niethammer, M., Fast Predictive Image Registration. 2016, DLMIA/MICCAI, 48–57.
69. Yang, X., Kwitt, R., Styner, M., Niethammer, M., Fast Predictive Multimodal Image Registration. 2017, ISBI, 858–862.
70. Zach, C., Pock, T., Bischof, H., A duality based approach for realtime TV-L1 optical flow. Pattern Recognit., 2007, 214–223.
71. Zhang, M., Fletcher, P., Finite-dimensional Lie Algebras for Fast Diffeomorphic Image Registration. 2015, IPMI, 249–260.