Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation

IEEE Transactions on Medical Imaging

Volumn 37, Issue 2, 2018, Pages 384-395

  Oktay, Ozan ^a   Ferrante, Enzo ^a   Kamnitsas, Konstantinos ^a   Heinrich, Mattias ^b   Bai, Wenjia ^a   Caballero, Jose ^a   Cook, Stuart A ^c   De Marvao, Antonio ^c   Dawes, Timothy ^c   O'Regan, Declan P ^c   Kainz, Bernhard ^a   Glocker, Ben ^a   Rueckert, Daniel ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF LÜBECK   (Germany)

^c MRC CLINICAL SCIENCES CENTRE   (United Kingdom)

Author keywords

convolutional neural network; image super resolution; medical image segmentation; Shape prior

Indexed keywords

IMAGE ANALYSIS; IMAGE RESOLUTION; IMAGE SEGMENTATION; MEDICAL IMAGING; NEURAL NETWORKS;

BIOMEDICAL IMAGING; COMPUTATIONAL MODEL; CONVOLUTIONAL NEURAL NETWORK; IMAGE SUPER RESOLUTIONS; MOTION SEGMENTATION; SHAPE; SHAPE PRIORS;

IMAGE ENHANCEMENT;

ARTICLE; CARDIAC IMAGING; CONTROLLED STUDY; HEART DISEASE; HEART FUNCTION; HEURISTICS; HUMAN; IMAGE ANALYSIS; IMAGE ENHANCEMENT; IMAGE RECONSTRUCTION; IMAGE SEGMENTATION; LEARNING; NERVE CELL NETWORK; NORMAL HUMAN; RECEPTIVE FIELD; THREE DIMENSIONAL IMAGING; ULTRASOUND; ALGORITHM; ARTIFICIAL NEURAL NETWORK; CARDIOMYOPATHY; DIAGNOSTIC IMAGING; FACTUAL DATABASE; HEART; NUCLEAR MAGNETIC RESONANCE IMAGING; PROCEDURES;

ALGORITHMS; CARDIAC IMAGING TECHNIQUES; CARDIOMYOPATHIES; DATABASES, FACTUAL; HEART; HUMANS; IMAGING, THREE-DIMENSIONAL; MAGNETIC RESONANCE IMAGING; NEURAL NETWORKS (COMPUTER);

EID: 85030790219     PISSN: 02780062     EISSN: 1558254X     Source Type: Journal    
DOI: 10.1109/TMI.2017.2743464     Document Type: Article

References (49)

1. G. Alain and Y. Bengio, "What regularized auto-encoders learn from the data-generating distribution, " J. Mach. Learn. Res., vol. 15, no. 1, pp. 3563-3593, 2014.
2. M. R. Avendi, A. Kheradvar, and H. Jafarkhani, "A combined deeplearning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI, " MedIA Image Anal., vol. 30, pp. 108-119, May 2016.
3. M. R. Avendi, A. Kheradvar, and H. Jafarkhani, "Automatic segmentation of the right ventricle from cardiac MRI using a learning-based approach, " in Magnetic Resonance in Medicine. New York, NY, USA: Wiley, Feb. 2017, doi: 10. 1002/mrm. 26631.
4. W. Bai et al., "A bi-ventricular cardiac atlas built from 1000+ high resolution MR images of healthy subjects and an analysis of shape and motion, " MedIA Image Anal., vol. 26, no. 1, pp. 133-145, 2015.
5. W. Bai et al., "A probabilistic patch-based label fusion model for multiatlas segmentation with registration refinement: Application to cardiac MR images, " IEEE Trans. Med. Imag., vol. 32, no. 7, pp. 1302-1315, Jul. 2013.
6. D. Barbosa et al., "Fast and fully automatic 3-D echocardiographic segmentation using B-spline explicit active surfaces: Feasibility study and validation in a clinical setting, " Ultrasound Med. Biol., vol. 39, no. 1, pp. 89-101, 2013.
7. A. BenTaieb and G. Hamarneh, "Topology aware fully convolutional networks for histology gland segmentation, " in Proc. Int. Conf. MICCAI, 2016, pp. 460-468.
8. O. Bernard et al., "Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography, " IEEE Trans. Med. Imag., vol. 35, no. 4, pp. 967-977, Apr. 2016.
9. F. Chen, H. Yu, R. Hu, and X. Zeng, "Deep learning shape priors for object segmentation, " in Proc. IEEE CVPR, Jun. 2013, pp. 1870-1877.
10. H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P.-A. Heng, "DCAN: Deep contour-aware networks for object instance segmentation from histology images, " MedIA Image Anal., vol. 36, pp. 135-146, Feb. 2017.
11. H. Chen, Y. Zheng, J.-H. Park, P.-A. Heng, and S. K. Zhou, "Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images, " in Proc. Int. Conf. MICCAI, 2016, pp. 487-495.
12. Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, "3D U-Net: Learning dense volumetric segmentation from sparse annotation, " in Proc. MICCAI, 2016, pp. 424-432.
13. T. F. Cootes and C. J. Taylor, "Combining point distribution models with shape models based on finite element analysis, " Image Vis. Comput., vol. 13, no. 5, pp. 403-409, 1995.
14. C. Davatzikos, X. Tao, and D. Shen, "Hierarchical active shape models, using the wavelet transform, " IEEE Trans. Med. Imag., vol. 22, no. 3, pp. 414-423, Mar. 2003.
15. A. de Marvao et al., "Population-based studies of myocardial hypertrophy: High resolution cardiovascular magnetic resonance atlases improve statistical power, " J. Cardiovascular Magn. Reson., vol. 16, no. 1, p. 16, 2014.
16. C. Dong, C. C. Loy, and X. Tang, "Accelerating the super-resolution convolutional neural network, " in Proc. ECCV, 2016, pp. 391-407.
17. S. Eslami and C. Williams, "A generative model for parts-based object segmentation, " in Proc. NIPS, 2012, pp. 100-107.
18. S. M. A. Eslami, N. Heess, and J. Winn, "The shape Boltzmann machine: A strong model of object shape, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 406-413.
19. R. Girdhar, D. F. Fouhey, M. Rodriguez, and A. Gupta, "Learning a predictable and generative vector representation for objects, " in Proc. ECCV, 2016, pp. 484-499.
20. H. Greenspan, "Super-resolution in medical imaging, " Comput. J., vol. 52, no. 1, pp. 43-63, 2009.
21. J. Johnson, A. Alahi, and L. Fei-Fei, "Perceptual losses for real-time style transfer and super-resolution, " in Proc. ECCV, 2016, pp. 694-711.
22. K. Kamnitsas et al., "Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, " MedIA Image Anal., vol. 36, pp. 61-78, Feb. 2017.
23. T. D. Karamitsos, J. M. Francis, S. Myerson, J. B. Selvanayagam, and S. Neubauer, "The role of cardiovascular magnetic resonance imaging in heart failure, " J. Amer. College Cardiol., vol. 54, no. 15, pp. 1407-1424, 2009.
24. J. Lafferty, A. McCallum, and F. C. N. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data, " in Proc. ICML, vol. 1. 2001, pp. 282-289.
25. R. M. Lang et al., "EAE/ASE recommendations for image acquisition and display using three-dimensional echocardiography, " J. Amer. Soc. Echocardiogr., vol. 25, no. 1, pp. 1-46, 2012.
26. C. Ledig et al., (Sep. 2016). "Photo-realistic single image superresolution using a generative adversarial network. " [Online]. Available: https://arxiv. org/abs/1609. 04802
27. K. Li, B. Hariharan, and J. Malik, "Iterative instance segmentation, " in Proc. CVPR, 2016, pp. 3659-3667.
28. G. Litjens et al., (2017). "A survey on deep learning in medical image analysis. " [Online]. Available: https://arxiv. org/abs/1702. 05747
29. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation, " in Proc. CVPR, 2015, pp. 3431-3440.
30. P. Luc, C. Couprie, S. Chintala, and J. Verbeek. (Dec. 2016). "Semantic segmentation using adversarial networks. " [Online]. Available: https://arXiv preprint arXiv:1611. 08408
31. J. Masci, U. Meier, D. Cireşan, and J. Schmidhuber, "Stacked convolutional auto-encoders for hierarchical feature extraction, " in Proc. Int. Conf. Artif. Neural Netw., 2011, pp. 52-59.
32. M. S. Nosrati and G. Hamarneh. (2016). "Incorporating prior knowledge in medical image segmentation: A survey. " [Online]. Available: https://arxiv. org/abs/1607. 01092
33. O. Oktay et al., "Stratified decision forests for accurate anatomical landmark localization in cardiac images, " IEEE Trans. Med. Imag., vol. 36, no. 1, pp. 332-342, Jan. 2017.
34. O. Oktay et al., "Multi-input cardiac image super-resolution using convolutional neural networks, " in Proc. Int. Conf. MICCAI, 2016, pp. 246-254.
35. S. E. Petersen et al., "UK Biobank's cardiovascular magnetic resonance protocol, " J. Cardiovascular Magn. Reson., vol. 18, no. 1, p. 8, 2016.
36. H. Ravishankar, S. Thiruvenkadam, R. Venkataramani, and V. Vaidya, "Joint deep learning of foreground, background and shape for robust contextual segmentation, " in Proc. IPMI, 2017, pp. 622-632.
37. H. Ravishankar, R. Venkataramani, S. Thiruvenkadam, P. Sudhakar, and V. Vaidya, "Learning and incorporating shape models for semantic segmentation, " in Proc. ResearchGate, 2017, pp. 203-211.
38. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation, " in Proc. Int. Conf. MICCAI, 2015, pp. 234-241.
39. M. Shakeri, H. Lombaert, S. Tripathi, and S. Kadoury, "Deep spectralbased shape features for Alzheimer's disease classification, " in Proc. SASHIMI, 2016, pp. 15-24.
40. A. Sharma, O. Grau, and M. Fritz, "VConv-DAE: Deep volumetric shape learning without object labels, " in Proc. ECCV Workshops, 2016, pp. 236-250.
41. W. Shi et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, " in Proc. CVPR, 2016, pp. 1874-1883.
42. K. Simonyan and A. Zisserman. (Sep. 2014). "Very deep convolutional networks for large-scale image recognition. " [Online]. Available: https://arxiv. org/abs/1409. 1556
43. A. Torralba and Y. Weiss, "Small codes and large image databases for recognition, " in Proc. CVPR, 2008, pp. 1-8.
44. P. V. Tran. (2016). "A fully convolutional neural network for cardiac segmentation in short-axis MRI. " [Online]. Available: https://arxiv. org/abs/1604. 00494
45. Z. Tu and X. Bai, "Auto-context and its application to high-level vision tasks and 3D brain image segmentation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 10, pp. 1744-1757, Oct. 2010.
46. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, " J. Mach. Learn. Res., vol. 11, pp. 3371-3408, Dec. 2010
47. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity, " IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
48. Z. Wu et al., "3D shapenets: A deep representation for volumetric shapes, " in Proc. CVPR, 2015, pp. 1912-1920.
49. M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus, "Deconvolutional networks, " in Proc. CVPR, 2010, pp. 2528-2535.