Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 11, 2013, Pages 2592-2607

  Carneiro, Gustavo ^a   Nascimento, Jacinto C ^b  

^a UNIVERSITY OF ADELAIDE   (Australia)

^b INSTITUTO SUPERIOR TÉCNICO   (Portugal)

Author keywords

deep belief networks; discriminative classifiers; dynamical model; Left ventricle segmentation; particle filters

Indexed keywords

DEEP BELIEF NETWORKS; DISCRIMINATIVE CLASSIFIERS; DYNAMICAL MODEL; LEFT VENTRICLES; PARTICLE FILTER;

ULTRASONICS;

IMAGE SEGMENTATION;

ALGORITHM; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; ECHOCARDIOGRAPHY; ECHOGRAPHY; ENDOCARDIUM; HEART LEFT VENTRICLE FUNCTION; HEART VENTRICLE; HUMAN; PROCEDURES; ARTICLE; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; ECHOCARDIOGRAPHY; ENDOCARDIUM; HEART VENTRICLES; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MODELS, CARDIOVASCULAR; PATTERN RECOGNITION, AUTOMATED; VENTRICULAR DYSFUNCTION, LEFT;

EID: 84884546164     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2013.96     Document Type: Article

References (76)

1. J.A. Noble and D. Boukerroui, "Ultrasound Image Segmentation: A Survey," IEEE Trans. Medical Imaging, vol. 25, no. 8, pp. 987- 1010, Aug. 2006. (Pubitemid 44192172)
2. J. Bosch, S. Mitchell, B. Lelieveldt, F. Nijland, O. Kamp, M. Sonka, and J. Reiber, "Automatic Segmentation of Echocardiographic Sequences by Active Appearance Motion Models," IEEE Trans. Medical Imaging, vol. 21, no. 11, pp. 1374-1383, Nov. 2002.
3. E. Bardinet, L. Cohen, and N. Ayache, "Tracking and Motion Analysis of the Left Ventricle with Deformable Superquadratics," Medical Image Analysis, vol. 2, no. 1, pp. 129-149, 1996. (Pubitemid 126421960)
4. D. Geiger, A. Gupta, L.A. Costa, and J. Vlontzos, "Dynamic Programming for Detecting, Tracking, and Matching Deformable Contours," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 17, pp. 294-302, Mar. 1995.
5. T. McInerney and D. Terzopoulos, "A Dynamic Finite Element Surface Model for Segmentation and Tracking in Multidimensional Medical Images with Application to 4-D Image Analysis," Computerized Medical Imaging and Graphics, vol. 1, no. 19, pp. 69-83, 1995.
6. J. Montagnat and H. Delingette, "4D Deformable Models with Temporal Constraints: Application to 4D Cardiac Image Segmentation," Medical Image Analysis, vol. 9, pp. 87-100, 2005. (Pubitemid 40003896)
7. D. Comaniciu, X. Zhou, and S. Krishnan, "Robust Real-Time Myocardial Border Tracking for Echocardiography: An Information Fusion Approach," IEEE Trans. Medical Imaging, vol. 23, no. 7, pp. 849-860, July 2004.
8. M. Lynch, O. Ghita, and P. Whelan, "Segmentation of the Left Ventricle of the Heart in 3-D+T MRI Data Using an Optimized Nonrigid Temporal Model," IEEE Trans. Medical Imaging, vol. 27, no. 2, pp. 195-203, Feb. 2008.
9. N. Paragios, "4-D Deformable Models with Temporal Constraints: Application to 4-D Cardiac Image Segmentation," Medical Image Analysis, vol. 9, no. 1, pp. 87-100, 2005.
10. J. Senegas, T. Netsch, C. Cocosco, G. Lund, and A. Stork, "Segmentation of Medical Images with a Shape and Motion Model: A Bayesian Perspective," Proc. European Conf. Computer Vision Workshops CVAMIA and MMBIA, pp. 157-168, 2004. (Pubitemid 39746719)
11. D. Terzopoulos and R. Szeliski, Tracking with Kalman Snakes. MIT Press, 1993.
12. G. Jacob, J.A. Noble, C. Behrenbruch, A.D. Kelion, and A.P. Banning, "A Shape-Space-Based Approach to Tracking Myocardial Borders and Quantifying Regional Left-Ventricular Function Applied in Echocardiography, " IEEE Trans. Medical Imaging, vol. 21, no. 3, pp. 226-238, Mar. 2002. (Pubitemid 34630840)
13. J. Dias and J. Leitao, "Wall Position and Thickness Estimation from Sequences of Echocardiograms Images," IEEE Trans. Medical Imaging, vol. 15, no. 1, pp. 25-38, Feb. 1996. (Pubitemid 126780296)
14. N. Friedland and D. Adam, "Automatic Ventricular Cavity Boundary Detection from Sequential Ultrasound Images Using Simulated Annealing," IEEE Trans. Medical Imaging, vol. 8, no. 4, pp. 344-353, Dec. 1989. (Pubitemid 20642538)
15. J. Montagnat, M. Sermesant, H. Delingette, G. Maladain, and N. Ayache, "Anisotropic Filtering for Model-Based Segmentation of 4-D Cylindrical Echocardiographic Images," Pattern Recognition Letters, vol. 24, nos. 4/5, pp. 815-828, 2003.
16. S.K. Setarehdan and J.J. Soraghan, "Automatic Cardiac LV Boundary Detection and Tracking Using Hybrid Fuzzy Temporal and Fuzzy Multiscale Edge Detection," IEEE Trans. Biomedical Eng., vol. 46, no. 11, pp. 1364-1378, Nov. 1999. (Pubitemid 29501554)
17. M. Lorenzo-Valdes, G. Sanchez-Ortiz, A. Elkington, R. Mohiaddin, and D. Rueckert, "Segmentation of 4D Cardiac MR Images Using a Probabilistic Atlas and the EM Algorithm," Medical Image Analysis, vol. 8, pp. 255-265, 2004. (Pubitemid 39185909)
18. W. Sun, M. Cetin, R. Chan, and A. Willsky, "Learning the Dynamics and Time-Recursive Boundary Detection of Deformable Objects," IEEE Trans. Image Processing, vol. 17, no. 11, pp. 2186- 2200, Nov. 2008.
19. G. Carneiro and J.C. Nascimento, Multiple Dynamic Models for Tracking the Left Ventricle of the Heart from Ultrasound Data Using Particle Filters and Deep Learning Architectures, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
20. Z. Qian, D. Metaxas, and L. Axel, "Boosting and Nonparametric Based Tracking of Tagged MRI Cardiac Boundaries," Medical Image Computing and Computer-Assisted Intervention, vol. 9, pp. 636- 644, 2006. (Pubitemid 44609159)
21. W. Sun, M. Cetin, R. Chan, V. Reddy, G. Holmvang, V. Chandar, and A. Willsky, "Segmenting and Tracking the Left Ventricle by Learning the Dynamics in Cardiac Images," Proc. 19th Int'l Conf. Information Processing in Medical Imaging, vol. 19, pp. 553-565, 2005. (Pubitemid 41425345)
22. L. Yang, B. Georgescu, Y. Zheng, P. Meer, and D. Comaniciu, "3D Ultrasound Tracking of the Left Ventricle Using One-Step Forward Prediction and Data Fusion of Collaborative Trackers," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
23. Y. Zhu, X. Papademetris, A. Sinusas, and J. Duncan, "Segmentation of the Left Ventricle from Cardiac MR Images Using a Subject-Specific Dynamical Model," IEEE Trans. Medical Imaging, vol. 29, no. 3, pp. 669-687, Mar. 2010.
24. V. Chalana, D. Linker, D. Haynor, and Y. Kim, "A Multiple Active Contour Model for Cardiac Boundary Detection on Echocardiographic Sequences," IEEE Trans. Medical Imaging, vol. 15, no. 3, pp. 290-298, June 1996. (Pubitemid 126780320)
25. M. Jolly, H. Xue, L. Grady, and J. Guehring, "Combining Registration and Minimum Surfaces for the Segmentation of the left Ventricle in Cardiac Cine MR Images," Medical Image Computing and Computer-Assisted Intervention, vol. 12, pp. 910- 918, 2009.
26. J.C. Nascimento and J.S. Marques, "Robust Shape Tracking with Multiple Models in Ultrasound Images," IEEE Trans. Imaging Processing, vol. 17, no. 3, pp. 392-406, Mar. 2008. (Pubitemid 351386298)
27. M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active Contour Models," Int'l J. Computer Vision, vol. 4, no. 1, pp. 321-331, 1987.
28. O. Bernard, B. Touil, A. Gelas, R. Prost, and D. Friboulet, "A RBFBased Multiphase Level Set Method for Segmentation in Echocardiography Using the Statistics of the Radiofrequency Signal," Proc. IEEE Int'l Conf. Image Processing, 2007.
29. O. Bernard, D. Friboulet, P. Thevenaz, and M. Unser, "Variational B-Spline Level-Set: A Linear Filtering Approach for Fast Deformable Model Evolution," IEEE Trans. Image Processing, vol. 18, no. 6, pp. 1179-1991, June 2009.
30. C. Corsi, G. Saracino, A. Sarti, and C. Lamberti, "Left Ventricular Volume Estimation for Real-Time Three-Dimensional Echocardiography," IEEE Trans. Medical Imaging, vol. 21, no. 9, pp. 1202-1208, Sept. 2002.
31. D. Cremers, S. Osher, and S. Soatto, "Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation," Int'l J. Computer Vision, vol. 69, no. 3, pp. 335-351, 2006. (Pubitemid 43847176)
32. E. Debreuve, M. Barlaud, G. Aubert, I. Laurette, and J. Darcourt, "Space-Time Segmentation Using Level Set Active Contours Applied to Myocardial Gated SPECT," IEEE Trans. Medical Imaging, vol. 20, no. 7, pp. 643-659, July 2001. (Pubitemid 32697982)
33. N. Lin, W. Yu, and J. Duncan, "Combinative Multi-Scale Level Set Framework for Echocardiographic Image Segmentation," Medical Image Analysis, vol. 7, no. 4, pp. 529-537, 2003. (Pubitemid 37322315)
34. R. Malladi, J. Sethian, and B. Vemuri, "Shape Modeling with Front Propagation: A Level Set Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 2 pp. 158-175, Feb. 1995.
35. N. Paragios and R. Deriche, "Geodesic Active Regions and Level Set Methods for Supervised Texture Segmentation," Int'l J. Computer Vision, vol. 46, no. 3, pp. 223-247, 2002. (Pubitemid 34265799)
36. N. Paragios, "A Level Set Approach for Shape-Driven Segmentation and Tracking of the Left Ventricle," IEEE Trans. Medical Imaging, vol. 21, no. 9, pp. 773-776, June 2003.
37. A. Sarti, C. Corsi, E. Mazzini, and C. Lamberti, "Maximum Likelihood Segmentation of Ultrasound Images with Rayleigh Distribution," IEEE Trans. Ultrasonic, Ferroelectrics and Frequency Controller, vol. 52, no. 6, pp. 947-960, June 2005. (Pubitemid 41092627)
38. T. Chen, J. Babb, P. Kellman, L. Axel, and D. Kim, "Semiautomated Segmentation of Myocardial Contours for Fast Strain Analysis in Cine Displacement-Encoded MRI," IEEE Trans. Medical Imaging, vol. 27, no. 8, pp. 1084-1094, Aug. 2008.
39. Q. Duan, E.D. Angelini, and A. Laine, "Real Time Segmentation by Active Geometric Functions," Computer Methods and Programs Biomedicine, vol. 98, no. 3, pp. 223-230, 2010.
40. M. Mignotte, J. Meunier, and J. Tardif, "Endocardial Boundary Estimation and Tracking in Echocardiographic Images Using Deformable Template and Markov Random Fields," Pattern Analysis and Applications, vol. 4, no. 4, pp. 256-271, 2001. (Pubitemid 33406041)
41. V. Zagrodsky, V. Walimbe, C. Castro-Pareja, J.X. Qin, J.-M. Song, and R. Shekhar, "Registration-Assisted Segmentation of Real- Time 3-D Echocardiographic Data Using Deformable Models," IEEE Trans. Medical Imaging, vol. 24, no. 9, pp. 1089-1099, Sept. 2005. (Pubitemid 41337150)
42. T. Cootes, C. Taylor, D. Cooper, and J. Graham, "Active Shape Models - Their Training and Application," Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
43. T. Cootes, C. Beeston, G. Edwards, and C. Taylor, "A Unified Framework for Atlas Matching Using Active Appearance Models," Proc. 16th Int'l Conf. Information Processing in Medical Imaging, pp. 322-333, 1999.
44. S. Mitchell, B. Lelieveldt, R. van der Geest, H. Bosch, J. Reiber, and M. Sonka, "Multistage Hybrid Active Appearance Model Matching: Segmentation of Left and Right Ventricles in Cardiac MR Images," IEEE Trans. Medical Imaging, vol. 20, no. 5, pp. 415-423, May 2001. (Pubitemid 32576604)
45. G. Carneiro, B. Georgescu, S. Good, and D. Comaniciu, "Detection and Measurement of Fetal Anatomies from Ultrasound Images Using a Constrained Probabilistic Boosting Tree," IEEE Trans. Medical Imaging, vol. 27, no. 9, pp. 1342-1355, Sept. 2008.
46. B. Georgescu, X.S. Zhou, D. Comaniciu, and A. Gupta, "Databased- Guided Segmentation of Anatomical Structures with Complex Appearance," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
47. J. Weng, A. Singh, and M. Chiu, "Learning-Based Ventricle Detection from Cardiac MR and CT images," IEEE Trans. Medical Imaging, vol. 16, no. 4, pp. 378-391, Aug. 1997. (Pubitemid 127770423)
48. Y. Zheng, A. Barbu, B. Georgescu, M. Scheuering, and D. Comaniciu, "Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features," IEEE Trans. Medical Imaging, vol. 27, no. 11, pp. 1668-1681, Nov. 2008.
49. Y. Freund and R. Schapire, "A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting," J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997. (Pubitemid 127433398)
50. M. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, "A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking," IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 174-188, Feb. 2002. (Pubitemid 34291500)
51. R. Salakhutdinov and G. Hinton, "Learning a Non-Linear Embedding by Preserving Class Neighbourhood Structure," AI and Statistics, 2007.
52. G. Carneiro, J.C. Nascimento, and A. Freitas, Robust Left Ventricle Segmentation from Ultrasound Data Using Deep Neural Networks and Efficient Search Methods, Proc. IEEE Int'l Symp. Biomedical Imaging, from Nano to Macro, pp. 1085-1088, 2010.
53. K. Okuma, A. Taleghani, N. de Freitas, J. Little, and D. Lowe, "A Boosted Particle Filter: Multitarget Detection and Tracking," Proc. European Conf. Computer Vision, 2004.
54. Y. Bengio, "Learning Deep Architectures for AI," Foundations and Trends in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
55. D. Rumelhart, G. Hinton, and R. Williams, "Learning Representations by Back-Propagating Errors," Nature, vol. 323, pp. 533-536, 1986. (Pubitemid 16025374)
56. M. Carreira-Perpinan and G. Hinton, "On Contrastive Divergence Learning," Proc. Workshop Artificial Intelligence and Statistics, 2005.
57. G. Hinton and R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, vol. 313, no. 5786, pp. 504- 507, 2006. (Pubitemid 44148451)
58. G. Hinton, S. Osindero, and Y. Teh, "A Fast Learning Algorithm for Deep Belief Nets," Neural Computation, vol. 18, pp. 1527-1554, 2006. (Pubitemid 44024729)
59. G. Hinton, http://videolectures.net/nips09-hinton-dlmi/, 2013.
60. R. Gonzalez and R. Woods, Digital Image Processing. Prentice Hall, 2008.
61. R. Duda, P. Hart, and D. Stork, Pattern Classification. John Wiley and Sons, 2001.
62. K. Fukunaga, Introduction to Statistical Pattern Recognition. Elsevier, 1990.
63. A. Dempster, M. Laird, and D. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., vol. 39, no. 1, pp. 1-38, 1977.
64. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, Mar. 2004.
65. P. Viola and M. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 511-518, 2001.
66. G. Carneiro, J.C. Nascimento, and A. Freitas, "The Segmentation of the Left Ventricle of the Heart from Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods," IEEE Trans. Image Processing, vol. 21, no. 3, pp. 968-982, Mar. 2012.
67. A. Hammoude, "Computer-Assisted Endocardial Border Identification from a Sequence of Two-Dimensional Echocardiographic Images," PhD dissertation, Univ. of Washington, 1988.
68. X.S. Zhou, D. Comaniciu, and A. Gupta, "An Information Fusion Framework for Robust Shape Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 115-129, Jan. 2005. (Pubitemid 40195902)
69. I. Mikic, S. Krucinki, and J.D. Thomas, "Segmentation and Tracking in Echocardiographic Sequences: Active Contours Guided by Optical Flow Estimates," IEEE Trans. Medical Imaging, vol. 17, no. 2, pp. 274-284, Mar. 1998. (Pubitemid 128747680)
70. V. Chalana and Y. Kim, "A Methodology for Evaluation of Boundary Detection Algorithms on Medical Images," IEEE Trans. Medical Imaging, vol. 16, no. 10, pp. 642-652, Oct. 1997. (Pubitemid 127767643)
71. C. Alberola-Lopez, M. Martin-Fernandez, and J. Ruiz-Alzola, "Comments on: A Methodology for Evaluation of Boundary Detection Algorithms on Medical Images," IEEE Trans. Medical Imaging, vol. 23, no. 5, pp. 658-660, May 2004.
72. J. Bland and A. Altman, "Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement," Lancet, vol. 1, no. 8476, pp. 307-310, 1986. (Pubitemid 16134762)
73. J.C. Reiber, A.R. Viddeleer, G. Koning, M.J. Schalij, and P.E. Lange, "Left Ventricular Regression Equations from Single Plane Cine and Digital X-Ray Ventriculograms Revisited," Clinical Cardiology, vol. 12, no. 2, pp. 69-78, 1996. (Pubitemid 26280872)
74. H. Sandler and H.T. Dodge, "The Use of Single Plane Angiocardiograms for the Calculation of Left Ventricular Volume in Man," Am. Heart J., vol. 75, no. 3, pp. 325-334, 1968.
75. R. Fisher, "Questions and Answers Number 14," The Am. Statistician, vol. 2, no. 5, pp. 30-31, 1948.
76. X. Zhu, "Semi-Supervised Learning Literature Survey," Technical Report 1530, Computer Sciences, Univ. of Wisconsin-Madison, 2005.