A deep convolutional neural network-based framework for automatic fetal facial standard plane recognition

IEEE Journal of Biomedical and Health Informatics

Volumn 22, Issue 3, 2018, Pages 874-885

  Yu, Zhen ^a   Tan, Ee Leng ^b   Ni, Dong ^a   Qin, Jing ^c   Chen, Siping ^a   Li, Shengli ^d   Lei, Baiying ^a   Wang, Tianfu ^a  

^a Shenzhen University School of Medicine   (China)

^b Beijing Datum Science and Technology Development Co Ltd   (China)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^d Nanfang Medical University   (China)

Author keywords

Deep convolutional neural network; standard plane recognition; transfer learning; ultrasound image

Indexed keywords

CONVOLUTION; DEEP NEURAL NETWORKS; DIAGNOSIS; FACE RECOGNITION; TRANSFER LEARNING; ULTRASONIC IMAGING;

EXAMINATION METHODS; INTRA-CLASS VARIATION; LIMITED TRAINING DATA; NETWORK PARAMETERS; OVER FITTING PROBLEM; PLANE RECOGNITION; TRADITIONAL APPROACHES; ULTRASOUND IMAGES;

CONVOLUTIONAL NEURAL NETWORKS;

ARTICLE; DIAGNOSIS; FACE; NERVOUS SYSTEM; PRENATAL DIAGNOSIS; TRANSFER OF LEARNING; ULTRASOUND; ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER ASSISTED DIAGNOSIS; DIAGNOSTIC IMAGING; FEMALE; FETUS; FETUS ECHOGRAPHY; HUMAN; PREGNANCY; PROCEDURES;

ALGORITHMS; FACE; FEMALE; FETUS; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; NEURAL NETWORKS (COMPUTER); PREGNANCY; ULTRASONOGRAPHY, PRENATAL;

EID: 85045917483     PISSN: 21682194     EISSN: 21682208     Source Type: Journal    
DOI: 10.1109/JBHI.2017.2705031     Document Type: Article

References (68)

1. B. Lei et al., "Automatic recognition of fetal facial standard plane in ultrasound image via fisher vector," PLoS One, vol. 10, no. 5, 2015, Art. no. e0121838.
2. B. Lei et al., "Discriminative learning for automatic staging of placental maturity via multi-layer fisher vector," Sci. Rep., vol. 5, no. 2015, Art. no. 12818.
3. B. Rahmatullah,A. Papageorghiou, and J.A.Noble, "Automated selection of standardized planes from ultrasound volume," in Proc. Mach. Learn. Med. Imag., 2011, pp. 35-42.
4. H. Chen et al., "Automatic fetal ultrasound standard pane detection using knowledge transferred recurrent neural networks," in Proc. Med. Imag. Comput. Comput. Assist. Interv., 2015, pp. 507-514.
5. H. Chen et al., "Standard plane localization in fetal ultrasound via domain transferred deep neural networks," IEEE J. Biomed. Health. Inf., vol. 19, no. 5, pp. 1627-1636, 2015.
6. B. Lei, L. Zhuo, S. Chen, S. Li, D. Ni, and T.Wang, "Automatic recognition of fetal standard plane in ultrasound image," in Proc. IEEE 11th Int. Symp. Biomed. Imag., 2014, pp. 85-88.
7. L. Zhang, S. Chen, C. T. Chin, T. Wang, and S. Li, "Intelligent scanning: Automated standard plane selection and biometric measurement of early gestational sac in routine ultrasound examination," Med. Phys., vol. 39, no. 8, pp. 5015-5027, 2012.
8. B. Rahmatullah and J. A. Noble, "Anatomical object detection in fetal ultrasound: Computer-expert agreements," in Proc. Biomed. Inf. Technol., 2014, pp. 207-218.
9. N. Dudley and E. Chapman, "The importance of quality management in fetal measurement," Ultrasound Obstet. Gynecol., vol. 19, no. 2, pp. 190-196, 2002.
10. M. Yaqub, B. Kelly, A. Papageorghiou, and J. A. Noble, "Guided random forests for identification of key fetal anatomy and image categorization in ultrasound scans," in Proc. Med. Imag. Comput. Comput. Assist. Interv., 2015, pp. 687-694.
11. C. F. Baumgartner, K. Kamnitsas, J. Matthew, S. Smith, B. Kainz, and D. Rueckert, "Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks," in Proc. Med. Imag. Comput. Comput. Assist. Interv., 2016, pp. 203-211.
12. K. Chatfield, V. S. Lempitsky, A. Vedaldi, and A. Zisserman, "The devil is in the details: An evaluation of recent feature encoding methods," in Proc. Brit. Mach. Vis. Conf., 2011, pp. 76.1-76.12.
13. J. Shi, Q. Jiang, R. Mao, M. Lu, and T. Wang, "FR-KECA: Fuzzy robust kernel entropy component analysis," Neurocomputing, vol. 149, Part C, pp. 1415-1423, 2015.
14. J. Shi, J. Wu, Y. Li, Q. Zhang, and S. Ying, "Histopathological image classification with color pattern random binary hashing based PCANet and matrix-form classifier," IEEE J. Biomed. Health Inf., doi: 10.1109/JBHI.2016.2602823.
15. J. Shi, S. Zhou, X. Liu, Q. Zhang, M. Lu, and T. Wang, "Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset," Neurocomputing, vol. 194, pp. 87-94, 2016.
16. D. Ni et al., "Standard plane localization in ultrasound by radial component model and selective search," Ultrasound Med. Biol., vol. 40, no. 11, pp. 2728-2742, 2014.
17. B. Lei, E.-L. Tan, S. Chen, D. Ni, and T. Wang, "Saliency-driven image classification method based on histogram mining and image score," Pattern Recognit., vol. 48, no. 8, pp. 2567-2580, 2015.
18. X. Zhu, X. Li, and S. Zhang, "Block-row sparse multiview multilabel learning for image classification," IEEE Trans. Cybern., vol. 46, no. 2, pp. 450-461, Feb. 2016.
19. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
20. C. Liu, J. Yuen, and A. Torralba, "SIFT Flow: Dense correspondence across scenes and its applications," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 978-994, May 2011.
21. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2005, pp. 886-893.
22. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," in Proc. Eur. Conf. Comput. Vis. Workshop Statist. Learn. Comput. Vis., pp. 950-953, 2011.
23. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramidmatching for recognizing natural scene categories," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2169-2178.
24. H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, and C. Schmid, "Aggregating local image descriptors into compact codes," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 1704-1716.
25. F. Perronnin, J. Sanchez, and T.Mensink, "Improving the fisher kernel for large-scale image classification," in Proc. 11th Eur. Conf. Comput. Vis., 2010, pp. 143-156.
26. J. Sanchez, F. Perronnin, T. Mensink, and J.Verbeek, "Image classification with the fisher vector: Theory and practice," Int. J. Comput. Vis., vol. 105, no. 3, pp. 222-245, 2013.
27. J. Deng, W. Dong, R. Socher, K. L. L.-J. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 248-255.
28. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
29. M. Lin, Qi. Chen, and S. Yan, "Network in network," arXiv: 1312.4400, 2013.
30. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770-778.
31. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in Proc. Int. Conf. Mach. Learn., 2015, pp. 448-456.
32. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int.Conf. Learn. Represent., 2015, pp. 1-14.
33. C. Szegedy et al., "Going deeper with convolutions," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1-9.
34. J. Donahue et al., "Decaf: A deep convolutional activation feature for generic visual recognition," in Proc. Int. Conf. Mach. Learn., 2014, pp. 647-655.
35. Q. Dou et al., "Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks," IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1182-1195, Feb. 2016.
36. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 580-587.
37. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3431-3440.
38. O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Proc. Med. Imag. Comput. Comput. Assist. Interv., 2015, pp. 234-241.
39. Q. Li,W. Cai, X.Wang, Y. Zhou, D. Feng, and M. Chen, "Medical image classification with convolutional neural network," in Proc. 13th Int. Conf. Control Autom. Robot. Vis., 2014, pp. 844-848.
40. X. Zhu, H. Suk, L. Wang, S.-W. Lee, and D. Shen, "A novel relational regularization feature selection method for joint regression and classification in AD diagnosis," Med. Imag. Anal., vol. 38, pp. 205-214, 2017.
41. X. Zhu, X. Li, S. Zhang, C. Ju, and X. Wu, "Robust joint graph sparse coding for unsupervised spectral feature selection," IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 6, pp. 1263-1275, Jun. 2017.
42. X. Zhu, L. Zhang, and Z. Huang, "A sparse embedding and least variance encoding approach to hashing," IEEE Trans. Image Process., vol. 23, no. 9, pp. 3737-3750, Sep. 2014.
43. S. Maji, A. C. Berg, and J. Malik, "Classification using intersection kernel support vector machines is efficient," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
44. K. Grauman and T. Darrell, "The pyramid match kernel: Discriminative classification with sets of image features," in Proc. IEEE Int. Conf. Comput. Vis., 2005, pp. 1458-1465.
45. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
46. H. Greenspan, B. van Ginneken, and R. M. Summers, "Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique," IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1153-1159, May 2016.
47. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks," in Proc. Int. Conf. Artif. Intell. Statist., 2010, pp. 249-256.
48. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1026-1034.
49. N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting," J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929-1958, 2014.
50. I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. C. Courville, and Y. Bengio, "Maxout networks," in Proc. 30th Int. Conf. Mach. Learn., 2013, pp. 1319-1327.
51. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks?," in Proc. Adv. Neural Inform. Process. Syst., 2014, pp. 3320-3328.
52. Y. Bengio, "Deep learning of representations for unsupervised and transfer learning," in Proc. ICML Workshop Unsupervised Transfer Learn., 2012, pp. 1-20.
53. M. Anthimopoulos, S. Christodoulidis, A. C. L. Ebner, and S. Mougiakakou, "Lung pattern classification for interstitial lung diseases using a deep convolutional neural network," IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1207-1216, May 2016.
54. Z. Gao, L. Wang, L. Zhou, and J. Zhang, "Hep-2 cell image classification with deep convolutional neural networks," IEEE J. Biomed. Health. Inf., vol. 21, no. 2, pp. 416-428, Mar. 2017.
55. Z. Yan et al., "Multi-instance deep learning: Discover discriminative local anatomies for bodypart recognition," IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1332-1343, May 2016.
56. Y. Song, L. Zhang, S. Chen, D. Ni, B. Lei, and T. Wang, "Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning," IEEE Trans. Biomed. Eng., vol. 62, no. 10, pp. 2421-2433, Oct. 2015.
57. Z. Yu, D. Ni, S. Chen, S. Li, T. Wang, and B. Lei, "Fetal facial standard plane recognition via very deep convolutional networks," in Proc IEEE 38th Annu. Int. Conf. Eng. Med. Biol. Soc, 2016, pp. 627-630.
58. H. Chen, Q. Dou, X. Wang, J. Qin, and P. A. Heng, "Mitosis detection in breast cancer histology images via deep cascaded networks," in Proc. AAAI Conf. Artif. Intell., 2016, pp. 1160-1166.
59. Y. Song, L. He, F. Zhou, S. Chen, D. Ni, B. Lei, and T. Wang, "Segmentation, splitting, and classification of overlapping bacteria in microscope images for automatic bacterial vaginosis diagnosis," doi: 10.1109/JBHI.2016.2594239.
60. H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P. A. Heng, "DCAN: Deep contour-aware networks for object instance segmentation from histology images," Med. Image Anal., vol. 36, no. pp. 135-146, 2017.
61. H. Chen, Q. Dou, L. Yu, J. Qin, and P. A. Heng, "VoxResNet: Deep voxelwise residual networks for brain segmentation from 3DMR images," Neuroimage, doi: https://doi.org/10.1016/j.neuroimage.2017.04.041.
62. H. Chen, L. Wu, Q. Dou, J. Qin, S. Li, J. Z. Cheng, D. Ni, and P. A. Heng, "Ultrasound standard plane detection using a composite neural network framework," IEEE Trans. Cybern., vol. 47, no. 6, pp. 1576-1586, Jun. 2017.
63. K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, "Return of the devil in the details: Delving deep into convolutional nets," in Proc. Brit. Mach. Vis. Conf., 2014, pp. 1-12.
64. P. Hensman and D. Masko, "The impact of imbalanced training data for convolutional neural networks," Degree Project in Computer Science DD143X, 2015.
65. N. Tajbakhsh et al., "Convolutional neural networks for medical image analysis: Full training or fine tuning?," IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1299-1312, May 2016.
66. S. Shalev-Shwartz and Z. Tong, "Stochastic dual coordinate ascent methods for regularized loss minimization," J. Mach. Learn. Res., vol. 14, no. 2, pp. 567-599, 2013.
67. B. Scholkopf and A. J. Smola, Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond.Cambridge, MA, USA: MIT Press, 2001.
68. A. Vedaldi and B. Fulkerson, "VLFeat: An open and portable library of computer vision algorithms," in Proc. 18th Int. Conf. Multimedia, 2010, pp. 1469-1472.