Integrating Online and Offline Three-Dimensional Deep Learning for Automated Polyp Detection in Colonoscopy Videos

IEEE Journal of Biomedical and Health Informatics

Volumn 21, Issue 1, 2017, Pages 65-75

  Yu, Lequan ^a   Chen, Hao ^a   Dou, Qi ^a   Qin, Jing ^b   Heng, Pheng Ann ^a,c  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^c SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

Author keywords

Automated polyp detection; colonoscopy video; computer aided diagnosis; convolutional neural networks (CNNs); deep learning

Indexed keywords

AUTOMATION; COMPUTER AIDED DIAGNOSIS; COMPUTER AIDED INSTRUCTION; CONVOLUTION; CONVOLUTIONAL NEURAL NETWORKS; DEEP NEURAL NETWORKS; DISEASES; E-LEARNING; ENDOSCOPY; LEARNING SYSTEMS; TEXTURES;

COLONOSCOPY; CONVOLUTIONAL NETWORKS; INTEGRATION FRAMEWORKS; INTRA-CLASS VARIATION; ONLINE LEARNING SCHEME; POLYP DETECTION; SPATIO TEMPORAL FEATURES; THREEDIMENSIONAL (3-D);

DEEP LEARNING;

ARCHITECTURE; ARTICLE; ARTIFICIAL NEURAL NETWORK; CANCER DIAGNOSIS; CLINICAL PRACTICE; COLONOSCOPY; COLORECTAL CANCER; MACHINE LEARNING; MEASUREMENT PRECISION; POLYP; SOFTWARE; THREE DIMENSIONAL DEEP LEARNING; TRAINING; COLON POLYP; COMPUTER ASSISTED DIAGNOSIS; DIAGNOSTIC IMAGING; HUMAN; PROCEDURES; THREE DIMENSIONAL IMAGING; VIDEORECORDING;

COLONIC POLYPS; COLONOSCOPY; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; NEURAL NETWORKS (COMPUTER); VIDEO RECORDING;

EID: 85014899291     PISSN: 21682194     EISSN: 21682208     Source Type: Journal    
DOI: 10.1109/JBHI.2016.2637004     Document Type: Article

References (46)

1. R. L. Siegel, K. D. Miller, and A. Jemal, "Cancer statistics, 2016, " CA, Cancer J. Clinicians, vol. 66, pp. 7-30, 2016.
2. A. Leufkens, M. Van Oijen, F. Vleggaar, and P. Siersema, "Factors influencing the miss rate of polyps in a back-to-back colonoscopy study, " Endoscopy, vol. 44, no. 5, pp. 470-475, 2012.
3. L. Rabeneck, H. B. El-Serag, J. A. Davila, and R. S. Sandler, "Outcomes of colorectal cancer in the United States: No change in survival (1986-1997), " Amer. J. Gastroenterology, vol. 98, no. 2, pp. 471-477, 2003.
4. M. Ganz, X. Yang, and G. Slabaugh, "Automatic segmentation of polyps in colonoscopic narrow-band imaging data, " IEEE Trans. Biomed. Eng., vol. 59, no. 8, pp. 2144-2151, Aug. 2012.
5. S. Karkanis, D. K. Iakovidis, D. E. Maroulis, D. Karras, and M. Tzivras, "Computer-aided tumor detection in endoscopic video using color wavelet features, " IEEE Trans. Inf. Technol. Biomed., vol. 7, no. 3, pp. 141-152, Sep. 2003.
6. D. K. Iakovidis, D. E. Maroulis, S. A. Karkanis, and A. Brokos, "A comparative study of texture features for the discrimination of gastric polyps in endoscopic video, " in Proc. 18th IEEE Symp. Comput.-Based Med. Syst., 2005, pp. 575-580.
7. L. A. Alexandre, N. Nobre, and J. Casteleiro, "Color and position versus texture features for endoscopic polyp detection, " in Proc. IEEE 2008 Int. Conf. Biomed. Eng. Informat., 2008, vol. 2, pp. 38-42.
8. S. Ameling, S. Wirth, D. Paulus, G. Lacey, and F. Vilarino, "Texture-based polyp detection in colonoscopy, " in Bildverarbeitung für dieMedizin 2009, New York, NY, USA: Springer, 2009, pp. 346-350.
9. S. Hwang, J. Oh, W. Tavanapong, J. Wong, and P. C. De Groen, "Polyp detection in colonoscopy video using elliptical shape feature, " in Proc. IEEE Int. Conf. Image Process., 2007, vol. 2, pp. II-465-II-468.
10. J. Bernal, J. Sánchez, and F. Vilarino, "Towards automatic polyp detection with a polyp appearance model, " Pattern Recognit., vol. 45, no. 9, pp. 3166-3182, 2012.
11. J. Bernal, F. J. Sánchez, G. Fernández-Esparrach, D. Gil, C. Rodrguez, and F. Vilariño, "WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians, " Comput. Med. Imag. Graph., vol. 43, pp. 99-111, 2015.
12. Y. Wang, W. Tavanapong, J. Wong, J. Oh, and P. C. de Groen, "Partbased multiderivative edge cross-sectional profiles for polyp detection in colonoscopy, " IEEE J. Biomed. Health Informat., vol. 18, no. 4, pp. 1379-1389, Jul. 2014.
13. Y. Wang, W. Tavanapong, J. Wong, J. H. Oh, and P. C. De Groen, "Polypalert: Near real-time feedback during colonoscopy, " Comput. Methods Programs Biomed., vol. 120, no. 3, pp. 164-179, 2015.
14. S. Y. Park, D. Sargent, I. Spofford, K. G. Vosburgh, and A. Yousif, "A colon video analysis framework for polyp detection, " IEEE Trans. Biomed. Eng., vol. 59, no. 5, pp. 1408-1418, May 2012.
15. J. Silva, A. Histace, O. Romain, X. Dray, andB. Granado, "Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer, " Int. J. Comput. Assisted Radiol. Surg., vol. 9, no. 2, pp. 283-293, 2014.
16. N. Tajbakhsh, S. R. Gurudu, and J. Liang, "Automated polyp detection in colonoscopy videos using shape and context information, " IEEE Trans. Med. Imag., vol. 35, no. 2, pp. 630-644, Feb. 2016.
17. A. A. A. Setio et al., "Pulmonary nodule detection in CT images: False positive reduction using multi-view convolutional networks, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1160-1169, May 2016.
18. H. Chen et al., "Automatic fetal ultrasound standard plane detection using knowledge transferred recurrent neural networks, " in International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, NY, USA: Springer, 2015, pp. 507-514.
19. M. Anthimopoulos, S. Christodoulidis, L. Ebner, A. Christe, and S. Mougiakakou, "Lung pattern classification for interstitial lung diseases using a deep convolutional neural network, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1207-1216, May 2016.
20. H.-C. Shin et al., "Deep convolutional neural networks for computeraided detection: CNN architectures, dataset characteristics and transfer learning, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1285-1298, May 2016.
21. N. Tajbakhsh et al., "Convolutional neural networks for medical image analysis: Full training or fine tuning" IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1299-1312, May 2016.
22. H. R. Roth et al., "A new 2. 5D representation for lymph node detection using random sets of deep convolutional neural network observations, " in Proc. Med. Image Comput. Comput.-Assisted Intervention Conf. 2014, New York, NY, USA: Springer, 2014, pp. 520-527.
23. H. Chen, Q. Dou, X. Wang, J. Qin, and P. A. Heng, "Mitosis detection in breast cancer histology images via deep cascaded networks, " in Proc. 30th AAAI Conf. Artif. Intell., 2016, pp. 1160-1166.
24. H. R. Roth et al., "DeepOrgan: Multi-level deep convolutional networks for automated pancreas segmentation, " in International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, NY, USA: Springer, 2015, pp. 556-564.
25. O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation, " in International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, NY, USA: Springer, 2015, pp. 234-241.
26. H. Chen, X. Qi, L. Yu, and P.-A. Heng, "DCAN: Deep contour-aware networks for accurate gland segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 2487-2496.
27. P. Moeskops, M. A. Viergever, A. M. Mendrik, L. S. de Vries, M. J. Benders, and I. Isgum, "Automatic segmentation of MR brain images with a convolutional neural network, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1252-1261, May 2016.
28. H. Chen, X. J. Qi, J. Z. Cheng, and P. A. Heng, "Deep contextual networks for neuronal structure segmentation, " in Proc. 13th AAAI Conf. Artif. Intell., 2016, pp. 1167-1173.
29. D. Nie, L. Wang, Y. Gao, and D. Sken, "Fully convolutional networks for multi-modality isointense infant brain image segmentation, " in Proc. 2016 IEEE 13th Int. Symp. Biomed. Imag., 2016, pp. 1342-1345.
30. N. Tajbakhsh, S. R. Gurudu, and J. Liang, "Automatic polyp detection in colonoscopy videos using an ensemble of convolutional neural networks, " in Proc. 2015 IEEE 12th Int. Symp. Biomed. Imag., 2015, pp. 79-83.
31. T. Brosch, L. Y. Tang, Y. Yoo, D. K. Li, A. Traboulsee, and R. Tam, "Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1229-1239, May 2016.
32. K. Kamnitsas et al., "Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation, " arXiv:1603. 05959, 2016.
33. Q. Dou et al., "Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks, " IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1182-1195, May 2016.
34. R. Li et al., "Deep learning based imaging data completion for improved brain disease diagnosis, " in International Conference on Medical Image Computing and Computer-Assisted Intervention. New York, NY, USA: Springer, 2014, pp. 305-312.
35. S. Ji, W. Xu, M. Yang, and K. Yu, "3D convolutional neural networks for human action recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 221-231, Jan. 2013.
36. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3D convolutional networks, " in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 4489-4497.
37. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3431-3440.
38. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
39. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " CoRR, arXiv:abs/1409. 1556, 2014.
40. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770-778.
41. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
42. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 1725-1732.
43. Y. Jia et al., "Caffe: Convolutional architecture for fast feature embedding, " in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 675-678.
44. B. Thomee et al., "YFCC100M: The new data in multimedia research, " Commun. ACM, vol. 59, no. 2, pp. 64-73, Jan. 2016.
45. A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, "Fitnets: Hints for thin deep nets, " arXiv:1412. 6550, 2014.
46. M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi, "XNOR-Net: ImageNet classification using binary convolutional neural networks, " in Proc. 14th Eur. Conf. Comput. Vis., Amsterdam, The Netherlands, Oct. 11-14, 2016, pp. 525-542