Computer-aided prognosis on breast cancer with hematoxylin and eosin histopathology images: A review

Tumor Biology

Volumn 39, Issue 3, 2017, Pages

  Chen, Jia Mei ^a   Li, Yan ^a,b   Xu, Jun ^c   Gong, Lei ^c   Wang, Lin Wei ^a   Liu, Wen Lou ^a   Liu, Juan ^d  

^a ZHONGNAN HOSPITAL OF WUHAN UNIVERSITY   (China)

^b CAPITAL MEDICAL UNIVERSITY   (China)

^c NANJING UNIVERSITY OF INFORMATION SCIENCE AND TECHNOLOGY   (China)

^d WUHAN UNIVERSITY   (China)

Author keywords

Breast cancer; computer aided prognosis; hematoxylin and eosin histopathology image; image analysis

Indexed keywords

EOSIN; HEMATOXYLIN;

BREAST CANCER; CANCER ASSOCIATED FIBROBLAST; CANCER PROGNOSIS; COMPUTER AIDED DESIGN; DIAGNOSTIC ACCURACY; HISTOPATHOLOGY; HUMAN; IMAGE ANALYSIS; IMAGE PROCESSING; IMAGE SEGMENTATION; IMAGING AND DISPLAY; LIMIT OF QUANTITATION; MEASUREMENT PRECISION; MITOSIS; PRIORITY JOURNAL; REPRODUCIBILITY; REVIEW; TUMOR MICROENVIRONMENT; WHOLE SLIDE IMAGE; ALGORITHM; BREAST TUMOR; CLASSIFICATION; COMPUTER ASSISTED DIAGNOSIS; DIAGNOSTIC IMAGING; FEMALE; PATHOLOGY; PROGNOSIS;

ALGORITHMS; BREAST NEOPLASMS; EOSINE YELLOWISH-(YS); FEMALE; HEMATOXYLIN; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PROGNOSIS;

EID: 85016923456     PISSN: 10104283     EISSN: 14230380     Source Type: Journal    
DOI: 10.1177/1010428317694550     Document Type: Review

References (100)

1. Brachtel E, Yagi Y., Digital imaging in pathology—current applications and challenges. J Biophotonics 2012; 5(4): 327–335.
2. Hamilton PW, Bankhead P, Wang YH. Digital pathology and image analysis in tissue biomarker research. Methods 2014; 70(1): 59–73.
3. Rimm DL., C-path: a Watson-like visit to the pathology lab. Sci Transl Med 2011; 3(108): 4601–4608.
4. Bourzac K., Software: the computer will see you now. Nature 2013; 502(7473): S92–S94.
5. He L, Long LR, Antani S. Histology image analysis for carcinoma detection and grading. Comput Methods Programs Biomed 2012; 107(3): 538–556.
6. Chan JK., The wonderful colors of the hematoxylin-eosin stain in diagnostic surgical pathology. Int J Surg Pathol 2014; 22(1): 12–32.
7. Meyer JS, Alvarez C, Milikowski C. Breast carcinoma malignancy grading by Bloom-Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index. Mod Pathol 2005; 18(8): 1067–1078.
8. Shinde V, Burke KE, Chakravarty A. Applications of pathology-assisted image analysis of immunohistochemistry-based biomarkers in oncology. Vet Pathol 2014; 51(1): 292–303.
9. Helin HO, Tuominen VJ, Ylinen O. Free digital image analysis software helps to resolve equivocal scores in her2 immunohistochemistry. Virchows Arch 2016; 468(2): 191–198.
10. Wiesmann V, Franz D, Held C. Review of free software tools for image analysis of fluorescence cell micrographs. J Microsc 2015; 257(1): 39–53.
11. Siegel RL, Miller KD, Jemal A., Cancer statistics, 2015. CA Cancer J Clin 2015; 65(6): 5–29.
12. Veta M, Pluim JP, van Diest PJ. Breast cancer histopathology image analysis: a review. IEEE Trans Biomed Eng 2014; 61(5): 1400–1411.
13. Dong F, Irshad H, Oh EY. Computational pathology to discriminate benign from malignant intraductal proliferations of the breast. PLoS ONE 2014; 9(12): e114885.
14. Filipczuk P, Fevens T, Krzyzak A. Computer-aided breast cancer diagnosis based on the analysis of cytological images of fine needle biopsies. IEEE Trans Med Imaging 2013; 32(12): 2169–2178.
15. Beck AH, Sangoi AR, Leung S. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med 2011; 3(108): 4601–4608.
16. Madabhushi A, Agner S, Basavanhally A. Computer-aided prognosis: predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data. Comput Med Imaging Graph 2011; 35(7–8): 506–514.
17. Wang C, Pecot T, Zynger DL. Identifying survival associated morphological features of triple negative breast cancer using multiple datasets. J Am Med Inform Assoc 2013; 20(4): 680–687.
18. Chen JM, Qu AP, Wang LW. New breast cancer prognostic factors identified by computer-aided image analysis of HE stained histopathology images. Sci Rep 2015; 5: 10690.
19. Rivenbark AG, O’Connor SM, Coleman WB., Molecular and cellular heterogeneity in breast cancer: challenges for personalized medicine. Am J Pathol 2013; 183(4): 1113–1124.
20. Goldhirsch A, Winer EP, Coates AS. Personalizing the treatment of women with early breast cancer: highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer 2013. Ann Oncol 2013; 24(9): 2206–2223.
21. Rakha EA, Ellis IO., Modern classification of breast cancer: should we stick with morphology or convert to molecular profile characteristics. Adv Anat Pathol 2011; 18(4): 255–267.
22. Reis-Filho JS, Pusztai L., Gene expression profiling in breast cancer: classification, prognostication, and prediction. Lancet 2011; 378(9805): 1812–1823.
23. Drukker CA, Nijenhuis MV, Bueno-de-Mesquita JM. Optimized outcome prediction in breast cancer by combining the 70-gene signature with clinical risk prediction algorithms. Breast Cancer Res Treat 2014; 145(3): 697–705.
24. Jegadeesh NK, Kim S, Prabhu RS. The 21-gene recurrence score and locoregional recurrence in breast cancer patients. Ann Surg Oncol 2015; 22(4): 1088–1094.
25. Fumagalli D, Andre F, Piccart-Gebhart MJ. Molecular biology in breast cancer: should molecular classifiers be assessed by conventional tools or by gene expression arrays? Crit Rev Oncol Hematol 2012; 84(Suppl. 1): e58–e69.
26. Weigelt B, Reis-Filho JS., Molecular profiling currently offers no more than tumour morphology and basic immunohistochemistry. Breast Cancer Res 2010; 12(Suppl. 4): 1–4.
27. Elsawaf Z, Sinn HP, Rom J. Biological subtypes of triple-negative breast cancer are associated with distinct morphological changes and clinical behaviour. Breast 2013; 22(5): 986–992.
28. Rakha EA, Reis-Filho JS, Baehner F. Breast cancer prognostic classification in the molecular era: the role of histological grade. Breast Cancer Res 2010; 12(4): 1–12.
29. Bansal C, Singh US, Misra S. Comparative evaluation of the modified Scarff-Bloom-Richardson grading system on breast carcinoma aspirates and histopathology. Cytojournal 2012; 9(1): 4.
30. Zhang R, Chen HJ, Wei B. Reproducibility of the Nottingham modification of the Scarff-Bloom-Richardson histological grading system and the complementary value of Ki-67 to this system. Chin Med J 2010; 123(15): 1976–1982.
31. Liu Y, Xin T, Jiang QY. CD147, MMP9 expression and clinical significance of basal-like breast cancer. Med Oncol 2013; 30(1): 366.
32. Zeng H, Zheng R, Zhang S. Female breast cancer statistics of 2010 in china: estimates based on data from 145 population-based cancer registries. J Thorac Dis 2014; 6(6): 466–470.
33. Heindl A, Nawaz S, Yuan YY., Mapping spatial heterogeneity in the tumor microenvironment: a new era for digital pathology. Lab Invest 2015; 95(4): 377–384.
34. Matsumoto H, Koo SL, Dent R. Role of inflammatory infiltrates in triple negative breast cancer. J Clin Pathol 2015; 68(7): 506–510.
35. Sahoo PK, Jana D, Mandal PK. Effect of lymphangiogenesis and lymphovascular invasion on the survival pattern of breast cancer patients. Asian Pac J Cancer Prev 2014; 15(15): 6287–6293.
36. Jang SY, Kim A, Kim JK. Metformin inhibits tumor cell migration via down-regulation of MMP9 in tamoxifen-resistant breast cancer cells. Anticancer Res 2014; 34(8): 4127–4134.
37. Al-Janabi S, Huisman A, Willems SM. Digital slide images for primary diagnostics in breast pathology: a feasibility study. Hum Pathol 2012; 43(12): 2318–2325.
38. Krishnamurthy S, Mathews K, McClure S. Multi-institutional comparison of whole slide digital imaging and optical microscopy for interpretation of hematoxylin-eosin-stained breast tissue sections. Arch Pathol Lab Med 2013; 137(12): 1733–1739.
39. Gurcan MN, Boucheron LE, Can A. Histopathological image analysis: a review. IEEE Rev Biomed Eng 2009; 2: 147–171.
40. Huang CH, Veillard A, Roux L. Time-efficient sparse analysis of histopathological whole slide images. Comput Med Imaging Graph 2011; 35(7–8): 579–591.
41. Xu J, Xiang L, Liu Q. Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans Med Imaging 2016; 35(1): 119–130.
42. Bhattacharyya S., A brief survey of color image preprocessing and segmentation techniques. J Pattern Recognit Res 2011; 6: 120–129.
43. Khan AM, Rajpoot N, Treanor D. A nonlinear mapping approach to stain normalization in digital histopathology images using image-specific color deconvolution. IEEE Trans Biomed Eng 2014; 61(6): 1729–1738.
44. Reinhard E, Ashikhmin M, Gooch B. Color transfer between images. IEEE Comput Graph 2001; 21(5): 34–41.
45. Vantaram SR, Saber E., Survey of contemporary trends in color image segmentation. J Electron Imaging 2012; 21(4): 177–187.
46. Petushi S, Garcia FU, Haber MM. Large-scale computations on histology image reveal grade-differentiating parameters for breast cancer. BMC Med Imaging 2006; 6(1): 1–11.
47. Irshad H, Veillard A, Roux L. Methods for nuclei detection, segmentation, and classification in digital histopathology: a review—current status and future potential. IEEE Rev Biomed Eng 2014; 7(1–5): 97–114.
48. Veta M, van Diest PJ, Kornegoor R. Automatic nuclei segmentation in H&E stained breast cancer histopathology images. PLoS ONE 2013; 8(7): e70221.
49. Veta M, van Diest PJ, Willems SM. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 2015; 20(1): 237–248.
50. Roux L, Racoceanu D, Lomenie N. Mitosis detection in breast cancer histological images an ICPR 2012 contest. J Pathol Inform 2013; 4: 8.
51. Basavanhally A, Yu E, Xu J. Incorporating domain knowledge for tubule detection in breast histopathology using O’Callaghan neighborhoods. In: Proceedings of the SPIE medical imaging, Lake Buena Vista, FL, 12 February 2011, paper no. 796310. Bellingham, WA: SPIE.
52. Linder N, Konsti J, Turkki R. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn Pathol 2012; 7(22): 1–11.
53. Weigelt B, Geyer FC, Reis-Filho JS., Histological types of breast cancer: how special are they? Mol Oncol 2010; 4(3): 192–208.
54. Parvin B, Yang Q, Han J. Iterative voting for inference of structural saliency and characterization of subcellular events. IEEE Trans Image Process 2007; 16(3): 615–623.
55. Al-Kofahi Y, Lassoued W, Lee W. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Trans Biomed Eng 2010; 57(4): 841–852.
56. Arteta C, Lempitsky V, Alison N. Learning to detect cells using non-overlapping extremal regions. Med Image Comput Comput Assist Interv 2012; 7510(Pt 1): 348–356.
57. Yan C, Li A, Zhang B. Automated and accurate detection of soma location and surface morphology in large-scale 3D neuron images. PLoS ONE 2013; 8(4): e62579.
58. Esmaeilsabzali H, Sakaki K, Dechev N. Machine vision-based localization of nucleic and cytoplasmic injection sites on low-contrast adherent cells. Med Biol Eng Comput 2012; 50(1): 11–21.
59. Jung C, Kim C., Segmenting clustered nuclei using H-minima transform-based marker extraction and contour parameterization. IEEE Trans Biomed Eng 2010; 57(10): 2600–2604.
60. Li F, Zhou X, Ma J. Multiple nuclei tracking using integer programming for quantitative cancer cell cycle analysis. IEEE Trans Med Imaging 2010; 29(1): 96–105.
61. Ali S, Madabhushi A., An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery. IEEE Trans Med Imaging 2012; 31(17): 1448–1460.
62. Cheng JR, Rajapakse J., Segmentation of clustered nuclei with shape markers and marking function. IEEE Trans Biomed Eng 2009; 56(3): 741–748.
63. Basavanhally AN, Ganesan S, Agner S. Computerized image-based detection and grading of lymphocytic infiltration in her2+ breast cancer histopathology. IEEE Trans Biomed Eng 2010; 57(3): 642–653.
64. Fatakdawala H, Xu J, Basavanhally A. Expectation-maximization-driven geodesic active contour with overlap resolution (EMaGACOR): application to lymphocyte segmentation on breast cancer histopathology. IEEE Trans Biomed Eng 2010; 57(7): 1676–1689.
65. Vink JP, Van Leeuwen MB, Van Deurzen CH. Efficient nucleus detector in histopathology images. J Microsc 2013; 249(2): 124–135.
66. Hoque A, Lippman SM, Boiko IV. Quantitative nuclear morphometry by image analysis for prediction of recurrence of ductal carcinoma in situ of the breast. Cancer Epidemiol Biomarkers Prev 2001; 10(3): 249–259.
67. Qi X, Xing FY, Foran DJ. Robust segmentation of overlapping cells in histopathology specimens using parallel seed detection and repulsive level set. IEEE Trans Biomed Eng 2012; 59(3): 754–765.
68. Latson L, Sebek B, Powell KA., Automated cell nuclear segmentation in color images of hematoxylin and eosin-stained breast biopsy. Anal Quant Cytol Histol 2003; 25(6): 321–331.
69. Veta M, Kornegoor R, Huisman A. Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer. Mod Pathol 2012; 25(12): 1559–1565.
70. Mouelhi A, Sayadi M, Fnaiech F. A supervised segmentation scheme based on multilayer neural network and color active contour model for breast cancer nuclei detection. In: 2013 international conference on electrical engineering and software applications (ICEESA), Hammamet, Tunisia, 21–23 March 2013, vol. 53, pp. 1–6. New York: IEEE.
71. Ruifrok AC, Johnston DA., Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol 2001; 23(4): 291–299.
72. Ciresan DC, Giusti A, Gambardella LM. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 2013; 16(2): 411–418.
73. Pang BC, Zhang Y, Chen QQ. Cell nucleus segmentation in color histopathological imagery using convolutional networks. In: Chinese conference on pattern recognition, Chongqing, China, 21–23 October 2010, pp. 1–5. New York: IEEE.
74. Wienert S, Heim D, Saeger K. Detection and segmentation of cell nuclei in virtual microscopy images: a minimum-model approach. Sci Rep 2012; 2(7): 503.
75. Yuan Y, Failmezger H, Rueda OM. Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Sci Transl Med 2012; 4(157): 157ra143.
76. Yuan Y., Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer. J R Soc Interface 2015; 12: 103.
77. Kulikova M, Veillard A, Roux L. Nuclei extraction from histopathological images using a marked point process approach. In: Proceedings of the SPIE medical imaging, San Diego, CA, 4 February 2012, paper no. 831428. Bellingham, WA: SPIE.
78. Jung C, Kim C, Chae SW. Unsupervised segmentation of overlapped nuclei using Bayesian classification. IEEE Trans Biomed Eng 2010; 57(12): 2825–2832.
79. Chang JM, McCullough AE, Dueck AC. Back to basics: traditional Nottingham grade mitotic counts alone are significant in predicting survival in invasive breast carcinoma. Ann Surg Oncol 2015; 22(Suppl. 3): S509–S515.
80. Belien JA, Baak JP, van Diest PJ. Counting mitoses by image processing in feulgen stained breast cancer sections: the influence of resolution. Cytometry 1997; 28: 135–140.
81. Giusti A, Caccia C, Ciresari DC. A comparison of algorithms and humans for mitosis detection. In: 2014 IEEE 11th international symposium on biomed imaging (ISBI), Beijing, China, 29 April–2 May 2014, pp. 1360–1363. New York: IEEE.
82. Al-Janabi S, van Slooten HJ, Visser M. Evaluation of mitotic activity index in breast cancer using whole slide digital images. PLoS ONE 2013; 8(12): e82576.
83. Kawalkar P, Talmale G. Review paper on histopathological image analysis approach for automatic detection of glandular structures in human tissue. In: 2015 international conference on pervasive computing (ICPC), Pune, India, 8–10 January 2015, pp. 1–5. New York: IEEE.
84. Dalle JR, Leow WK, Racoceanu D. Automatic breast cancer grading of histopathological images. Conf Proc IEEE Eng Med Bio 2008; 2008: 3052–3055.
85. Naik S, Doyle S, Agner S. Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology. In: 5th IEEE international symposium on biomedical imaging: from nano to macro, Paris, France, 14–17 May 2008, pp. 284–287. New York: IEEE.
86. Nguyen K, Srinivas C. Automatic glandular and tubule region segmentation in histological grading of breast cancer. In: Proceedings of the SPIE medical imaging, Orlando, FL, 21 February 2015, paper no. 94200G. Bellingham, WA: SPIE.
87. Irshad H., Automated mitosis detection in histopathology using morphological and multi-channel statistics features. J Pathol Inform 2013; 4(1): 10.
88. Tek FB., Mitosis detection using generic features and an ensemble of cascade adaboosts. J Pathol Inform 2013; 4: 12.
89. Basavanhally A, Ganesan S, Feldman M. Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides. IEEE Trans Biomed Eng 2013; 60(8): 2089–2099.
90. Chapman JA, Miller NA, Lickley HL. Ductal carcinoma in situ of the breast (DCIS) with heterogeneity of nuclear grade: prognostic effects of quantitative nuclear assessment. BMC Cancer 2007; 7(1): 174.
91. Irshad H, Jalali S, Roux L. Automated mitosis detection using texture, SIFT features and HMAX biologically inspired approach. J Pathol Inform 2013; 4(Suppl): S12.
92. Loukas C, Kostopoulos S, Tanoglidi A. Breast cancer characterization based on image classification of tissue sections visualized under low magnification. Comput Math Methods Med 2013; 2013(5): 637–652.
93. Khan AM, Eldaly H, Rajpoot NM., A gamma-Gaussian mixture model for detection of mitotic cells in breast cancer histopathology images. J Pathol Inform 2013; 4: 11.
94. Cosatto E, Miller M, Graf HP. Grading nuclear pleomorphism on histological micrographs. In: 9th international conference on pattern recognition (ICPR), Tampa, FL, 8–11 December 2008, pp. 1–4. New York: IEEE
95. Bouzas D, Arvanitopoulos N, Tefas A., Graph embedded nonparametric mutual information for supervised dimensionality reduction. IEEE Trans Neural Netw Learn Syst 2015; 26(5): 951–963.
96. Mignotte M., MDS-based multiresolution nonlinear dimensionality reduction model for color image segmentation. IEEE Trans Neural Netw 2011; 22(3): 447–460.
97. Tutac AE, Racoceanu D, Putti T. Knowledge-guided semantic indexing of breast cancer histopathology images. In: International conference on biomedical engineering and informatics (BMEI), Sanya, Hainan China, 27–30 May 2008, vol. 2, pp. 107–112. New York: IEEE.
98. Nawaz S, Heindl A, Koelble K. Beyond immune density: critical role of spatial heterogeneity in estrogen receptor-negative breast cancer. Mod Pathol 2015; 28(6): 1621.
99. Steyerberg EW, Moons KG, van der Windt DA. Prognosis research strategy (PROGRESS) 3: prognostic model research. PLoS Med 2013; 10(2): 458–459.
100. Basavanhally A, Feldman M, Shih N. Multi-field-of-view strategy for image-based outcome prediction of multi-parametric estrogen receptor-positive breast cancer histopathology: comparison to Oncotype DX. J Pathol Inform 2011; 2(2): S1.