Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples

Journal of Pathology Informatics

Volumn 7, Issue 1, 2016, Pages

  Turkki, Riku ^a   Linder, Nina ^a   Kovanen, Panu E ^b   Pellinen, Teijo ^a   Lundin, Johan ^a,c  

^a UNIVERSITY OF HELSINKI   (Finland)

^b HELSINKI UNIVERSITY CENTRAL HOSPITAL   (Finland)

^c KAROLINSKA INSTITUTE   (Sweden)

Author keywords

breast cancer; convolutional neural network; digital pathology; tumor microenvironment; tumor infiltrating immune cells

Indexed keywords

EID: 85009285527     PISSN: 22295089     EISSN: 21533539     Source Type: Journal    
DOI: 10.4103/2153-3539.189703     Document Type: Article

References (38)

1. Savas P, Salgado R, Denkert C, Sotiriou C, Darcy PK, Smyth MJ, et al. Clinical relevance of host immunity in breast cancer: From TILs to the clinic. Nat Rev Clin Oncol 2016;13:228-41.
2. Loi S, Sirtaine N, Piette F, Salgado R, Viale G, Van Eenoo F, et al. Prognostic and predictive value of tumor-infiltrating lymphocytes in a phase III randomized adjuvant breast cancer trial in node-positive breast cancer comparing the addition of docetaxel to doxorubicin with doxorubicin-based chemotherapy: BIG 02-98. J Clin Oncol 2013;31:860-7.
3. Denkert C, Loibl S, Noske A, Roller M, Müller BM, Komor M, et al. Tumor-associated lymphocytes as an independent predictor of response to neoadjuvant chemotherapy in breast cancer. J Clin Oncol 2010;28:105-13.
4. Salgado R, Denkert C, Campbell C, Savas P, Nuciforo P, Aura C, et al. Tumor-infiltrating lymphocytes and associations with pathological complete response and event-free survival in HER2-positive early-stage breast cancer treated with lapatinib and trastuzumab: A secondary analysis of the neoALTTO trial. JAMA Oncol 2015;1:448-54.
5. Adams S, Gray RJ, Demaria S, Goldstein L, Perez EA, Shulman LN, et al. Prognostic value of tumor-infiltrating lymphocytes in triple-negative breast cancers from two phase III randomized adjuvant breast cancer trials: ECOG 2197 and ECOG 1199. J Clin Oncol 2014;32:2959-66.
6. Salgado R, Denkert C, Demaria S, Sirtaine N, Klauschen F, Pruneri G, et al. The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: Recommendations by an International TILs Working Group 2014. Ann Oncol 2015;26:259-71.
7. Elmore JG, Longton GM, Carney PA, Geller BM, Onega T, Tosteson AN, et al. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA 2015;313:1122-32.
8. Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 2015;20:237-48.
9. Linder N, Konsti J, Turkki R, Rahtu E, Lundin M, Nordling S, et al. Identification of tumor epithelium and stroma in tissue microarrays using texture analysis. Diagn Pathol 2012;7:22.
10. Tuominen VJ, Ruotoistenmäki S, Viitanen A, Jumppanen M, Isola J. ImmunoRatio: A publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Cancer Res 2010;12:R56.
11. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-44.
12. Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 2013;16(Pt 2):411-8.
13. Wang H, Cruz-Roa A, Basavanhally A, Gilmore H, Shih N, Feldman M, et al. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J Med Imaging (Bellingham) 2014;1:034003.
14. Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 2016;6:26286.
15. Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? Advances in neural information processing systems; 2014. p. 3320-8.
16. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 2012;34:2274-82.
17. Girshick R, Donahue J, Darrell T, Malik J. Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 2016;38:142-58.
18. Gong Y, Wang L, Guo R, Lazebnik S. Multi-scale orderless pooling of deep convolutional activation features. Springer International Publishing; 2014.
19. Liu F, Lin G, Shen C. CRF learning with CNN features for image segmentation. Pattern Recognition; 2015.
20. Cimpoi M, Maji S, Vedaldi A. Deep convolutional filter banks for texture recognition and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2014.
21. Chatfield K, Simonyan K, Vedaldi A, Zisserman A. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531; 2014.
22. Socher R. ImageNet: A Large-scale Hierarchical Image Database. 2009 IEEE Conference on Computer Vision Pattern Recognition, IEEE; 2009. p. 248-55.
23. Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:273-97.
24. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ. LIBLINEAR: A library for large linear classification. J Mach Learn Res 2008;9:1871-4.
25. Vedaldi A, Fulkerson B. VLFeat: An Open and Portable Library of Computer Vision Algorithms. Proceedings of the 18th ACM international conference on Multimedia. ACM, 2010.
26. Vedaldi A, Lenc K. MatConvNet-Convolutional neural networks for MATLAB. Proceedings of the 23rd ACM international conference on Multimedia. ACM; 2015.
27. Ojala T, Pietikäinen M, Mäenpää T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 2002;24:971-87.
28. Vedaldi A, Zisserman A. Sparse kernel approximations for efficient classification and detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2012.
29. Basavanhally AN, Ganesan S, Agner S, Monaco JP, Feldman MD, Tomaszewski JE, et al. Computerized image-based detection and grading of lymphocytic infiltration in HER2 breast cancer histopathology. IEEE Trans Biomed Eng 2010;57:642-53.
30. Fatakdawala H, Xu J, Basavanhally A, Bhanot G, Ganesan S, Feldman M, et al. Expectation-maximization-driven geodesic active contour with overlap resolution (EMaGACOR): Application to lymphocyte segmentation on breast cancer histopathology. IEEE Trans Biomed Eng 2010;57:1676-89.
31. Kuse M, Sharma T, Gupta S. A classification scheme for lymphocyte segmentation in H and E stained histology images. Heidelberg: Springer Berlin; 2010. p. 235-43.
32. Panagiotakis C, Ramasso E, Tziritas G. Lymphocyte segmentation using the transferable belief model. Heidelberg: Springer Berlin; 2010. p. 253-62.
33. Yuan Y, Failmezger H, Rueda OM, Ali HR, Gräf S, Chin SF, et al. Quantitative image analysis of cellular heterogeneity in breast tumors complements genomic profiling. Sci Transl Med 2012;4:157ra143.
34. Janowczyk A, Chandran S, Feldman M, Madabhushi A. Local morphologic scale: Application to segmenting tumor infiltrating lymphocytes in ovarian cancer TMAs. SPIE Medical Imaging. International Society for Optics and Photonics; 2011.
35. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Advantages in Neural Information Processing System. Advances in neural information processing systems; 2012. p. 1097-105.
36. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.
37. Turkki R, Linder N, Holopainen T, Wang Y, Grote A, Lundin M, et al. Assessment of tumour viability in human lung cancer xenografts with texture-based image analysis. J Clin Pathol 2015;68:614-21.
38. Nanni L, Lumini A, Brahnam S. Local binary patterns variants as texture descriptors for medical image analysis. Artif Intell Med 2010;49:117-25.