Deep Learning Automates the Quantitative Analysis of Individual Cells in Live-Cell Imaging Experiments

PLoS Computational Biology

Volumn 12, Issue 11, 2016, Pages

  Van Valen, David A ^a   Kudo, Takamasa ^b   Lane, Keara M ^a   Macklin, Derek N ^a   Quach, Nicolas T ^a   DeFelice, Mialy M ^a   Maayan, Inbal ^a   Tanouchi, Yu ^a   Ashley, Euan A ^b   Covert, Markus W ^a,b  

^a Stanford University   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CELLS; CONVOLUTION; CYTOLOGY; DEEP NEURAL NETWORKS; FLUORESCENCE; IMAGE SEGMENTATION; SUPERVISED LEARNING;

CELL TYPES; CELLULARS; CONVOLUTIONAL NEURAL NETWORK; CRITICAL CHALLENGES; CURATION; IMAGING EXPERIMENTS; INDIVIDUAL CELLS; LIVE-CELL IMAGING; LIVING SYSTEMS; MAMMALIAN CELLS;

MAMMALS;

BACTERIAL CELL; CELL NUCLEUS; COCULTURE; CYTOPLASM; IMAGING; INTERMETHOD COMPARISON; MAMMAL CELL; NERVOUS SYSTEM; QUANTITATIVE ANALYSIS; QUANTITATIVE STUDY; SUPERVISED MACHINE LEARNING; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CELL TRACKING; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; INTRAVITAL MICROSCOPY; MACHINE LEARNING; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

CELL TRACKING; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INTRAVITAL MICROSCOPY; MACHINE LEARNING; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84999836246     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1005177     Document Type: Article

References (66)

1. Rosenfeld N, Young JW, Alon U, Swain PS, Elowitz MB, Gene regulation at the single-cell level. Science. 2005;307(5717):1962–5. doi: 10.1126/science.110691415790856
2. Weinert FM, Brewster RC, Rydenfelt M, Phillips R, Kegel WK, Scaling of gene expression with transcription-factor fugacity. Physical review letters. 2014;113(25):258101. doi: 10.1103/PhysRevLett.113.25810125554908
3. Selimkhanov J, Taylor B, Yao J, Pilko A, Albeck J, Hoffmann A, et al. Accurate information transmission through dynamic biochemical signaling networks. Science. 2014;346(6215):1370–3. doi: 10.1126/science.125493325504722
4. Tay S, Hughey JJ, Lee TK, Lipniacki T, Quake SR, Covert MW, Single-cell NF-[kgr] B dynamics reveal digital activation and analogue information processing. Nature. 2010;466(7303):267–71. doi: 10.1038/nature0914520581820
5. Lee TK, Denny EM, Sanghvi JC, Gaston JE, Maynard ND, Hughey JJ, et al. A noisy paracrine signal determines the cellular NF-κB response to lipopolysaccharide. Science signaling. 2009;2(93):ra65. doi: 10.1126/scisignal.200059919843957
6. Regot S, Hughey JJ, Bajar BT, Carrasco S, Covert MW, High-sensitivity measurements of multiple kinase activities in live single cells. Cell. 2014;157(7):1724–34. doi: 10.1016/j.cell.2014.04.03924949979
7. Cheong R, Rhee A, Wang CJ, Nemenman I, Levchenko A, Information transduction capacity of noisy biochemical signaling networks. Science. 2011;334(6054):354–8. doi: 10.1126/science.120455321921160
8. Hughey JJ, Lee TK, Covert MW, Computational modeling of mammalian signaling networks. Wiley Interdisciplinary Reviews: Systems Biology and Medicine. 2010;2(2):194–209. doi: 10.1002/wsbm.5220836022
9. Lee TK, Covert MW, High-throughput, single-cell NF-κB dynamics. Current opinion in genetics & development. 2010;20(6):677–83.
10. Sliusarenko O, Heinritz J, Emonet T, Jacobs-Wagner C, High-throughput, subpixel precision analysis of bacterial morphogenesis and intracellular spatio-temporal dynamics. Molecular microbiology. 2011;80(3):612–27. doi: 10.1111/j.1365-2958.2011.07579.x21414037
11. Young JW, Locke JC, Altinok A, Rosenfeld N, Bacarian T, Swain PS, et al. Measuring single-cell gene expression dynamics in bacteria using fluorescence time-lapse microscopy. Nature protocols. 2012;7(1):80–8.
12. Carpenter AE, Jones TR, Lamprecht MR, Clarke C, Kang IH, Friman O, et al. CellProfiler: image analysis software for identifying and quantifying cell phenotypes. Genome biology. 2006;7(10):R100. doi: 10.1186/gb-2006-7-10-r10017076895
13. Malpica N, Ortiz de Solorzano C, Vaquero JJ, Santos A, Vallcorba I, Garcia-Sagredo JM, et al. Applying watershed algorithms to the segmentation of clustered nuclei. 1997.
14. Martin DR, Fowlkes CC, Malik J, Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE transactions on pattern analysis and machine intelligence. 2004;26(5):530–49. doi: 10.1109/TPAMI.2004.127391815460277
15. Mao KZ, Zhao P, Tan P-H, Supervised learning-based cell image segmentation for p53 immunohistochemistry. IEEE Transactions on Biomedical Engineering. 2006;53(6):1153–63. doi: 10.1109/TBME.2006.87353816761842
16. Bertelli L, Yu T, Vu D, Gokturk B, editors. Kernelized structural SVM learning for supervised object segmentation. Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on; 2011: IEEE.
17. Paintdakhi A, Parry B, Campos M, Irnov I, Elf J, Surovtsev I, et al. Oufti: an integrated software package for high-accuracy, high-throughput quantitative microscopy analysis. Molecular microbiology. 2016.
18. Sommer C, Straehle C, Koethe U, Hamprecht FA, editors. ilastik: Interactive learning and segmentation toolkit. Biomedical Imaging: From Nano to Macro, 2011 IEEE International Symposium on; 2011: IEEE.
19. Belevich I, Joensuu M, Kumar D, Vihinen H, Jokitalo E, Microscopy Image Browser: A Platform for Segmentation and Analysis of Multidimensional Datasets. PLoS Biol. 2016;14(1):e1002340. doi: 10.1371/journal.pbio.100234026727152
20. Tanouchi Y, Pai A, Park H, Huang S, Stamatov R, Buchler NE, et al. A noisy linear map underlies oscillations in cell size and gene expression in bacteria. Nature. 2015;523(7560):357–60. doi: 10.1038/nature1456226040722
21. Ali R, Gooding M, Christlieb M, Brady M, editors. Phase-based segmentation of cells from brightfield microscopy. Biomedical Imaging: From Nano to Macro, 2007 ISBI 2007 4th IEEE International Symposium on; 2007: IEEE.
22. Lindblad J, Wahlby C, Bengtsson E, Zaltsman A, Image analysis for automatic segmentation of cytoplasms and classification of Rac1 activation. Cytometry Part A: the journal of the International Society for Analytical Cytology. 2004;57(1):22–33. Epub 2003/12/31. doi: 10.1002/cyto.a.1010714699602.
23. Wang Q, Niemi J, Tan CM, You L, West M, Image segmentation and dynamic lineage analysis in single-cell fluorescence microscopy. Cytometry Part A. 2010;77(1):101–10.
24. Mualla F, Scholl S, Sommerfeldt B, Maier A, Hornegger J, Automatic cell detection in bright-field microscope images using SIFT, random forests, and hierarchical clustering. Medical Imaging, IEEE Transactions on. 2013;32(12):2274–86.
25. Gudla PR, Nandy K, Collins J, Meaburn K, Misteli T, Lockett S, A high-throughput system for segmenting nuclei using multiscale techniques. Cytometry Part A. 2008;73(5):451–66.
26. Chalfoun J, Kociolek M, Dima A, Halter M, Cardone A, Peskin A, et al. Segmenting time-lapse phase contrast images of adjacent NIH 3T3 cells. Journal of microscopy. 2013;249(1):41–52. doi: 10.1111/j.1365-2818.2012.03678.x23126432
27. Rees P, Wills JW, Brown MR, Tonkin J, Holton MD, Hondow N, et al. Nanoparticle vesicle encoding for imaging and tracking cell populations. Nature methods. 2014;11(11):1177–81. doi: 10.1038/nmeth.310525218182
28. Chong YT, Cox MJ, Andrews B, Proteome-wide screens in Saccharomyces cerevisiae using the yeast GFP collection. Advances in Systems Biology: Springer; 2012. p. 169–78.
29. Shedden K, Li Q, Liu F, Chang YT, Rosania GR, Machine vision-assisted analysis of structure-localization relationships in a combinatorial library of prospective bioimaging probes. Cytometry Part A. 2009;75(6):482–93.
30. Chalfoun J, Majurski M, Peskin A, Breen C, Bajcsy P, Brady M, Empirical gradient threshold technique for automated segmentation across image modalities and cell lines. Journal of microscopy. 2015;260(1):86–99. doi: 10.1111/jmi.1226926046924
31. Chen L, Zhang J, Chen S, Lin Y, Yao C, Zhang J, Hierarchical Mergence Approach to Cell Detection in Phase Contrast Microscopy Images. Computational and Mathematical Methods in Medicine. 2014;2014:10. doi: 10.1155/2014/75858724987452
32. LeCun Y, Bengio Y, Hinton G, Deep learning. Nature. 2015;521(7553):436–44. doi: 10.1038/nature1453926017442
33. Krizhevsky A, Sutskever I, Hinton G, Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press; 2012.
34. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:13126229. 2013.
35. Long J, Shelhamer E, Darrell T, editors. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015.
36. Li H, Zhao R, Wang X. Highly efficient forward and backward propagation of convolutional neural networks for pixelwise classification. arXiv preprint arXiv:14124526. 2014.
37. Ronneberger O, Fischer P, Brox T, SpringerU-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 2015. p. 234–41.
38. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J, editors. Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems; 2012.
39. Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J, SpringerMitosis detection in breast cancer histology images with deep neural networks. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2013: 2013. p. 411–8.
40. Kraus OZ, Ba LJ, Frey B. Classifying and Segmenting Microscopy Images Using Convolutional Multiple Instance Learning. arXiv preprint arXiv:151105286. 2015.
41. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R, Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research. 2014;15(1):1929–58.
42. Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:150203167. 2015.
43. Nair V, Hinton GE, editors. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10); 2010.
44. Tieleman T, Hinton G, Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning. 2012;4:2.
45. He K, Zhang X, Ren S, Sun J, editors. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision; 2015.
46. Rumelhart DE, Hinton GE, Williams RJ, Learning representations by back-propagating errors. Nature. 1986;323(6088):533–6.
47. Bergstra J, Breuleux O, Bastien F, Lamblin P, Pascanu R, Desjardins G, et al., editors. Theano: a CPU and GPU math expression compiler. Proceedings of the Python for scientific computing conference (SciPy); 2010: Austin, TX.
48. Van Der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, et al. scikit-image: image processing in Python. PeerJ. 2014;2:e453. doi: 10.7717/peerj.45325024921
49. Chollet F. Keras. GitHub repository: GitHub; 2015.
50. Jones E, Oliphant T, Peterson P. SciPy: Open source scientific tools for Python, 2001. http://www.scipy.org. 2015.
51. Van Der Walt S, Colbert SC, Varoquaux G, The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering. 2011;13(2):22–30.
52. Marquez-Neila P, Baumela L, Alvarez L, A morphological approach to curvature-based evolution of curves and surfaces. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 2014;36(1):2–17.
53. Bajcsy P, Cardone A, Chalfoun J, Halter M, Juba D, Kociolek M, et al. Survey statistics of automated segmentations applied to optical imaging of mammalian cells. BMC bioinformatics. 2015;16(1):1.
54. Jaqaman K, Loerke D, Mettlen M, Kuwata H, Grinstein S, Schmid SL, et al. Robust single-particle tracking in live-cell time-lapse sequences. Nature methods. 2008;5(8):695–702. doi: 10.1038/nmeth.123718641657
55. Taheri-Araghi S, Bradde S, Sauls JT, Hill NS, Levin PA, Paulsson J, et al. Cell-size control and homeostasis in bacteria. Current Biology. 2015;25(3):385–91. doi: 10.1016/j.cub.2014.12.00925544609
56. Wang P, Robert L, Pelletier J, Dang WL, Taddei F, Wright A, et al. Robust growth of Escherichia coli. Current biology. 2010;20(12):1099–103. doi: 10.1016/j.cub.2010.04.04520537537
57. Campos M, Surovtsev IV, Kato S, Paintdakhi A, Beltran B, Ebmeier SE, et al. A constant size extension drives bacterial cell size homeostasis. Cell. 2014;159(6):1433–46. doi: 10.1016/j.cell.2014.11.02225480302
58. Liu J, Prindle A, Humphries J, Gabalda-Sagarra M, Asally M, Dong-yeon DL, et al. Metabolic co-dependence gives rise to collective oscillations within biofilms. Nature. 2015;523(7562):550–4. doi: 10.1038/nature1466026200335
59. Nghe P, Boulineau S, Gude S, Recouvreux P, van Zon JS, Tans SJ, Microfabricated polyacrylamide devices for the controlled culture of growing cells and developing organisms. PloS one. 2013;8(9):e75537. doi: 10.1371/journal.pone.007553724086559
60. Boulineau S, Tostevin F, Kiviet DJ, ten Wolde PR, Nghe P, Tans SJ, Single-cell dynamics reveals sustained growth during diauxic shifts. PLoS One. 2013;8(4):e61686. doi: 10.1371/journal.pone.006168623637881
61. Kiviet DJ, Nghe P, Walker N, Boulineau S, Sunderlikova V, Tans SJ, Stochasticity of metabolism and growth at the single-cell level. Nature. 2014;514(7522):376–9. doi: 10.1038/nature1358225186725
62. Chen L- C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:14127062. 2014.
63. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al., editors. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2015.
64. Helaine S, Cheverton AM, Watson KG, Faure LM, Matthews SA, Holden DW, Internalization of Salmonella by macrophages induces formation of nonreplicating persisters. Science. 2014;343(6167):204–8. doi: 10.1126/science.124470524408438
65. Raj A, Peskin CS, Tranchina D, Vargas DY, Tyagi S, Stochastic mRNA Synthesis in Mammalian Cells. PLoS Biol. 2006;4(10):e309. doi: 10.1371/journal.pbio.004030917048983
66. Nielsen HJ, Youngren B, Hansen FG, Austin S, Dynamics of Escherichia coli chromosome segregation during multifork replication. Journal of bacteriology. 2007;189(23):8660–6. doi: 10.1128/JB.01212-0717905986