High-precision automated reconstruction of neurons with flood-filling networks

Nature Methods

Volumn 15, Issue 8, 2018, Pages 605-610

  Januszewski, Michał ^a   Kornfeld, Jörgen ^b   Li, Peter H ^c   Pope, Art ^c   Blakely, Tim ^c   Lindsey, Larry ^c   Maitin Shepard, Jeremy ^c   Tyka, Mike ^c   Denk, Winfried ^b   Jain, Viren ^c  

^a GOOGLE   (Switzerland)

^b MAX PLANCK INSTITUTE OF NEUROBIOLOGY   (Germany)

^c GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ANIMAL TISSUE; ARTICLE; AXON; BRAIN; CELL BODY; CLASSIFICATION; CLASSIFIER; DENDRITE; DENDRITIC SPINE; DROSOPHILA; ELECTRON MICROSCOPY; FLOOD FILLING NETWORK; HUMAN; HYSTERESIS; MACHINE LEARNING; NERVE CELL; NEURITE; NEUROPIL; NONHUMAN; OPTIC LOBE; PRIORITY JOURNAL; TAENIOPYGIA GUTTATA; ALGORITHM; ANATOMY AND HISTOLOGY; ANIMAL; FINCH; IMAGE PROCESSING; MALE; MOUSE; NERVE CELL NETWORK; PROCEDURES; THREE DIMENSIONAL IMAGING; TRANSMISSION ELECTRON MICROSCOPY; ULTRASTRUCTURE;

ALGORITHMS; ANIMALS; BRAIN; DROSOPHILA; FINCHES; IMAGE PROCESSING, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MACHINE LEARNING; MALE; MICE; MICROSCOPY, ELECTRON, TRANSMISSION; NERVE NET; NEURITES; NEURONS;

EID: 85049970432     PISSN: 15487091     EISSN: 15487105     Source Type: Journal    
DOI: 10.1038/s41592-018-0049-4     Document Type: Article

References (49)

1. Macagno, E. R., Levinthal, C. & Sobel, I. Three-dimensional computer reconstruction of neurons and neuronal assemblies. Annu. Rev. Biophys. Bioeng. 8, 323–351 (1979)
2. Harris, K. M., Jensen, F. E. & Tsao, B. Three-dimensional structure of dendritic spines and synapses in rat hippocampus (CA1) at postnatal day 15 and adult ages: implications for the maturation of synaptic physiology and long-term potentiation. J. Neurosci. 12, 2685–2705 (1992)
3. Ventura, R. & Harris, K. M. Three-dimensional relationships between hippocampal synapses and astrocytes. J. Neurosci. 19, 6897–6906 (1999)
4. Fiala, J. C. Reconstruct: a free editor for serial section microscopy. J. Microsc. 218, 52–61 (2005)
5. Briggman, K. L. & Bock, D. D. Volume electron microscopy for neuronal circuit reconstruction. Curr. Opin. Neurobiol. 22, 154–161 (2012)
6. Jain, V., Seung, H. S. & Turaga, S. C. Machines that learn to segment images: a crucial technology for connectomics. Curr. Opin. Neurobiol. 20, 653–666 (2010)
7. Kim, J. S. et al. Space-time wiring specificity supports direction selectivity in the retina. Nature 509, 331–336 (2014)
8. Takemura, S.-Y. et al. Synaptic circuits and their variations within different columns in the visual system of Drosophila. Proc. Natl. Acad. Sci. USA 112, 13711–13716 (2015)
9. Helmstaedter, M., Briggman, K. L. & Denk, W. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nat. Neurosci. 14, 1081–1088 (2011)
10. Helmstaedter, M. et al. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature 500, 168–174 (2013)
11. Cardona, A. TrakEM2: an ImageJ-based program for morphological data mining and 3D modeling. in Proc. ImageJ User and Developer Conference 18–19 (Centre de Recherche Public Henri Tudor: Luxembourg, 2006)
12. Berning, M., Boergens, K. M. & Helmstaedter, M. SegEM: efficient image analysis for high-resolution connectomics. Neuron 87, 1193–1206 (2015)
13. Plaza, S. M. Focused proofreading to reconstruct neural connectomes from EM images at scale. in Deep Learning and Data Labeling for Medical Applications (eds. Carneiro, G. et al.) 249–258 (Springer, Cham, 2016)
14. Jain, V. et al. Supervised learning of image restoration with convolutional networks. in Proc. IEEE 11th International Conference on Computer Vision 636–643 (IEEE, New York, 2007)
15. Ciresan, D., Giusti, A., Gambardella, L. M. & Schmidhuber, J. Deep neural networks segment neuronal membranes in electron microscopy images. in Advances in Neural Information Processing Systems 25 (eds. Pereira, F. et al.) 2852–2860 (Neural Information Processing Systems Foundation, La Jolla, CA, 2012)
16. Turaga, S. C. et al. Convolutional networks can learn to generate affinity graphs for image segmentation. Neural Comput. 22, 511–538 (2010)
17. Funke, J. et al. A deep structured learning approach towards automating connectome reconstruction from 3D electron micrographs. arXiv Preprint at https://arxiv.org/abs/1709.02974 (2017)
18. Lee, K., Zung, J., Li, P., Jain, V. & Seung, H. S. Superhuman accuracy on the SNEMI3D Connectomics Challenge. arXiv Preprint at https://arxiv.org/abs/1706.00120 (2017)
19. Andres, B., Koethe, U., Helmstaedter, M., Denk, W. & Hamprecht, F. A. Segmentation of SBFSEM volume data of neural tissue by hierarchical classification. in Pattern Recognition: Proceedings of the 30th DAGM Symposium (ed. Rigoll, G.)142–152 (Springer, Berlin, 2008)
20. Kaynig, V. et al. Large-scale automatic reconstruction of neuronal processes from electron microscopy images. Med. Image Anal. 22, 77–88 (2015)
21. Knowles-Barley, S. et al. RhoanaNet pipeline: dense automatic neural annotation. arXiv Preprint at https://arxiv.org/abs/1611.06973 (2016)
22. Beier, T. et al. Multicut brings automated neurite segmentation closer to human performance. Nat. Methods 14, 101–102 (2017)
23. Turaga, S. C., Briggman, K. L., Helmstaedter, M., Denk, W. & Seung, H. S. Maximin affinity learning of image segmentation. in Advances in Neural Information Processing Systems 22 (eds. Bengio, Y. et al.) 1865–1873 (Neural Information Processing Systems, La Jolla, CA, 2009)
24. Jain, V. et al. Boundary learning by optimization with topological constraints. in Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2488–2495 (IEEE, New York, 2010)
25. Denk, W. & Horstmann, H. Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2, e329 (2004)
26. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, Cambridge, MA, 2016)
27. Ronneberger, O., Fischer, P. & Brox, T. U-Net: Convolutional networks for biomedical image segmentation. in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 (eds. Navab, N. et al.) 234–241 (Springer, Cham, 2015)
28. Nunez-Iglesias, J., Kennedy, R., Plaza, S. M., Chakraborty, A. & Katz, W. T. Graph-based active learning of agglomeration (GALA): a Python library to segment 2D and 3D neuroimages. Front. Neuroinform. 8, 34 (2014)
29. Maitin-Shepard, J., Jain, V., Januszewski, M., Li, P. & Abbeel, P. Combinatorial energy learning for image segmentation. arXiv Preprint at https://arxiv.org/abs/1506.04304 (2015)
30. Saalfeld, S., Fetter, R., Cardona, A. & Tomancak, P. Elastic volume reconstruction from series of ultra-thin microscopy sections. Nat. Methods 9, 717–720 (2012)
31. Dorkenwald, S. et al. Automated synaptic connectivity inference for volume electron microscopy. Nat. Methods 14, 435–442 (2017)
32. Pallotto, M., Watkins, P. V., Fubara, B., Singer, J. H. & Briggman, K. L. Extracellular space preservation aids the connectomic analysis of neural circuits. eLife 4, e08206 (2015)
33. Zlateski, A. & Seung, H. S. Image segmentation by size-dependent single linkage clustering of a watershed basin graph. arXiv Preprint at https://arxiv.org/abs/1505.00249 (2015)
34. Kasthuri, N. et al. Saturated reconstruction of a volume of neocortex. Cell 162, 648–661 (2015)
35. Martin, D. R., Fowlkes, C. C. & Malik, J. Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Mach. Intell. 26, 530–549 (2004)
36. Funke, J., Andres, B., Hamprecht, F. A., Cardona, A. & Cook, M. Efficient automatic 3D-reconstruction of branching neurons from EM data. in Proc. IEEE Conference on Computer Vision and Pattern Recognition 1004–1011 (IEEE, New York, 2012)
37. Wolf, S., Schott, L., Kothe, U. & Hamprecht, F. Learned watershed: end-to-end learning of seeded segmentation. in Proc. IEEE International Conference on Computer Vision (ICCV) 2030–2038 (IEEE, New York, 2017)
38. Bai, M. & Urtasun, R. Deep watershed transform for instance segmentation. in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2858–2866 (IEEE, New York, 2017)
39. Meirovitch, Y. et al. A multi-pass approach to large-scale connectomics. arXiv Preprint at https://arxiv.org/abs/1612.02120 (2016)
40. Romera-Paredes, B. & Torr, P. H. S. Recurrent instance segmentation. in Computer Vision—ECCV 2016 (eds. Leibe, B. et al.) 312–329 (Springer, Cham, 2016)
41. Pinheiro, P. O, Lin, T.-Y., Collobert, R., & Dollár, P. Learning to refine object segments. in Computer Vision–ECCV 2016 (eds. Leibe, B. et al.) 75–91 (Springer, Cham, 2016)
42. Ren, M. & Zemel, R. S. End-to-end instance segmentation with recurrent attention. in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 293–301 (IEEE, New York, 2017)
43. Abadi, M. et al. TensorFlow: large-scale machine learning on heterogeneous distributed systems. arXiv Preprint at https://arxiv.org/abs/1603.04467 (2016)
44. LeCun, Y. et al. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989)
45. LeCun, Y. A, Bottou, L, Orr, G. B., & Müller, K.-R. Efficient BackProp. in Neural Networks: Tricks of the Trade (eds. Montavon, G., Orr, G. & Müller, K.-R.) 9–48 (Springer, Berlin, 2012)
46. Tschopp, F. Efficient convolutional neural networks for pixelwise classification on heterogeneous hardware systems. arXiv Preprint at https://arxiv.org/abs/1509.03371 (2015)
47. He, K., Zhang, X., Ren, S. & Sun, J. Identity mappings in deep residual networks. arXiv Preprint at https://arxiv.org/abs/1603.05027 (2016)
48. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. arXiv Preprint at https://arxiv.org/abs/1512.03385 (2015)
49. Kornfeld, J. et al. EM connectomics reveals axonal target variation in a sequence-generating network. eLife 6, e24364 (2017)