Design and evaluation of interactive proofreading tools for connectomics

IEEE Transactions on Visualization and Computer Graphics

Volumn 20, Issue 12, 2014, Pages 2466-2475

  Haehn, Daniel ^a   Knowles Barley, Seymour ^b   Roberts, Mike ^c   Beyer, Johanna ^a   Kasthuri, Narayanan ^b   Lichtman, Jeff W ^b   Pfister, Hanspeter ^a  

^a HARVARD UNIVERSITY   (United States)

^b HARVARD UNIVERSITY   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

Connectomics; Proofreading; Quantitative Evaluation; Segmentation

Indexed keywords

CONNECTOMICS; DESIGN AND EVALUATIONS; PROOFREADING; QUANTITATIVE EVALUATION;

IMAGE SEGMENTATION;

ADULT; ALGORITHM; ANIMAL; COMPUTER GRAPHICS; COMPUTER PROGRAM; CONNECTOME; FEMALE; HUMAN; INTERNET; MALE; MOUSE; PROCEDURES; THREE DIMENSIONAL IMAGING; YOUNG ADULT;

ADULT; ALGORITHMS; ANIMALS; COMPUTER GRAPHICS; CONNECTOME; FEMALE; HUMANS; IMAGING, THREE-DIMENSIONAL; INTERNET; MALE; MICE; SOFTWARE; YOUNG ADULT;

EID: 84909586776     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2014.2346371     Document Type: Article

References (49)

1. Rambo3d. http://openconnectome.github.io/Rambo3D/, 2012. Accessed on 31/03/2014.
2. IEEE ISBI challenge: SNEMI3D - 3D segmentation of neurites in EM images. http://brainiac2.mit.edu/SNEMI3D, 2013. Accessed on 31/03/2014.
3. Eyewire. http://eyewire.org/, 2014. Accessed on 31/03/2014.
4. OpenSeaDragon. http://openseadragon.github.io/, 2014. Accessed on 31/03/2014.
5. A. Akselrod-Ballin, D. Bock, R. Reid, and S. Warfield. Improved registration for large electron microscopy images. In IEEE Int. Symp. on Biomedical Imaging (ISBI '09), pages 434-437, 2009.
6. J. Anderson, S. Mohammed, B. Grimm, B. Jones, P. Koshevoy, T. Tasdizen, R. Whitaker, and R. Marc. The Viking Viewer for connectomics: Scalable multi-user annotation and summarization of large volume data sets. Journal of Microscopy, 241(1):13-28, 2011.
7. J. Beyer, A. Al-Awami, N. Kasthuri, J. W. Lichtman, H. Pfister, and M. Hadwiger. ConnectomeExplorer: Query-guided visual analysis of large volumetric neuroscience data. IEEE Trans. on Vis. and Computer Graphics (Proc. IEEE SciVis 2013), 19(12):2868-2877, 2013.
8. J. Beyer, M. Hadwiger, A. Al-Awami, W.-K. Jeong, N. Kasthuri, J. W. Lichtman, and H. Pfister. Exploring the connectome: Petascale volume visualization of microscopy data streams. IEEE Computer Graphics and Applications, 33(4):50-61, 2013.
9. B. Cabral, N. Cam, and J. Foran. Accelerated volume rendering and tomographic reconstruction using texture mapping hardware. In IEEE Symposium on Volume Visualization, pages 91-98, 1994.
10. A. Cardona, S. Saalfeld, J. Schindelin, I. Arganda-Carreras, S. Preibisch, M. Longair, P. Tomancak, V. Hartenstein, and R. J. Douglas. TrakEM2 software for neural circuit reconstruction. PLoS ONE, 7(6):e38011+, 2012.
11. L. Faulkner. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3):379-383, 2003.
12. J. C. Fiala. Reconstruct: A free editor for serial section microscopy. Journal of Microscopy, 218(1):52-61, 2005.
13. S. Gerhard, A. Daducci, A. Lemkaddem, R. Meuli, J. Thiran, and P. Hagmann. The connectome viewer toolkit: An open source framework to manage, analyze, and visualize connectomes. Frontiers in Neuroinformatics, 5, 2011.
14. D. Ginsburg, S. Gerhard, J. E. C. Calle, and R. Pienaar. Realtime visualization of the connectome in the browser using WebGL. Frontiers in Neuroinformatics, (95), 2011.
15. R. J. Giuly, K.-Y. Kim, and M. H. Ellisman. DP2: Distributed 3D image segmentation using micro-labor workforce. Bioinformatics, 29(10):1359-1360, 2013.
16. M. Hadwiger, J. Beyer, W.-K. Jeong, and H. Pfister. Interactive volume exploration of petascale microscopy data streams using a visualization-driven virtual memory approach. IEEE Trans. on Visualization and Computer Graphics (Proceedings IEEE SciVis'12), 18(12):2285-2294, 2012.
17. D. Haehn. Slice:drop: Collaborative medical imaging in the browser. In ACM SIGGRAPH 2013 Comp. Anim. Festival, pages 1-1, 2013.
18. D. Haehn, N. Rannou, B. Ahtam, E. Grant, and R. Pienaar. Neuroimaging in the browser using the X Toolkit. Frontiers in Neuroinformatics, 2012.
19. S. G. Hart and L. E. Staveland. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload, volume 52 of Advances in Psychology, pages 139-183. 1988.
20. M. Helmstädter, K. L. Briggman, and W. Denk. High-accuracy neurite reconstruction for high-throughput neuroanatomy. Nature Neuroscience, 14(8):1081-1088, 2011.
21. W. Hwang and G. Salvendy. Number of people required for usability evaluation: The 10+-2 rule. Commun. ACM, 53(5):130-133, May 2010.
22. V. Jain, B. Bollmann, M. Richardson, D. Berger, M. Helmstädter, K. Briggman, W. Denk, J. Bowden, J. Mendenhall, W. Abraham, K. Harris, N. Kasthuri, K. Hayworth, R. Schalek, J. Tapia, J. Lichtman, and S. Seung. Boundary learning by optimization with topological constraints. In Proceedings of IEEE CVPR 2010, pages 2488-2495, 2010.
23. Janelia Farm. Raveler. https://openwiki.janelia.org/wiki/display/flyem/Raveler, 2014. Accessed on 31/03/2014.
24. A. Janke, H. Waxenegger, J. Ullmann, and G. Galloway. Tissuestack: A new way to view your imaging data. Frontiers in Neuroinformatics, (3).
25. W. Jeong, J. Schneider, A. Hansen, M. Lee, S. G. Turney, B. E. Faulkner-Jones, J. L. Hecht, R. Najarian, E. Yee, J. W. Lichtman, and H. Pfister. A collaborative digital pathology system for multi-touch mobile and desktop computing platforms. Comp. Graphics Forum, 32(6):227-242, 2013.
26. W.-K. Jeong, J. Beyer, M. Hadwiger, R. Blue, C. Law, A. Vázquez-Reina, C. Reid, J. W. Lichtman, and H. Pfister. SSECRETT and NeuroTrace: Interactive visualization and analysis tools for large-scale neuroscience datasets. IEEE Computer Graphics and Applications, 30(3):58-70, 2010.
27. E. Jurrus, S. Watanabe, R. J. Giuly, A. R. Paiva, M. H. Ellisman, E. M. Jorgensen, and T. Tasdizen. Semi-automated neuron boundary detection and nonbranching process segmentation in electron microscopy images. Neuroinformatics, 11(1):5-29, 2013.
28. V. Kaynig, A. Vázquez-Reina, S. Knowles-Barley, M. Roberts, T. Jones, N. Kasthuri, E. Miller, J. W. Lichtman, and H. Pfister. Large-scale automatic reconstruction of neuronal processes from electron microscopy images. In arXiv: 1303.7186 [q-bio.NC], 2013.
29. Khronos Group. WebGL specification. http://www.khronos.org/registry/webgl/specs, 2014. Accessed on 31/03/2014.
30. S. Knowles-Barley, M. Roberts, N. Kasthuri, D. Lee, H. Pfister, and J.W. Lichtman. Mojo 2.0: Connectome annotation tool. Frontiers in Neuroinformatics, (60), 2013.
31. A. Kuß, S. Prohaska, B. Meyer, J. Rybak, and H.-C. Hege. Ontology-based visualization of hierarchical neuroanatomical structures. In Proceedings of Visual Computing for Biomedicine, pages 177-184, 2008.
32. J. W. Lichtman and W. Denk. The big and the small: Challenges of imaging the brain's circuits. Science, 334(6056):618-623, 2011.
33. J. W. Lichtman, J. Livet, and J. R. Sanes. A technicolour approach to the connectome. Nature reviews. Neuroscience, 2008.
34. T. Liu, C. Jones, M. Seyedhosseini, and T. Tasdizen. A modular hierarchical approach to 3D electron microscopy image segmentation. Journal of Neuroscience Methods, 226(0):88-102, 2014.
35. D. S. Margulies, J. Böttger, A. Watanabe, and K. J. Gorgolewski. Visualizing the human connectome. NeuroImage, 80(0):445-461, 2013.
36. J. Nunez-Iglesias, R. Kennedy, T. Parag, J. Shi, and D. B. Chklovskii. Machine learning of hierarchical clustering to segment 2D and 3D images. PLoS ONE, 8(8):e71715+, 2013.
37. J. Nunez-Iglesias, R. Kennedy, S. M. Plaza, A. Chakraborty, and W. T. Katz. Graph-based active learning of agglomeration (GALA): A python library to segment 2D and 3D neuroimages. Frontiers in Neuroinformatics, 8(34), 2014.
38. H. Peng, F. Long, T. Zhao, and E. Myers. Proof-editing is the bottleneck of 3D neuron reconstruction: The problem and solutions. Neuroinformatics, 9(2-3):103-105, 2011.
39. H. Pfister, V. Kaynig, C. P. Botha, S. Bruckner, V. J. Dercksen, H.-C. Hege, and J. B. Roerdink. Visualization in connectomics. In arXiv:1206.1428v2 [cs.GR], 2012.
40. M. Roberts, W.-K. Jeong, A. Vázquez-Reina, M. Unger, H. Bischof, J.W. Lichtman, and H. Pfister. Neural process reconstruction from sparse user scribbles. In Proceedings of MICCAI 2011, pages 621-628, 2011.
41. S. Saalfeld, A. Cardona, V. Hartenstein, and P. Tomančák. CATMAID: collaborative annotation toolkit for massive amounts of image data. Bioinformatics, 25(15):1984-1986, 2009.
42. S. Saalfeld, A. Cardona, V. Hartenstein, and P. Tomančák. As-rigid-as-possible mosaicking and serial section registration of large ssTEM datasets. Bioinformatics, 26(12):i57-i63, 2010.
43. S. Seung. Reading the book of memory: Sparse sampling versus dense mapping of connectomes. Neuron, 62(1):17-29, 2009.
44. S. Seung. Connectome: How the Brain's Wiring Makes Us Who We Are. Houghton Mifflin Harcourt, Feb. 2012.
45. J. P. Shaffer. Multiple hypothesis testing. Annual Review of Psychology, 46(1):561-584, 1995.
46. R. Sicat, M. Hadwiger, and N. J. Mitra. Graph abstraction for simplified proofreading of slice-based volume segmentation. In EUROGRAPHICS Short Paper, 2013.
47. C. Sommer, C. Straehle, U. Kothe, and F. A. Hamprecht. Ilastik: Interactive learning and segmentation toolkit. In IEEE Int. Symp. on Biomedical Imaging: From Nano to Macro, pages 230-233, 2011.
48. A. Vázquez-Reina, M. Gelbart, D. Huang, J. Lichtman, E. Miller, and H. Pfister. Segmentation fusion for connectomics. In Proceedings of IEEE ICCV, pages 177-184, Nov 2011.
49. P. A. Yushkevich, J. Piven, H. Cody Hazlett, R. Gimpel Smith, S. Ho, J. C. Gee, and G. Gerig. User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability. Neuroimage, 31(3):1116-1128, 2006.