Deep learning is combined with massive-scale citizen science to improve large-scale image classification

Nature Biotechnology

Volumn 36, Issue 9, 2018, Pages 820-832

  Sullivan, Devin P ^a   Winsnes, Casper F ^a   Åkesson, Lovisa ^a   Hjelmare, Martin ^a   Wiking, Mikaela ^a   Schutten, Rutger ^a   Campbell, Linzi ^b   Leifsson, Hjalti ^b   Rhodes, Scott ^b   Nordgren, Andie ^b   Smith, Kevin ^a   Revaz, Bernard ^c   Finnbogason, Bergur ^b   Szantner, Attila ^c   Lundberg, Emma ^a,d,e  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^b CCP hf   (Iceland)

^c MMOS S rl   (Switzerland)

^d STANFORD UNIVERSITY   (United States)

^e CHAN ZUCKERBERG BIOHUB   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER GAMES; FLUORESCENCE MICROSCOPY; HUMAN COMPUTER INTERACTION; IMAGE CLASSIFICATION; IMAGE ENHANCEMENT; INTERACTIVE COMPUTER GRAPHICS; PROTEINS;

ANNOTATION TOOL; CITIZEN SCIENCE; FLUORESCENCE MICROSCOPY IMAGES; LARGE SCALE CLASSIFICATIONS; PATTERN RECOGNITION AND CLASSIFICATION; PROTEIN DISTRIBUTIONS; SUBCELLULAR LOCALIZATIONS; TRANSFER LEARNING;

DEEP LEARNING;

ARTICLE; CELLULAR DISTRIBUTION; FLUORESCENCE MICROSCOPY; HUMAN; HUMAN EXPERIMENT; TRANSFER OF LEARNING; VIDEO GAME; CELL FRACTIONATION; IMAGE PROCESSING; METABOLISM; PROCEDURES;

DEEP LEARNING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MICROSCOPY, FLUORESCENCE; SUBCELLULAR FRACTIONS;

EID: 85053076602     PISSN: 10870156     EISSN: 15461696     Source Type: Journal    
DOI: 10.1038/nbt.4225     Document Type: Article

References (39)

1. Bouwer, J. et al. Petabyte data management and automated data workflow in neuroscience: delivering data from the instruments to the researcher’s fingertips. Microsc. Microanal. 17, 276–277 (2011).
2. Ferrucci, D. et al. Building Watson: an overview of the DeepQA project. AI Magazine 31, 59–79 (2010).
3. Larrañaga, P. et al. Machine learning in bioinformatics. Brief. Bioinform. 7, 86–112 (2006).
4. Silver, D. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).
5. Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017).
6. Cohn, J.P. Citizen science: can volunteers do real research? Bioscience 58, 192–197 (2008).
7. Uhlen, M. et al. Towards a knowledge-based Human Protein Atlas. Nat. Biotechnol. 28, 1248–1250 (2010).
8. Thul, P.J. et al. A subcellular map of the human proteome. Science 356, eaai3321 (2017).
9. Boland, M.V. & Murphy, R.F. A neural network classifier capable of recognizing the patterns of all major subcellular structures in fluorescence microscope images of HeLa cells. Bioinformatics 17, 1213–1223 (2001).
10. Huang, K. & Murphy, R.F. Boosting accuracy of automated classification of fluorescence microscope images for location proteomics. BMC Bioinformatics 5, 78 (2004).
11. Newberg, J.Y. et al. Automated analysis of Human Protein Atlas immunofluorescence images. Proc. IEEE Int. Symp. Biomed. Imaging 5193229, 1023–1026 (2009).
12. Li, J., Newberg, J.Y., Uhlén, M., Lundberg, E. & Murphy, R.F. Automated analysis and reannotation of subcellular locations in confocal images from the Human Protein Atlas. PLoS One 7, e50514 (2012).
13. Li, J., Xiong, L., Schneider, J. & Murphy, R.F. Protein subcellular location pattern classification in cellular images using latent discriminative models. Bioinformatics 28, i32–i39 (2012).
14. Coelho, L.P. et al. Determining the subcellular location of new proteins from microscope images using local features. Bioinformatics 29, 2343–2349 (2013).
15. Chebira, A. et al. A multiresolution approach to automated classification of protein subcellular location images. BMC Bioinformatics 8, 210 (2007).
16. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
17. Pärnamaa, T. & Parts, L. Accurate classification of protein subcellular localization from high-throughput microscopy images using deep learning. G3 (Bethesda) 7, 1385–1392 (2017).
18. Kraus, O.Z., Ba, J.L. & Frey, B.J. Classifying and segmenting microscopy images with deep multiple instance learning. Bioinformatics 32, i52–i59 (2016).
19. Nathalie Japkowicz, S.S. The class imbalance problem: A systematic study. Intell. Data Anal. 6, 429–449 (2002).
20. Coelho, L.P., Peng, T. & Murphy, R.F. Quantifying the distribution of probes between subcellular locations using unsupervised pattern unmixing. Bioinformatics 26, i7–i12 (2010).
21. Zhao, T., Velliste, M., Boland, M.V. & Murphy, R.F. Object type recognition for automated analysis of protein subcellular location. IEEE Trans. Image Process. 14, 1351–1359 (2005).
22. Shen, Y.-Y.X.L.-X.Y.H.-B. Bioimage-based protein subcellular location prediction: a comprehensive review. Front. Comput. Sci. 12, 26–39 (2018).
23. Khatib, F. et al. Algorithm discovery by protein folding game players. Proc. Natl. Acad. Sci. USA 108, 18949–18953 (2011).
24. Khatib, F. et al. Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nat. Struct. Mol. Biol. 18, 1175–1177 (2011).
25. Chris, J. et al. Galaxy Zoo: ‘Hanny’s Voorwerp’, a quasar light echo? Mon. Not. R. Astron. Soc. 399, 129–140 (2009).
26. Clery, D. Galaxy evolution. Galaxy zoo volunteers share pain and glory of research. Science 333, 173–175 (2011).
27. Raddick, M.J. et al. Galaxy Zoo: exploring the motivations of citizen science volunteers. Astron. Educ. Rev. 9, 18 (2010).
28. Lee, J. et al. RNA design rules from a massive open laboratory. Proc. Natl. Acad. Sci. USA 111, 2122–2127 (2014).
29. Sørensen, J.J. et al. Exploring the quantum speed limit with computer games. Nature 532, 210–213 (2016).
30. 5 -clicks-per-hour. Preprint at https://www.biorxiv.org/content/early/2017/07/15/164087 (2017).
31. Danielle, N., Shapiro, J.C. & Mueller, P.A. Using mechanical turk to study clinical populations. Clin. Pyschol. Sci. 1, 213–220 (2013).
32. Cox, J. et al. How is success defined and measured in online citizen science? A case study of Zooniverse projects. Comput. Sci. Eng. 17, 28–41 (2015).
33. Feng, W., Brandt, D. & Shah, D. A long-term study of a popular MMORPG. Proceedings of the 6th ACM SIGCOMM Workshop on Network and System Support for Games 19–24 (2007).
34. Warfield, S.K., Zou, K.H. & Wells, W.M. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans. Med. Imaging 23, 903–921 (2004).
35. Snow, R., O’Connor, B., Jurafsky, D. & Ng, A. Cheap and fast, but is it good? Evaluating non-expert annotations for natural language tasks. Conference on Empirical Methods in Natural Language Processing 254–263 (2008).
36. Calise, S.J. et al. Glutamine deprivation initiates reversible assembly of mammalian rods and rings. Cell. Mol. Life Sci. 71, 2963–2973 (2014).
37. Carcamo, W.C. et al. Induction of cytoplasmic rods and rings structures by inhibition of the CTP and GTP synthetic pathway in mammalian cells. PLoS One 6, e29690 (2011).
38. Handfield, L.F., Chong, Y.T., Simmons, J., Andrews, B.J. & Moses, A.M. Unsupervised clustering of subcellular protein expression patterns in high-throughput microscopy images reveals protein complexes and functional relationships between proteins. PLOS Comput. Biol. 9, e1003085 (2013).
39. Hasanpour, S., Rouhani, M., Fayyaz, M. & Sabokrou, M. Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures. Preprint at https://arxiv.org/abs/1608.06037 (2016).