A systematic performance evaluation of clustering methods for single-cell RNA-seq data

F1000Research

Volumn 7, Issue , 2018, Pages

  Duò, Angelo ^a,b   Robinson, Mark D ^a,b   Soneson, Charlotte ^a,b  

^a UNIVERSITY OF ZURICH   (Switzerland)

^b SWISS INSTITUTE OF BIOINFORMATICS   (Switzerland)

Author keywords

Benchmarking; Clustering; Clustering methods; RNA seq; Single cell RNA seq

Indexed keywords

ARTICLE; BENCHMARKING; CLUSTERING ALGORITHM; FEATURE SELECTION; INTERMETHOD COMPARISON; SIMULATION; SINGLE CELL RNA SEQ;

EID: 85089317715     PISSN: 20461402     EISSN: 1759796X     Source Type: Journal    
DOI: 10.12688/f1000research.15666.2     Document Type: Article

References (54)

1. Tang F Barbacioru C Wang Y : mRNA-Seq whole-transcriptome analysis of a single cell. Nat Methods. 2009;6(5):377-382. 19349980 10.1038/nmeth.1315
2. Picelli S Björklund ÅK Faridani OR : Smart-seq2 for sensitive full-length transcriptome profiling in single cells. Nat Methods. 2013;10(11):1096-1098. 24056875 10.1038/nmeth.2639
3. Klein AM Mazutis L Akartuna I : Droplet barcoding for single-cell transcriptomics applied to embryonic stem cells. Cell. 2015;161(5):1187-1201. 26000487 10.1016/j.cell.2015.04.044 4441768
4. Macosko EZ Basu A Satija R : Highly Parallel Genome-wide Expression Profiling of Individual Cells Using Nanoliter Droplets. Cell. 2015;161(5):1202-1214. 26000488 10.1016/j.cell.2015.05.002 4481139
5. Zheng GX Terry JM Belgrader P : Massively parallel digital transcriptional profiling of single cells. Nat Commun. 2017;8: 14049. 28091601 10.1038/ncomms14049 5241818
6. Svensson V Natarajan KN Ly LH : Power analysis of single-cell RNA-sequencing experiments. Nat Methods. 2017;14(4):381-387. 28263961 10.1038/nmeth.4220 5376499
7. Svensson V Vento-Tormo R Teichmann SA : Exponential scaling of single-cell RNA-seq in the past decade. Nat Protoc. 2018;13(4):599-604. 29494575 10.1038/nprot.2017.149
8. Ziegenhain C Vieth B Parekh S : Quantitative single-cell transcriptomics. Brief Funct Genomics. 2018;17(4):220-232. 29579145 10.1093/bfgp/ely009 6063296
9. Grün D Kester L van Oudenaarden A : Validation of noise models for single-cell transcriptomics. Nat Methods. 2014;11(6):637-640. 24747814 10.1038/nmeth.2930
10. Bacher R Kendziorski C : Design and computational analysis of single-cell RNA-sequencing experiments. Genome Biol. 2016;17(1):63. 27052890 10.1186/s13059-016-0927-y 4823857
11. Tung PY Blischak JD Hsiao CJ : Batch effects and the effective design of single-cell gene expression studies. Sci Rep. 2017;7: 39921. 28045081 10.1038/srep39921 5206706
12. Hicks SC Townes FW Teng M : Missing data and technical variability in single-cell RNA-sequencing experiments. Biostatistics. 2017;kxx053. 29121214 10.1093/biostatistics/kxx053
13. Zappia L Phipson B Oshlack A : Exploring the single-cell RNA-seq analysis landscape with the scRNA-tools database. PLoS Comput Biol. 2018;14(6):e1006245. 29939984 10.1371/journal.pcbi.1006245 6034903
14. Aghaeepour N Finak G , FlowCAP Consortium, et al.: Critical assessment of automated flow cytometry data analysis techniques. Nat Methods. 2013;10(3):228-238. 23396282 10.1038/nmeth.2365 3906045
15. Weber LM Robinson MD : Comparison of clustering methods for high-dimensional single-cell flow and mass cytometry data. Cytometry A. 2016;89(12):1084-1096. 27992111 10.1002/cyto.a.23030
16. Menon V : Clustering single cells: a review of approaches on high-and low-depth single-cell RNA-seq data. Brief Funct Genomics. 2018;17(4):240-245. 29236955 10.1093/bfgp/elx044 6063268
17. Satija R Farrell JA Gennert D : Spatial reconstruction of single-cell gene expression data. Nat Biotechnol. 2015;33(5):495-502. 25867923 10.1038/nbt.3192 4430369
18. Langfelder P Horvath S : WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics. 2008;9:559. 19114008 10.1186/1471-2105-9-559 2631488
19. Zeisel A Muñoz-Manchado AB Codeluppi S : Brain structure. Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq. Science. 2015;347(6226):1138-1142. 25700174 10.1126/science.aaa1934
20. Freytag S Tian L Lönnstedt I : Comparison of clustering tools in R for medium-sized 10x Genomics single-cell RNA-sequencing data [version 1; referees: 1 approved, 2 approved with reservations]. F1000Res. 2018;7:1297. 10.12688/f1000research.15809.1
21. Andrews TS Hemberg M : Identifying cell populations with scRNASeq. Mol Aspects Med. 2018;59:114-122. 28712804 10.1016/j.mam.2017.07.002
22. Soneson C Robinson MD : Bias, robustness and scalability in single-cell differential expression analysis. Nat Methods. 2018;15(4):255-261. 29481549 10.1038/nmeth.4612
23. Kumar RM Cahan P Shalek AK : Deconstructing transcriptional heterogeneity in pluripotent stem cells. Nature. 2014;516(7529):56-61. 25471879 10.1038/nature13920 4256722
24. Koh PW Sinha R Barkal AA : An atlas of transcriptional, chromatin accessibility, and surface marker changes in human mesoderm development. Sci Data. 2016;3: 160109. 27996962 10.1038/sdata.2016.109 5170597
25. Trapnell C Cacchiarelli D Grimsby J : The dynamics and regulators of cell fate decisions are revealed by pseudotemporal ordering of single cells. Nat Biotechnol. 2014;32(4):381-386. 24658644 10.1038/nbt.2859 4122333
26. Ramos M Schiffer L Re A : Software for the integration of Multi-Omics experiments in Bioconductor. bioRxiv. 2017. 10.1101/144774
27. Bray NL Pimentel H Melsted P : Near-optimal probabilistic RNA-seq quantification. Nat Biotechnol. 2016;34(5):525-527. 27043002 10.1038/nbt.3519
28. Ntranos V Kamath GM Zhang JM : Fast and accurate single-cell RNA-Seq analysis by clustering of transcript-compatibility counts. Genome Biol. 2016;17(1):112. 27230763 10.1186/s13059-016-0970-8 4881296
29. Zappia L Phipson B Oshlack A : Splatter: simulation of single-cell RNA sequencing data. Genome Biol. 2017;18(1):174. 28899397 10.1186/s13059-017-1305-0 5596896
30. Soneson C Robinson MD : Towards unified quality verification of synthetic count data with countsimQC. Bioinformatics. 2018;34(4):691-692. 29028961 10.1093/bioinformatics/btx631 5860609
31. McCarthy DJ Campbell KR Lun AT : Scater: pre-processing, quality control, normalization and visualization of single-cell RNA-seq data in R. Bioinformatics. 2017;33(8):1179-1186. 28088763 10.1093/bioinformatics/btw777 5408845
32. Lun AT Bach K Marioni JC : Pooling across cells to normalize single-cell RNA sequencing data with many zero counts. Genome Biol. 2016;17(1):75. 27122128 10.1186/s13059-016-0947-7 4848819
33. Pearson K : On lines and planes of closest fit to systems of points in space. Philos Mag. 1901;2:559-572. 10.1080/14786440109462720
34. van der Maaten L Hinton G : Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579-2605. Reference Source
35. Andrews TS Hemberg M : Dropout-based feature selection for scRNASeq. bioRxiv. 2018. 10.1101/065094
36. Senabouth A Lukowski S Alquicira J : ascend: R package for analysis of single cell RNA-seq data. bioRxiv. 2017. 10.1101/207704
37. Lin P Troup M Ho JW : CIDR: Ultrafast and accurate clustering through imputation for single-cell RNA-seq data. Genome Biol. 2017;18(1):59. 28351406 10.1186/s13059-017-1188-0 5371246
38. Van Gassen S Callebaut B Van Helden MJ : Flowsom: Using self-organizing maps for visualization and interpretation of cytometry data. Cytometry A. 2015;87(7):636-645. 25573116 10.1002/cyto.a.22625
39. Qiu X Mao Q Tang Y : Reversed graph embedding resolves complex single-cell trajectories. Nat Methods. 2017;14(10):979-982. 28825705 10.1038/nmeth.4402 5764547
40. Ward JH Jr : Hierarchical grouping to optimize an objective function. J Am Stat Assoc. 1963;58(301):236-244. 10.1080/01621459.1963.10500845
41. Hartigan JA Wong MA : Algorithm as-136: A k-means clustering algorithm. J R Stat Soc Ser C Appl Stat. 1979;28(1):100-108. 10.2307/2346830
42. Žurauskiene J Yau C : pcaReduce: hierarchical clustering of single cell transcriptional profiles. BMC Bioinformatics. 2016;17(1):140. 27005807 10.1186/s12859-016-0984-y 4802652
43. Grün D Muraro MJ Boisset JC : De Novo Prediction of Stem Cell Identity using Single-Cell Transcriptome Data. Cell Stem Cell. 2016;19(2):266-277. 27345837 10.1016/j.stem.2016.05.010 4985539
44. Van Der Maaten L : Accelerating t-SNE using tree-based algorithms. J Mach Learn Res. 2014;15:1-21. Reference Source
45. Yang Y Huh R Culpepper HW : SAFE-clustering: Single-cell aggregated (from ensemble) clustering for single-cell RNA-seq data. bioRxiv. 2017. 10.1101/215723
46. Kiselev VY Kirschner K Schaub MT : SC3: consensus clustering of single-cell RNA-seq data. Nat Methods. 2017;14(5):483-486. 28346451 10.1038/nmeth.4236 5410170
47. Cortes C Vapnik V : Support-vector networks. Mach Learn. 1995;20(3):273-297. 10.1023/A:1022627411411
48. Ji Z Ji H : TSCAN: Pseudo-time reconstruction and evaluation in single-cell RNA-seq analysis. Nucleic Acids Res. 2016;44(13):e117. 27179027 10.1093/nar/gkw430 4994863
49. Hubert L Arabie P : Comparing partitions. J Classif. 1985;2(1):193-218. 10.1007/BF01908075
50. Shannon CE : A mathematical theory of communication. Bell Syst Tech J. 1948;27(3):379-423. 10.1002/j.1538-7305.1948.tb01338.x
51. Hornik K : A CLUE for CLUster Ensembles. J Stat Softw. 2005;14(12):1-25. 10.18637/jss.v014.i12
52. Kruskal WH Wallis WA : Use of ranks in one-criterion variance analysis. J Am Stat Assoc. 1952;47(260):583-621. 10.2307/2280779
53. Von Luxburg U : Clustering stability: an overview. Foundations and Trends in Machine Learning. 2010;2(3):235-274. 10.1561/2200000008
54. Severson DT Owen RP White MJ : BEARscc determines robustness of single-cell clusters using simulated technical replicates. Nat Commun. 2018;9(1): 1187. 29567991 10.1038/s41467-018-03608-y 5864873