Best practices for evaluating single nucleotide variant calling methods for microbial genomics

Frontiers in Genetics

Volumn 6, Issue JUL, 2015, Pages

  Olson, Nathan D ^a   Lund, Steven P ^a   Colman, Rebecca E ^b   Foster, Jeffrey T ^c,e   Sahl, Jason W ^b,c   Schupp, James M ^b   Keim, Paul ^b,c   Morrow, Jayne B ^a   Salit, Marc L ^a,d   Zook, Justin M ^a  

^a NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^b TRANSLATIONAL GENOMICS RESEARCH INSTITUTE   (United States)

^c NORTHERN ARIZONA UNIVERSITY   (United States)

^d Stanford University   (United States)

^e UNIVERSITY OF NEW HAMPSHIRE   (United States)

Author keywords

Indel; Next generation sequencing; Performance metrics; Single nucleotide variants; Variant calling

Indexed keywords

DNA;

BACILLUS ANTHRACIS; CLOSTRIDIUM SPOROGENES; ESCHERICHIA COLI; GENE MAPPING; GENE SEQUENCE; GENETIC PROCEDURES; GENETIC VARIABILITY; GENOMICS; MICROBIAL GENOME; MUTATION; NEXT GENERATION SEQUENCING; NONHUMAN; PHYLOGENY; POLYMERASE CHAIN REACTION; PRACTICE GUIDELINE; PSEUDOMONAS AERUGINOSA; REVIEW; SALMONELLA ENTERICA SEROVAR TYPHIMURIUM; SINGLE NUCLEOTIDE POLYMORPHISM; SINGLE NUCLEOTIDE VARIANT CALLING; STAPHYLOCOCCUS AUREUS; VIBRIO CHOLERAE;

BACTERIA (MICROORGANISMS);

EID: 84940106715     PISSN: None     EISSN: 16648021     Source Type: Journal    
DOI: 10.3389/fgene.2015.00235     Document Type: Review

References (92)

1. Alkan, C., Coe, B. P., and Eichler, E. E. (2011). Genome structural variation discovery and genotyping. Nat. Rev. Genet. 12, 363-376. doi: 10.1038/nrg2958.
2. Altmann, A., Weber, P., Bader, D., Preuss, M., Binder, E. B., and Muller-Myhsok, B. (2012). A beginners guide to SNP calling from high-throughput DNA-sequencing data. Hum. Genet. 131, 1541-1554. doi: 10.1007/s00439-012-1213-z.
3. Altschul, S. F., Gish, W., Miller, W., Myers, E. W., and Lipman, D. J. (1990). Basic local alignment search tool. J. Mol. Biol. 215, 403-410. doi: 10.1016/S0022-2836(05)80360-2.
4. Angiuoli, S. V., and Salzberg, S. L. (2011). Mugsy: fast multiple alignment of closely related whole genomes. Bioinformatics 27, 334-342. doi: 10.1093/bioinformatics/btq665.
5. Angly, F. E., Willner, D., Rohwer, F., Hugenholtz, P., and Tyson, G. W. (2012). Grinder: a versatile amplicon and shotgun sequence simulator. Nucleic Acids Res. 40, e94. doi: 10.1093/nar/gks251.
6. Baker, M. (2012). De novo genome assembly: what every biologist should know. Nat. Methods 9, 333-337. doi: 10.1038/nmeth.1935.
7. Baldi, P., Brunak, S., Chauvin, Y., Andersen, C. A. F., and Nielsen, H. (2000). Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16, 412-424. doi: 10.1093/bioinformatics/16.5.412.
8. Balzer, S., Malde, K., Lanzen, A., Sharma, A., and Jonassen, I. (2010). Characteristics of 454 pyrosequencing data-enabling realistic simulation with flowsim. Bioinformatics 26, 420-425. doi: 10.1093/bioinformatics/btq365.
9. Bankevich, A., Nurk, S., Antipov, D., Gurevich, A. A., Dvorkin, M., Kulikov, A. S., et al. (2012). SPAdes: a new genome assembly algorithm and its applications to single-cell sequencing. J. Comput. Biol. 19, 455-477. doi: 10.1089/cmb.2012.0021.
10. Bertels, F., Silander, O. K., Pachkov, M., Rainey, P. B., and Van Nimwegen, E. (2014). Automated reconstruction of whole-genome phylogenies from short-sequence reads. Mol. Biol. Evol. 31, 1077-1088. doi: 10.1093/molbev/msu088.
11. Bian, J., Liu, C., Wang, H., Xing, J., Kachroo, P., and Zhou, X. (2013). SNVHMM: predicting single nucleotide variants from next generation sequencing. BMC Bioinformatics 14:225. doi: 10.1186/1471-2105-14-225.
12. Cantarel, B. L., Weaver, D., Mcneill, N., Zhang, J., Mackey, A. J., and Reese, J. (2014). BAYSIC: a Bayesian method for combining sets of genome variants with improved specificity and sensitivity. BMC Bioinformatics 15:104. doi: 10.1186/1471-2105-15-104.
13. Cantor, R. M., Lange, K., and Sinsheimer, J. S. (2010). Prioritizing GWAS results: a review of statistical methods and recommendations for their application. Am. J. Hum. Genet. 86, 6-22. doi: 10.1016/j.ajhg.2009.11.017.
14. Chaisson, M. J., and Pevzner, P. A. (2008). Short read fragment assembly of bacterial genomes. Genome Res. 18, 324-330. doi: 10.1101/gr.7088808.
15. Chen-Harris, H., Borucki, M. K., Torres, C., Slezak, T. R., and Allen, J. E. (2013). Ultra-deep mutant spectrum profiling: improving sequencing accuracy using overlapping read pairs. BMC Genomics 14:96. doi: 10.1186/1471-2164-14-96.
16. Chikhi, R., and Medvedev, P. (2014). Informed and automated k-mer size selection for genome assembly. Bioinformatics 30, 31-37. doi: 10.1093/bioinformatics/btt310.
17. Cline, J., Braman, J. C., and Hogrefe, H. H. (1996). PCR fidelity of Pfu DNA polymerase and other thermostable DNA polymerases. Nucleic Acids Res. 24, 3546-3551. doi: 10.1093/nar/24.18.3546.
18. Cock, P. J. A., Bonfield, J. K., Chevreux, B., and Li, H. (2015). SAM/BAM format v1.5 extensions for de novo assemblies. bioRxiv 1-3. doi: 10.1101/020024.
19. Colman, R. E., Schupp, J. M., Hicks, N. D., Smith, D. E., Buchhagen, J. L., Valafar, F., et al. (2015). Detection of low-level mixed-population drug resistance in Mycobacterium tuberculosis using high fidelity amplicon sequencing. PLoS ONE 10:e0126626. doi: 10.1371/journal.pone.0126626.
20. Cushing, A., Flaherty, P., Hopmans, E., Bell, J., and Ji, H. (2013). RVD: a command-line program for ultrasensitive rare single nucleotide variant detection using targeted next-generation DNA resequencing. BMC Res. Notes 6:206. doi: 10.1186/1756-0500-6-206.
21. Darling, A. C., Mau, B., Blattner, F. R., and Perna, N. T. (2004). Mauve: multiple alignment of conserved genomic sequence with rearrangements. Genome Res. 14, 1394-1403. doi: 10.1101/gr.2289704.
22. Del Fabbro, C., Scalabrin, S., Morgante, M., and Giorgi, F. M. (2013). An extensive evaluation of read trimming effects on Illumina NGS data analysis. PLoS ONE 8:e85024. doi: 10.1371/journal.pone.0085024.
23. Delcher, A. L., Phillippy, A., Carlton, J., and Salzberg, S. L. (2002). Fast algorithms for large-scale genome alignment and comparison. Nucleic Acids Res. 30, 2478-2483. doi: 10.1093/nar/30.11.2478.
24. DePristo, M. A., Banks, E., Poplin, R., Garimella, K. V., Maguire, J. R., Hartl, C., et al. (2011). A framework for variation discovery and genotyping using next-generation DNA sequencing data. Nat. Genet. 43, 491-498. doi: 10.1038/ng.806.
25. Dohm, J. C., Lottaz, C., Borodina, T., and Himmelbauer, H. (2008). Substantial biases in ultra-short read data sets from high-throughput DNA sequencing. Nucleic Acids Res. 36, e105. doi: 10.1093/nar/gkn425.
26. Ellison, S. L. R., and Williams, A. (2012). Eurachem/CITAC Guide: Quantifying Uncertainty in Analytical Measurement, 3rd Edn. ISBN 978-0-948926-30-3. Available at: http://www.eurachem.org.
27. Engle, M. L., and Burks, C. (1992). Artificially generated data sets for testing DNA sequence assembly algorithms. Genomics 16, 286-288. doi: 10.1006/geno.1993.1180.
28. Engle, M. L., and Burks, C. (1994). GenFrag 2.1: new features for more robust fragment assembly benchmarks. Comput. Appl. Biosci. 10, 567-568. doi: 10.1093/bioinformatics/10.5.567.
29. Ewing, B., and Green, P. (1998). Base-calling of automated sequencer traces using phred. II. Error probabilities. Genome Res. 8, 186-194. doi: 10.1101/gr.8.3.175.
30. Fonseca, N. A., Rung, J., Brazma, A., and Marioni, J. C. (2012). Tools for mapping high-throughput sequencing data. Bioinformatics 28, 3169-3177. doi: 10.1093/bioinformatics/bts605.
31. Foster, J. T., Beckstrom-Sternberg, S. M., Pearson, T., Beckstrom-Sternberg, J. S., Chain, P. S. G., Roberto, F. F., et al. (2009). Whole genome-based phylogeny and divergence of the genus Brucella. J. Bacteriol. 191, 2864-2870. doi: 10.1128/JB.01581-08.
32. Frampton, M., and Houlston, R. (2012). Generation of artificial FASTQ files to evaluate the performance of next-generation sequencing pipelines. PLoS ONE 7:e49110. doi: 10.1371/journal.pone.0049110.
33. Gallagher, S. R., and Desjardins, P. R. (2008). Quantitation of DNA and RNA with absorption and fluorescence spectroscopy. Curr. Protoc. Protein Sci. (Suppl. 52), A.4K.1-A.4K21. doi: 10.1002/0471140864.psa04ks52.
34. Gardner, S. N., and Hall, B. G. (2013). When whole-genome alignments just won't work: kSNP v2 software for alignment-free SNP discovery and phylogenetics of hundreds of microbial genomes. PLoS ONE 8:e81760. doi: 10.1371/journal.pone.0081760.
35. Goya, R., Sun, M. G. F., Morin, R. D., Leung, G., Ha, G., Wiegand, K. C., et al. (2010). SNVMix: predicting single nucleotide variants from next-generation sequencing of tumors. Bioinformatics 26, 730-736. doi: 10.1093/bioinformatics/btq040.
36. Grad, Y. H., Lipsitch, M., Feldgarden, M., Arachchi, H. M., Cerqueira, G. C., Fitzgerald, M., et al. (2012). Genomic epidemiology of the Escherichia coli O104:H4 outbreaks in Europe, 2011. Proc. Natl. Acad. Sci. U.S.A. 109, 3065-3070. doi: 10.1073/pnas.1121491109.
37. Harris, S. R., Torok, M. E., Cartwright, E. J., Quail, M. A., Peacock, S. J., and Parkhill, J. (2013). Read and assembly metrics inconsequential for clinical utility of whole-genome sequencing in mapping outbreaks. Nat. Biotechnol. 31, 592-594. doi: 10.1038/nbt.2616.
38. Hatem, A., Bozdag, D., Toland, A. E., and Çatalyürek, Ü. V. (2013). Benchmarking short sequence mapping tools. BMC Bioinformatics 14:184. doi: 10.1186/1471-2105-14-184.
39. Hazelhurst, S., and Bergheim, A. (2003). SeqGen: a Tool for Creating Benchmarks for EST Clustering Algorithms. Technical Report-on ESTSim. Available at: http://gi.cebitec.uni-bielefeld.de/people/zsuzsa/ESTclust/ESTSim.pdf.
40. Hendriksen, R. S., Price, L. B., Schupp, J. M., Gillece, J. D., Kaas, R. S., Engelthaler, D. M., et al. (2011). Population genetics of Vibrio cholerae from Nepal in 2010: evidence on the origin of the Haitian outbreak. MBio 2, e00157-00111. doi: 10.1128/mBio.00157-11.
41. Holtgrewe, M. (2010). Mason-A Read Simulator for Second Generation Sequencing Data. Technical Report TR-B-10-06. Institut für Mathematik und Informatik, Freie Universität Berlin.
42. Homer, N., and Nelson, S. F. (2010). Improved variant discovery through local re-alignment of short-read next-generation sequencing data using SRMA. Genome Biol. 11, R99. doi: 10.1186/gb-2010-11-10-r99.
43. Hu, X., Yuan, J., Shi, Y., Lu, J., Liu, B., Li, Z., et al. (2012). pIRS: profile-based Illumina pair-end reads simulator. Bioinformatics 28, 1533-1535. doi: 10.1093/bioinformatics/bts187.
44. Huang, W., Li, L., Myers, J. R., and Marth, G. T. (2011). ART: a next-generation sequencing read simulator. Bioinformatics 28, 593-594. doi: 10.1093/bioinformatics/btr708.
45. Jia, B., Xuan, L., Cai, K., Hu, Z., Ma, L., and Wei, C. (2013). NeSSM: a next-generation sequencing simulator for metagenomics. PLoS ONE 8:e75448. doi: 10.1371/journal.pone.0075448.
46. Keim, P. S., and Wagner, D. M. (2009). Humans and evolutionary and ecological forces shaped the phylogeography of recently emerged diseases. Nat. Rev. Microbiol. 7, 813-821. doi: 10.1038/nrmicro2219.
47. Kim, S., and Speed, T. (2013). Comparing somatic mutation-callers: beyond Venn diagrams. BMC Bioinformatics 14:189. doi: 10.1186/1471-2105-14-189.
48. Koboldt, D. C., Chen, K., Wylie, T., Larson, D. E., Mclellan, M. D., Mardis, E. R., et al. (2009). VarScan: variant detection in massively parallel sequencing of individual and pooled samples. Bioinformatics 25, 2283-2285. doi: 10.1093/bioinformatics/btp373.
49. Koren, S., and Phillippy, A. M. (2015). One chromosome, one contig: complete microbial genomes from long-read sequencing and assembly. Curr. Opin. Microbiol. 23C, 110-120. doi: 10.1016/j.mib.2014.11.014.
50. Kozarewa, I., Ning, Z., Quail, M. A., Sanders, M. J., Berriman, M., and Turner, D. J. (2009). Amplification-free Illumina sequencing-library preparation facilitates improved mapping and assembly of (G+++C)-biased genomes. Nat. Methods 6, 291-295. doi: 10.1038/nmeth.1311.
51. Kurtz, S., Phillippy, A., Delcher, A., Smoot, M., Shumway, M., Antonescu, C., et al. (2004). Versatile and open software for comparing large genomes. Genome Biol. 5, R12. doi: 10.1186/gb-2004-5-2-r12.
52. Li, H. (2011). A statistical framework for SNP calling, mutation discovery, association mapping and population genetical parameter estimation from sequencing data. Bioinformatics 27, 2987-2993. doi: 10.1093/bioinformatics/btr509.
53. Li, H., Ruan, J., and Durbin, R. (2008). Mapping short DNA sequencing reads and calling variants using mapping quality scores. Genome Res. 18, 1851-1858. doi: 10.1101/gr.078212.108.
54. Loman, N. J., Constantinidou, C., Chan, J. Z., Halachev, M., Sergeant, M., Penn, C. W., et al. (2012). High-throughput bacterial genome sequencing: an embarrassment of choice, a world of opportunity. Nat. Rev. Microbiol. 10, 599-606. doi: 10.1038/nrmicro2850.
55. Lou, D. I., Hussmann, J. A., Mcbee, R. M., Acevedo, A., Andino, R., Press, W. H., et al. (2013). High-throughput DNA sequencing errors are reduced by orders of magnitude using circle sequencing. Proc. Natl. Acad. Sci. U.S.A. 110, 19872-19877. doi: 10.1073/pnas.1319590110.
56. Magoc, T., Pabinger, S., Canzar, S., Liu, X., Su, Q., Puiu, D., et al. (2013). GAGE-B: an evaluation of genome assemblers for bacterial organisms. Bioinformatics 29, 1718-1725. doi: 10.1093/bioinformatics/btt273.
57. McElroy, K., Thomas, T., and Luciani, F. (2014). Deep sequencing of evolving pathogen populations: applications, errors, and bioinformatic solutions. Microb. Inform. Exp. 4, 1. doi: 10.1186/2042-5783-4-1.
58. McElroy, K. E., Luciani, F., and Thomas, T. (2012). GemSIM: general, error-model based simulator of next-generation sequencing data. BMC Genomics 13:74. doi: 10.1186/1471-2164-13-74.
59. McKenna, A., Hanna, M., Banks, E., Sivachenko, A., Cibulskis, K., Kernytsky, A., et al. (2010). The genome analysis toolkit: a mapreduce framework for analyzing next-generation DNA sequencing data. Genome Res. 20, 1297-1303. doi: 10.1101/gr.107524.110.
60. Meacham, F., Boffelli, D., Dhahbi, J., Martin, D., Singer, M., and Pachter, L. (2011). Identification and correction of systematic error in high-throughput sequence data. BMC Bioinformatics 12:451. doi: 10.1186/1471-2105-12-451.
61. Medvedev, P., Scott, E., Kakaradov, B., and Pevzner, P. (2011). Error correction of high-throughput sequencing datasets with non-uniform coverage. Bioinformatics 27, i137-i141. doi: 10.1093/bioinformatics/btr208.
62. Metzker, M. L. (2010). Sequencing technologies-the next generation. Nat. Rev. Genet. 11, 31-46. doi: 10.1038/nrg2626.
63. Myers, G. (1999). A dataset generator for whole genome shotgun sequencing. Proc. Int. Conf. Intell. Syst. Mol. Biol. 202-210.
64. Nagarajan, N., and Pop, M. (2013). Sequence assembly demystified. Nat. Rev. Genet. 14, 157-167. doi: 10.1038/nrg3367.
65. Nelson, C. L., Pelak, K., Podgoreanu, M. V., Ahn, S. H., Scott, W. K., Allen, A. S., et al. (2014). A genome-wide association study of variants associated with acquisition of Staphylococcus aureus bacteremia in a healthcare setting. BMC Infect. Dis. 14:83. doi: 10.1186/1471-2334-14-83.
66. Nielsen, R., Paul, J. S., Albrechtsen, A., and Song, Y. S. (2011). Genotype and SNP calling from next-generation sequencing data. Nat. Rev. Genet. 12, 443-451. doi: 10.1038/nrg2986.
67. Nikolenko, S. I., Korobeynikov, A. I., and Alekseyev, M. A. (2013). BayesHammer: bayesian clustering for error correction in single-cell sequencing. BMC Genomics 14(Suppl. 1):S7. doi: 10.1186/1471-2164-14-S1-S7.
68. Pabinger, S., Dander, A., Fischer, M., Snajder, R., Sperk, M., Efremova, M., et al. (2013). A survey of tools for variant analysis of next-generation genome sequencing data. Brief. Bioinform. 15, 256-278. doi: 10.1093/bib/bbs086.
69. Pavlopoulos, G., Oulas, A., Iacucci, E., Sifrim, A., Moreau, Y., Schneider, R., et al. (2013). Unraveling genomic variation from next generation sequencing data. BioData Min. 6, 13. doi: 10.1186/1756-0381-6-13.
70. Pearson, T., Okinaka, R. T., Foster, J. T., and Keim, P. (2009). Phylogenetic understanding of clonal populations in an era of whole genome sequencing. Infect. Genet. Evol. 9, 1010-1019. doi: 10.1016/j.meegid.2009.05.014.
71. Peng, Y., Leung, H. M., Yiu, S. M., and Chin, F. L. (2010). "IDBA-a practical iterative de Bruijn graph De Novo assembler," in Research in Computational Molecular Biology, ed. B. Berger (Berlin: Springer), 426-440.
72. Pightling, A. W., Petronella, N., and Pagotto, F. (2014). Choice of reference sequence and assembler for alignment of Listeria monocytogenes short-read sequence data greatly influences rates of error in SNP analyses. PLoS ONE 9:e104579. doi: 10.1371/journal.pone.0104579.
73. Rasko, D. A., Worsham, P. L., Abshire, T. G., Stanley, S. T., Bannan, J. D., Wilson, M. R., et al. (2011). Bacillus anthracis comparative genome analysis in support of the Amerithrax investigation. Proc. Natl. Acad. Sci. U.S.A. 108, 5027-5032. doi: 10.1073/pnas.1016657108.
74. Richter, D. C., Ott, F., Auch, A. F., Schmid, R., and Huson, D. H. (2008). MetaSim: a sequencing simulator for genomics and metagenomics. PLoS ONE 3:e3373. doi: 10.1371/journal.pone.0003373.
75. Ross, M. G., Russ, C., Costello, M., Hollinger, A., Lennon, N. J., Hegarty, R., et al. (2013). Characterizing and measuring bias in sequence data. Genome Biol. 14, R51. doi: 10.1186/gb-2013-14-5-r51.
76. Sahl, J. W., Gillece, J. D., Schupp, J. M., Waddell, V. G., Driebe, E. M., Engelthaler, D. M., et al. (2013). Evolution of a pathogen: a comparative genomics analysis identifies a genetic pathway to pathogenesis in Acinetobacter. PLoS ONE 8:e54287. doi: 10.1371/journal.pone.0054287.
77. Schbath, S., Martin, V., Zytnicki, M., Fayolle, J., Loux, V., and Gibrat, J. F. (2012). Mapping reads on a genomic sequence: an algorithmic overview and a practical comparative analysis. J. Comput. Biol. 19, 796-813. doi: 10.1089/cmb.2012.0022.
78. Schmitt, M. W., Kennedy, S. R., Salk, J. J., Fox, E. J., Hiatt, J. B., and Loeb, L. A. (2012). Detection of ultra-rare mutations by next-generation sequencing. Proc. Natl. Acad. Sci. U.S.A. 109, 14508-14513. doi: 10.1073/pnas.1208715109.
79. Shen, Y., Wan, Z., Coarfa, C., Drabek, R., Chen, L., Ostrowski, E. A., et al. (2010). A SNP discovery method to assess variant allele probability from next-generation resequencing data. Genome Res. 20, 273-280. doi: 10.1101/gr.096388.109.
80. Showalter, A. K., and Tsai, M.-D. (2002). A reexamination of the nucleotide incorporation fidelity of DNA polymerases. Biochemistry 41, 10571-10576. doi: 10.1021/bi026021i.
81. Subramanian, S., Di Pierro, V., Shah, H., Jayaprakash, A. D., Weisberger, I., Shim, J., et al. (2013). MiST: a new approach to variant detection in deep sequencing datasets. Nucleic Acids Res. 41, e154. doi: 10.1093/nar/gkt551.
82. Swain, M. T., Tsai, I. J., Assefa, S. A., Newbold, C., Berriman, M., and Otto, T. D. (2012). A post-assembly genome-improvement toolkit (PAGIT) to obtain annotated genomes from contigs. Nat. Protoc. 7, 1260-1284. doi: 10.1038/nprot.2012.068.
83. Talwalkar, A., Liptrap, J., Newcomb, J., Hartl, C., Terhorst, J., Curtis, K., et al. (2013). SMaSH: a benchmarking toolkit for human genome variant calling. Bioinformatics 30, 2787-2795. doi: 10.1093/bioinformatics/btu345.
84. Treangen, T. J., Ondov, B. D., Koren, S., and Phillippy, A. M. (2014). The Harvest suite for rapid core-genome alignment and visualization of thousands of intraspecific microbial genomes. Genome Biol. 15, 524. doi: 10.1186/s13059-014-0524-x.
85. Van der Auwera, G. A., Carneiro, M. O., Hartl, C., Poplin, R., Del Angel, G., Levy-Moonshine, A., et al. (2012). From fastQ data to high-confidence variant calls: the Genome Analysis Toolkit Best Practices Pipeline. Curr. Protoc. Bioinformatics 43, 11.10.1-11.10.33. doi: 10.1002/0471250953.bi1110s43.
86. Vogler, A. J., Keys, C., Nemoto, Y., Colman, R. E., Jay, Z., and Keim, P. (2006). Effect of repeat copy number on variable-number tandem repeat mutations in Escherichia coli O157:H7. J. Bacteriol. 188, 4253-4263. doi: 10.1128/JB.00001-06.
87. Walker, B. J., Abeel, T., Shea, T., Priest, M., Abouelliel, A., Sakthikumar, S., et al. (2014). Pilon: an integrated tool for comprehensive microbial variant detection and genome assembly improvement. PLoS ONE 9:e112963. doi: 10.1371/journal.pone.0112963.
88. Zeng, F., Jiang, R., and Chen, T. (2013). PyroHMMvar: a sensitive and accurate method to call short indels and SNPs for Ion Torrent and 454 data. Bioinformatics 29, 2859-2868. doi: 10.1093/bioinformatics/btt512.
89. Zerbino, D. R., and Birney, E. (2008). Velvet: algorithms for de novo short read assembly using de Bruijn graphs. Genome Res. 18, 821-829. doi: 10.1101/gr.074492.107.
90. Zhong, C., Yang, Y., and Yooseph, S. (2015). GRASP: guided reference-based assembly of short peptides. Nucleic Acids Res. 43, e18. doi: 10.1093/nar/gku1210.
91. Zook, J. M., Chapman, B., Wang, J., Mittelman, D., Hofmann, O., Hide, W., et al. (2014). Integrating human sequence data sets provides a resource of benchmark SNP and indel genotype calls. Nat. Biotechnol. 32, 246-251. doi: 10.1038/nbt.2835.
92. Zook, J. M., Samarov, D., Mcdaniel, J., Sen, S. K., and Salit, M. (2012). Synthetic spike-in standards improve run-specific systematic error analysis for DNA and RNA sequencing. PLoS ONE 7:e41356. doi: 10.1371/journal.pone.0041356.