A benchmark for evaluation of algorithms for identification of cellular correlates of clinical outcomes

Cytometry Part A

Volumn 89, Issue 1, 2016, Pages 16-21

  Aghaeepour, Nima ^a,b,c   Chattopadhyay, Pratip ^d   Chikina, Maria ^e   Dhaene, Tom ^f   Van Gassen, Sofie ^f,g,h   Kursa, Miron ⁱ   Lambrecht, Bart N ^g,h   Malek, Mehrnoush ^a   Mclachlan, G J ^j   Qian, Yu ^k   Qiu, Peng ^l   Saeys, Yvan ^g,h   Stanton, Rick ^k   Tong, Dong ^m,n   Vens, Celine ^g,h,o   Walkowiak, Slawomir ⁱ   Wang, Kui ^j,p   Finak, Greg ^q   Gottardo, Raphael ^q   Mosmann, Tim ^r   Nolan, Garry P ^c   Scheuermann, Richard H ^k,s   Brinkman, Ryan R ^a,b   more..

^a BRITISH COLUMBIA CANCER AGENCY   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^d NATIONAL INSTITUTES OF HEALTH   (United States)

^e UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^f GHENT UNIVERSITY   (Belgium)

^g DEPARTMENT OF PLANT SYSTEMS BIOLOGY   (Belgium)

^h GHENT UNIVERSITY HOSPITAL   (Belgium)

ⁱ UNIVERSITY OF WARSAW   (Poland)

^j UNIVERSITY OF QUEENSLAND   (Australia)

^k J CRAIG VENTER INSTITUTE   (United States)

^l Emory University   (United States)

^m NOTTINGHAM TRENT UNIVERSITY   (United Kingdom)

ⁿ UNIVERSITY OF EDINBURGH   (United Kingdom)

^o UNIVERSITY OF LEUVEN   (Belgium)

^p SHIHEZI UNIVERSITY   (China)

^q Fred Hutchinson Cancer Research Center   (United States)

^r UNIVERSITY OF ROCHESTER MEDICAL CENTER   (United States)

^s UNIVERSITY OF CALIFORNIA   (United States)

more..

Author keywords

Bioinformatics; Classification; Clinical outcome; Clustering; Data analysis; Flow cytometry; HIV; Supervised analysis

Indexed keywords

ACQUIRED IMMUNODEFICIENCY SYNDROME; ALGORITHM; BENCHMARKING; BIOLOGY; CYTOLOGY; DISEASE COURSE; FLOW CYTOMETRY; HUMAN; HUMAN IMMUNODEFICIENCY VIRUS INFECTION; PATHOLOGY; PROCEDURES; STAINING; STATISTICAL ANALYSIS; T LYMPHOCYTE;

ACQUIRED IMMUNODEFICIENCY SYNDROME; ALGORITHMS; BENCHMARKING; COMPUTATIONAL BIOLOGY; DATA INTERPRETATION, STATISTICAL; DISEASE PROGRESSION; FLOW CYTOMETRY; HIV SEROPOSITIVITY; HUMANS; STAINING AND LABELING; T-LYMPHOCYTES;

EID: 84957081995     PISSN: 15524922     EISSN: 15524930     Source Type: Journal    
DOI: 10.1002/cyto.a.22732     Document Type: Article

References (40)

1. O'Neill K, Aghaeepour N, Spidlen J, Brinkman R. Flow cytometry bioinformatics. PLoS Comput. Biol. 2013;9:e10033656.
2. Aghaeepour N, Finak G, FlowCAP Consortium, DREAM Consortium, Hoos H, Mosmann TR, Brinkman R, Gottardo R, Scheuermann RH. Critical assessment of automated flow cytometry data analysis techniques. Nat Methods 2013; 10:228-238.
3. Weintrob AC, Fieberg AM, Agan BK, Ganesan A, Crum-Cianflone NF, Marconi VC, Roediger M, Fraser SL, Wegner SA, Wortmann GW. Increasing age at HIV seroconversion from 18 to 40 years is associated with favorable virologic and immunologic responses to HAART. JAIDS J Acquired Immune Deficiency Syndromes 2008; 49:40-47.
4. Ganesan A, Chattopadhyay PK, Brodie TM, Qin J, Gu W, Mascola JR, Michael NL, Follmann DA, Roederer M, Infectious Disease Clinical Research Program HIV Working Group. Immunologic and virologic events in early HIV infection predict subsequent rate of progression. J Infect Dis 2010; 201:272-284.
5. MacQueen J. Some methods for classification and analysis of multivariate observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Vol 1, no. 14, p 281-297.
6. Moore WA, Parks DR. Update for the logical data scale including operational code implementations. Cytom A 2012;81:273-277.
7. O'Neill K, Jalali A, Aghaeepour N, Hoos H, Brinkman RR. Enhanced flowType/RchyOptimyx: A bioconductor pipeline for discovery in high-dimensional cytometry data. Bioinformatics 2014; 30:1329-1330.
8. Aghaeepour N, Jalali A, O'Neill K, Chattopadhyay PK, Roederer M, Hoos HH, Brinkman RR. "RchyOptimyx: Cellular hierarchy optimization for flow cytometry. Cytom A 2012; 81:1022-1030.
9. Malek M, Taghiyar MJ, Chong L, Finak G, Gottardo R, Brinkman RR. flowDensity: Reproducing manual gating of flow cytometry data by automated density-based cell population identification. Bioinformatics 2014;31:606-607.
10. Aghaeepour N, Chattopadhyay PK, Ganesan A, O'Neill K, Zare H, Jalali A, Hoos HH, Roederer M, Brinkman RR. Early immunologic correlates of HIV protection can be identified from computational analysis of complex multivariate T-cell flow cytometry assays. Bioinformatics 2012;28:1009-1016.
11. Tong DL, Ball GR, Pockley AG. gEM/GANN: A multivariate computational strategy for auto-characterizing relationships between cellular and clinical phenotypes and predicting disease progression time using high-dimensional flow cytometry data. Cytometry Part A 2015;87AL616-623.
12. Van Gassen S, Vens C, Dhaene T, Lambrecht BN, Saeys Y. FloReMi: Flow density survival regression using minimal feature redundancy. Cytometry Part A 2015; DOI: 10.1002/cyto.a.22734.
13. Tong DL, Mintram R. Genetic algorithm-neural network (GANN): A study of neural network activation functions and depth of genetic algorithm search applied to feature selection. Int J Mach Learn Cybernet 2010;1:75-87.
14. Tong DL, Schierz AC. Hybrid genetic algorithm-neural network: Feature extraction for unpreprocessed microarray data. Artif Intell Med 2011;53:47-56.
15. Rogers WT, Holyst HA. FlowFP: A bioconductor package for fingerprinting flow cytometric data. Adv Bioinform 2009;2009:193947-11. http://www.hindawi.com/journals/abi/2009/193947/abs
16. Qiu P, Simonds EF, Bendall SC, Gibbs KD Jr, Bruggner RV, Linderman MD, Sachs K, Nolan GP, Plevritis SK. Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE. Nat Biotechnol 2011; 29:886-891.
17. McLachlan G, Krishnan T. The EM Algorithm and Extensions, Vol. 382. Hoboken, NJ: Wiley; 2007.
18. Chattopadhyay PK, Roederer M. Good cell, bad cell: Flow cytometry reveals T-cell subsets important in HIV disease. Cytom A 2010;77:614-622.
19. Papagno L, Spina CA, Marchant A, Salio M, Rufer N, Little S, Dong T, Chesney G, Waters A, Easterbrook P et al. Immune activation and CD8+ T-cell differentiation toward senescence in HIV-1 infection. PLoS Biol 2004; 2:e20.
20. Perfetto SP, Chattopadhyay PK, Lamoreaux L, Nguyen R, Ambrozak D, Koup RA, Roederer M. Amine reactive dyes: An effective tool to discriminate live and dead cells in polychromatic flow cytometry. J Immunol Methods 2006; 313:199-208.
21. Bruhns P. Properties of mouse and human IgG receptors and their contribution to disease models. Blood 2012; 119:5640-5649.
22. Müller-Trutwin M, Hosmalin A. 2005. Role for plasmacytoid dendritic cells in anti-HIV innate immunity. Immunol Cell Biol 2005; 83:578-583.
23. Sabado RL, O'Brien M, Subedi A, Qin L, Hu N, Taylor E, Dibben O, Stacey A, Fellay J, Shianna KV, et al. Evidence of dysregulation of dendritic cells in primary HIV infection. Blood 2010;116:3839-3852.
24. Sandler NG, Wand H, Roque A, Law M, Nason MC, Nixon DE, Pedersen C, Ruxrungtham K, Lewin SR, Emery S, et al., Plasma levels of soluble CD14 independently predict mortality in HIV infection. J Infect Dis 2011; 203:780-790.
25. Hasegawa A, Liu H, Ling B, Borda JT, Alvarez X, Sugimoto C, Vinet-Oliphant H, Kim W-K, Williams KC, Ribeiro RM, et al. The level of monocyte turnover predicts disease progression in the macaque model of AIDS. Blood 2009;114:2917-2925.
26. Qian Y, Liu Y, Campbell J, Thomson E, Kong YM, Scheuermann RH. FCSTrans: An open source software system for FCS file conversion and data transformation. Cytom A 2012;81:353-356.
27. Hahne F, Khodabakhshi AH, Bashashati A, Wong C-J, Gascoyne RD, Weng AP, Seyfert-Margolis V, Bourcier K, Asare A, Lumley T, et al. Per-channel basis normalization methods for flow cytometry data. Cytom A 2010;77:121-131.
28. Ge Y, Sealfon SC. flowPeaks: A fast unsupervised clustering for flow cytometry data via K-means and density peak finding. Bioinformatics 2012; 28:2052-2058.
29. Roederer M, Treister A, Moore W, Herzenberg LA. Probability binning comparison: A metric for quantitating univariate distribution differences. Cytometry 2001; 45:37-46.
30. Azad A, Khan A, Rajwa B, Pyne S, Pothen A. "Classifying Immunophenotypes With Templates From Flow Cytometry, " presented at the International Conference, New York, New York, USA, 2007, pp. 256-265.
31. Dundar M, Akova F, Yerebakan HZ, Rajwa B . A non-parametric Bayesian model with random effects: Joint cell clustering and cluster matching for anomalous sample phenotype identification. BMC Bioinformatics 2014;15(1):314.
32. Tusher VG, Tibshirani R, Chu G. Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci USA 2001; 98:5116-5121.
33. Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automated identification of stratifying signatures in cellular subpopulations. Proc Natl Acad Sci USA 2014; 111:E2770-E2777.
34. Gaudillière B, Fragiadakis GK, Bruggner RV, Nicolau M, Finck R, Tingle M, Silva J, Ganio EA, Yeh CG, Maloney WJ. et al., Clinical recovery from surgery correlates with single-cell immune signatures. Sci Transl Med 2014; 6:255ra131.
35. Zare H, Haffari G, Gupta A, Brinkman RR. Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis. BMC Genom 2013;14:S14.
36. Anchang B, Do MT, Zhao X, Plevritis SK. CCAST: A model-based gating strategy to isolate homogeneous subpopulations in a heterogeneous population of single cells. PLoS Comput Biol 2014; 10:e1003664.
37. Amir E-AD, Davis KL, Tadmor MD, Simonds EF, Levine JH, Bendall SC, Shenfeld DK, Krishnaswamy S, Nolan GP, Pe'er D. viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia. Nat Biotechnol 2013; 31:545-552.
38. Shekhar K, Brodin P, Davis MM, Chakraborty AK. Automatic classification of cellular expression by nonlinear stochastic embedding (ACCENSE). Proc Natl Acad Sci USA 2014;111:202-207.
39. Finak G, Frelinger J, Jiang W, Newell EW, Ramey J, Davis MM, Kalams SA, De Rosa SC, Gottardo R. OpenCyto: An open source infrastructure for scalable, robust, reproducible, and automated, end-to-end flow cytometry data analysis. PLoS Comput Biol 2014; 10:e1003806.
40. Spidlen J, Breuer K, Rosenberg C, Kotecha N, Brinkman RR. FlowRepository: A resource of annotated flow cytometry datasets associated with peer-reviewed publications. Cytometry Part A 2012; 81:727-731.