A review of ensemble methods in bioinformatics

Current Bioinformatics

Volumn 5, Issue 4, 2010, Pages 296-308

  Yang, Pengyi ^a,b,c   Yang, Yee Hwa ^b   Zhou, Bing B ^a,b   Zomaya, Albert Y ^a,b  

^a UNIVERSITY OF SYDNEY   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

^c CSIRO   (Australia)

Author keywords

Bioinformatics; Ensemble feature selection; Ensemble learning; Ensemble of support vector machines; Gene gene interaction; Mass spectrometry based proteomics; Meta ensemble; Microarray; Regulatory elements prediction

Indexed keywords

DNA;

ALGORITHM; AMINO ACID SEQUENCE; ARTICLE; BIOINFORMATICS; BIOLOGY; EXON; GENE EXPRESSION; GENE INTERACTION; GENOME; LEARNING; MACHINE; MASS SPECTROMETRY; PRIORITY JOURNAL; PROTEOMICS; RECOGNITION; SINGLE NUCLEOTIDE POLYMORPHISM;

EID: 78951491903     PISSN: 15748936     EISSN: None     Source Type: Journal    
DOI: 10.2174/157489310794072508     Document Type: Article

References (112)

1. Larranaga P, Calvo B, Santana R, et al. Machine learning in bioinformatics. Briefings Bioinform 2003; 7: 86-112.
2. Allison DB, Cui X, Page GP, et al. Microarray data analysis: from disarray to consolidation and consensus. Nat Rev Genet 2006; 7: 55-66.
3. Aebersold R, Mann M. Mass spectrometry-based proteomics. Nature 2003; 422: 198-207.
4. Hirschhorn J, Daly M. Genome-wide association studies for common diseases and complex traits. Nat Rev Genet 2005; 6: 95-108.
5. Zeng J, Zhu S, Yan H. Towards accurate human promoter recognition: a review of currently used sequence features and classification methods. Brief Bioinform 2009; 10: 498-508.
6. Valencia A, Pazos F. Computational methods for the prediction of protein interactions. Curr Opin Struct Biol 2002; 12: 368-73.
7. Jones DT. Protein structure prediction in genomics. Brief Bioin form 2001; 2: 111-25.
8. Dietterich TG. Ensemble methods in machine learning. In: Proceedings of Multiple Classifier System, LNCS 1857. Springer: Italy 2000; pp. 1-15.
9. Dietterich TG. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Mach Learn 2000; 40: 139-158.
10. Breiman L. Bagging predictors. Mach Learn 1996; 26: 123-140.
11. Freund Y, Schapire R. Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth National Conference on Mach Learn: Italy 1996; pp. 148-56.
12. Breiman L. Random forests. Mach Learn 2001; 45: 5-32.
13. Kuncheva L. Combining pattern classifiers: methods and algorithms. Wiley: New Jersey 2004.
14. Webb GI, Zheng Z. Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques. IEEE Trans Knowl Data Eng 2004; 16: 980-91.
15. Breiman L. Arcing classifiers (with discussion). Ann Appl Probab 1998; 26: 801-49.
16. Schapire RE, Freund Y, Bartlett P, et al. Boosting the margin: a new explanation for the effectiveness of voting methods. Ann Appl Probab 1998; 26: 1651-86.
17. Tsymbal A, Pechenizkiy M, Cunningham P. Diversity in search strategies for ensemble feature selection. Inf Fusion 2005; 6: 83-98.
18. Wolpert DH. Stacked generalization. IEEE Trans Neural Netw 1992; 5: 241-59.
19. Kuncheva LI. Switching between selection and fusion in combining classifiers: an experiment. IEEE Trans Syst Man Cybern 2002; 32: 146-56.
20. Gama J, Brazdil P. Cascade generalization. Mach Learn 2000; 41: 315-43.
21. Lam L, Suen Y. Application of majority voting to pattern recognition: an analysis of its behavior and performance. IEEE Trans Syst Man Cybern 1997; 27: 553-68.
22. Bahler D, Navarro L. Methods for combining heterogeneous sets of classifiers. Proceedings of the 17th National Conference on Artificial Intelligence (AAAI), Workshop on New Research Problems for Machine Learning. Austin, Texas 2000.
23. Kang H, Kim K, Kim J. A framework for probabilistic combination of multiple classifiers at an abstract level. Eng Appl Artif Intell 1997; 10: 379-85.
24. Kittler J, Hatef M, Duin RP, et al. On combining classifiers. IEEE Trans Pattern Anal Mach Intell 1998; 20: 226-39.
25. Asyali MH, Colak D, Demirkaya O, et al. Gene expression profile classification: a review. Curr Bioinform 2006; 1: 55-73.
26. Somorjai RL, Dolenko B, Baumgartner R, et al. Class prediction and discovery using gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions. Bioinformatics 2003; 19: 1484-91.
27. Saeys Y, Lnza I, Larranaga P. A review of feature selection techniques in bioinformatics. Bioinformatics 2007; 23: 2507-17.
28. Hilario M, Kalousis A. Approaches to dimensionality reduction in proteomic biomarker studies. Brief Bioinform 2008; 9: 102-18.
29. Braga-Neto U, Dougherty E. Is cross-validation valid for smallsample microarray classification? Bioinformatics 2004; 20: 374-80.
30. Ben-Dor A, Bruhn L, Friedman N, et al. Tissue classification with gene expression profiles. Int J Comput 2000; 7: 559-83.
31. Dudoit S, Fridlyand J, Speed T. Comparison of discrimination methods for the classification of tumors using gene expression data. J Am Stat Assoc 2002; 97: 77-87.
32. Dettling M, Buhlmann P. Boosting for tumor classification with gene expression data. Bioinformatics 2003; 19: 1061-9.
33. Long P. Boosting and microarray data. Mach Learn 2003; 53: 31-44.
34. Tan A, Gilbert D. Ensemble machine learning on gene expression data for cancer classification. Appl Bioinformatics 2003; 2: S75- S83.
35. Qu Y, Adam B, Yasui Y, et al. Boosted decision tree analysis of surface-enhanced laser desorption/ionization mass spectral serum profiles discriminates prostate cancer from noncancer patients. Clin Chem 2002; 48: 1835-43.
36. Wu B, Abbott T, Fishman D, et al. Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data. Bioinformatics 2003; 19: 1636-43.
37. Gertheiss J, Tutz G. Supervised feature selection in mass spectrometry-based proteomic profiling by blockwise boosting. Bioinformatics 2009; 25: 1076-7.
38. Yasui Y, Pepe M, Thompson M, et al. A data-analytic strategy for protein biomarker discovery: profiling of high-dimensional proteomic data for cancer detection. Biostatistics 2003; 4: 449-63.
39. Lee J, Lee J, Park M, et al. An extensive comparison of recent classification tools applied to microarray data. Comput Stat Data Anal 2005; 48: 869-85.
40. Diaz-Uriarte R, de Andres S. Gene selection and classification of microarray data using random forest. BMC Bioinformatics 2006; 7: 3.
41. Izmirlian G. Application of the random forest classification algorithm to a SELDI-TOF proteomics study in the setting of a cancer prevention trial. Ann N Y Acad Sci 2004; 1020: 154-74.
42. Zhang H, Yu C, Singer B. Cell and tumor classification using gene expression data: Construction of forests. Proc Indian Natl Sci Acad B Biol Sci 2003; 100: 4168-72.
43. Geurts P, Fillet M, Seny D, et al. Proteomic mass spectra classification using decision tree based ensemble methods. Bioinformatics 2005; 21: 3138-45.
44. Dougherty ER, Sima C, Hanczar B, et al. Performance of error estimators for classification. Curr Bioinform 2010; 5: 53-67.
45. Ge G, Wong G. Classification of premalignant pancreatic cancer mass-spectrometry data using decision tree ensembles. BMC Bioinformatics 2008; 9: 275.
46. Statnikov A, Wang L, Aliferis C. A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification. BMC Bioinformatics 2008; 9: 319.
47. Curtis RK, Oresic M, Vidal-Puig A. Pathways to the analysis of microarray data. Trends Biotechnol 2005; 23: 429-35.
48. Pang H, Lin A, Holford M, et al. Pathway analysis using random forests classification and regression. Bioinformatics 2006; 22: 2028-36.
49. Kanehisa M, Araki M, Goto S, et al. KEGG for linking genomes to life and the environment. Nucleic Acids Res 2008; 36: D480-D484.
50. Montana G. Statistical methods in genetics. Brief Bioinform 2006; 7: 297-308.
51. Cordell JH. Detecting gene-gene interactions that underlie human diseases. Nat Rev Genet 2009; 10: 392-404.
52. Zhang H, Bonney G. Use of classification trees for association studies. Genet Epidemiol 2000; 19: 323-32.
53. Huang J, Lin A, Narasimhan B, et al. Tree-structured supervised learning and the genetics of hypertension. Proc Natl Acad Sci USA 2004; 101: 10529-34.
54. Cook N, Zee R, Ridker P. Tree and spline based association analysis of gene-gene interaction models for ischemic stroke. Stat Med 2004; 23: 1439-53.
55. Ye Y, Zhong X, Zhang H. A genome-wide tree-and forest-based association analysis of comorbidity of alcoholism and smoking. BMC Genetics 2005; 6: S135.
56. Zhang Z, Zhang S, Wong MY, et al. An ensemble learning approach jointly modeling main and interaction effects in genetic association studies. Genet Epidemiol 2008; 32: 285-300.
57. McKinney BA, Reif DM, Ritchie MD, et al. Machine learning for detecting gene-gene interactions: a review. Appl Bioinformatics 2006; 5: 77-88.
58. Bureau A, Dupuis J, Hayward B, et al. Mapping complex traits using random forests. BMC Genet 2003; 4: S64.
59. Bureau A, Dupuis J, Falls K, et al. Identifying SNPs predictive of phenotype using random forests. Genet Epidemiol 2005; 28: 171-82.
60. Lunetta KL, Hayward LB, Segal J, et al. Screening large-scale association study data: exploiting interactions using random forests. BMC Genet 2004; 5: 32
61. Chen X, Liu C, Zhang M, et al. A forest-based approach to identifying gene and gene-gene interactions. Proc Natl Acad Sci USA 2007; 104: 19199-203.
62. Klein RJ, Zeiss C, Chew EY, et al. Complement factor H polymorphism in age-related macular degeneration. Science 2005; 308: 385-9.
63. Meng Y, Yu Y, Cupples L, et al. Performance of random forest when SNPs are in linkage disequilibrium. BMC Bioinform 2009; 10: 78.
64. Jiang R, Tang W, Wu X, et al. A random forest approach to the detection of epistatic interactions in case-control studies. BMC Bioinform 2009; 10: S65.
65. Zhang M. Computational analyses of eukaryotic promoters. BMC Bioinform 2007; 8: S3.
66. Down TA, Hubbard TJP. Computational detection and location of transcription start sites in mammalian genomic DNA. Genome Res 2002; 12: 458-61.
67. Blon N, Ponten T, Gupta R, et al. Prediction of post-translational glycosilation and phosphorylation of proteins from amino acid sequence. Proteomics Clin Appl 2004; 4: 1633-49.
68. Hong P, Liu XS, Zhou Q, et al. A boosting approach for motif modeling using ChIP-chip data. Bioinformatics 2005; 21: 2636-43.
69. Xie X, Wu S, Lam KM, et al. PromoterExplorer: an effective promoter identification method based on the AdaBoost algorithm. Bioinformatics 2006; 22: 2722-8.
70. Zhao X, Xuan Z, Zhang M. Boosting with stumps for predicting transcription start sites. Genome Biol 2007; 8: R17.
71. Wang X, Xuan Z, Zhao X, et al. High-resolution human corepromoter prediction with CoreBoost_HM. Genome Res 2009; 19: 266-75.
72. Gordon JJ, Towsey MW, Hogan JM, et al. Improved prediction of bacterial transcription start sites. Bioinformatics 2006; 22: 142-8.
73. Hamby SE, Hirst JD. Prediction of glycosylation sites using random forests. BMC Bioinformatics 2008; 9: 500.
74. Caragea C, Sinapov J, Silvescu A, et al. Glycosylation site prediction using ensembles of Support Vector Machine classifiers. BMC Bioinformatics 2007; 8: 438.
75. Yoo PD, Ho YS, Zhou BB, et al. SiteSeek: Post-translational modification analysis using adaptive locality-effective kernel methods and new profiles. BMC Bioinformatics 2008; 9: 272.
76. Guan Y, Myers C, Hess D, et al. Predicting gene function in a hierarchical context with an ensemble of classifiers. Genome Biol 2008; 9: S3.
77. Shen HB, Chou KC. Ensemble classifier for protein fold pattern recognition. Bioinformatics 2006; 22: 1717-22.
78. Melvin I, Weston J, Leslie CS, et al. Combining classifiers for improved classification of proteins from sequence or structure. BMC Bioinformatics 2008; 9: 389.
79. Lee B, Shin M, Oh Y, et al. Identification of protein functions using a machine learning approach based on sequence-derived properties. Proteome Sci 2009; 7: 27.
80. Wang SQ, Yang J, Chou KC. Using stacked generalization to predict membrane protein types based on pseudo-amino acid composition. J Theor Biol 2006; 242: 941-6.
81. Chen XW, Liu M. Prediction of protein-protein interactions using random decision forest framework. Bioinformatics 2005; 21: 4394-400.
82. Deng L, Guan J, Dong Q, et al. Prediction of protein-protein interaction sites using an ensemble method. BMC Bioinformatics 2009; 10: 426.
83. Wu CC, Asgharzadeh S, Triche TJ, et al. Prediction of human functional genetic networks from heterogeneous data using RVM- based ensemble learning. Bioinformatics 2010; 26: 807-813.
84. Altay G, Emmert-Streib F. Revealing differences in gene network inference algorithms on the network level by ensemble methods. Bioinformatics 2010; 26: 1738-44.
85. Caragea C, Sinapov J, Silvescu A, et al. Glycosylation site prediction using ensemble of support vector machine classifiers. BMC Bioinformatics 2007; 8: 438.
86. Peng Y. A novel ensemble machine learning for robust microarray data classification. Comput Biol Med 2006; 36: 553-73.
87. Dettling M. Bagboosting for tumor classification with gene expression data. Bioinformatics 2004; 20: 3583-93.
88. Liu KH, Xu CG. A genetic programming-based approach to the classification of multiclass microarray datasets. Bioinformatics 2009; 25: 331-7.
89. Bhanot G, Alexe G, Venkataraghavan B, et al. A robust metaclassification strategy for cancer detection from MS data. Proteomics 2006; 6: 592-604.
90. Kedarisetti KD, Kurgan L, Dick S. Classifier ensembles for protein structural class prediction with varying homology. Biochem Biophys Res Commun 2006; 348: 981-8.
91. Hassan MR, Hossain MM, Bailey J, et al. A voting approach to identify a small number of highly predictive genes using multiple classifiers. BMC Bioinformatics 2009; 10: S19.
92. Yang P, Zhou BB, Zhang Z, et al. A multi-filter enhanced genetic ensemble system for gene selection and sample classification of microarray data. BMC Bioinformatics 2010; 11: S5.
93. Yang P, Ho JWK, Zomaya AY, et al. A genetic ensemble approach for gene-gene interaction identification. BMC Bioinformatics 2010; 11: 524.
94. Liu B, Cui Q, Jiang T, et al. A combinational feature selection and ensemble neural network method for classification of gene expression data. BMC Bioinformatics 2004; 5: 136.
95. Koziol JA, Feng AC, Jia Z, et al. The wisdom of the commons: ensemble tree classifiers for prostate cancer prognosis. Bioinformatics 2009; 25: 54-60.
96. Amaratunga D, Cabrera J, Lee Y. Enriched random forests. Bioinformatics 2008; 24: 2010-4.
97. Yanover C, Singh M, Zaslavsky E. M are better than one: an ensemble-based motif finder and its application to regulatory element prediction. Bioinformatics 2009; 25: 868-74.
98. Yanover C, Weiss Y. Finding the AI most probable configurations using loopy belief propagation. In: Proceedings of advances in neural information processing systems. Vancover: The MIT Press 2004; p. 289.
99. Armananzas R, Inza I, Larranaga P. Detecting reliable gene interactions by a hierarchy of Bayesian network classifiers. Comput Methods Programs Biomed 2008; 91: 110-21.
100. Robles V, Larranaga P, Pena JM, et al. Bayesian network multiclassifiers for protein secondary structure prediction. Artif Intell Med 2004; 31: 117-36.
101. Hu J, Yang YD, Kihara D. EMD: an ensemble algorithm for discovering regulatory motifs in DNA sequences. BMC Bioinformatics 2006; 7: 342.
102. Wijaya E, Yiu SM, Son NT, et al. MotifVoter: a novel ensemble method for fine-grained integration of generic motif finders. Bioinformatics 2008; 24: 2288-95.
103. Boulesteix AL, Slawski M. Stability and aggregation of ranked gene lists. Brief Bioinform 2009; 10: 556-68.
104. Jiang H, Deng Y, Chen HS, et al. Joint analysis of two microarray gene-expression data sets to select lung adenocarcinoma marker genes. BMC Bioinformatics 2004; 5: 81.
105. Levner I. Feature selection and nearest centroid classification for protein mass spectrometry. BMC Bioinformatics 2005; 6: 68.
106. Saeys Y, Abeel T, Van de Peer Y. Robust feature selection using ensemble feature selection techniques. In: Proceedings of the european conference on machine learning and knowledge discovery in databases. Part II, LNCS 5212. Belgium: Springer 2008; pp. 313-25.
107. Dutkowski J, Gambin A. On consensus biomarker selection. BMC Bioinformatics 2007; 8: S5.
108. Zhang Z, Yang P, Wu X, et al. An agent-based hybrid system for microarray data analysis. IEEE Intel Syst 2009; 24: 53-63.
109. Abeel T, Helleputte T, Van de Peer Y, et al. Robust biomarker identification for cancer diagnosis with ensemble feature selection methods. Bioinformatics 2010; 26: 392-8.
110. Netzer M, Millonig G, Osl M, et al. A new ensemble-based algorithm for identifying breath gas marker candidates in liver disease using ion molecule reaction mass spectrometry. Bioinformatics 2009; 25: 941-7.
111. Yang YH, Xiao Y, Segal MR. Identifying differentially expressed genes from microarray experiments via statistic synthesis. Bioinformatics 2005; 21: 1084-93.
112. Chan D, Bridges SM, Burgess SC. An ensemble method for identifying robust features for biomarker identification. In: Liu H, Ed. Computational Methods of Feature Selection. Arizona State University, Tempe, AZ, Hiroshi Motoda, AFOSR/AOARD, Tokyo, Japan. Series: Chapman & Hall/CRC Data Mining and Knowledge Discovery Series 2007; pp. 377-92.