Ensemble Methods: A Review

Advances in Machine Learning and Data Mining for Astronomy

Volumn , Issue , 2012, Pages 563-593

  Re, Matteo ^a   Valentini, Giorgio ^a  

^a UNIVERSITY OF MILAN   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84899928683     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1201/b11822-20     Document Type: Chapter

References (215)

1. AA.VV. Special Issue on Applications of EnsembleMethods, volume 9 (1) of Information Fusion Journal. Elsevier, 2008.
2. D.W. Aha and R. Bankert. Cloud classification using error-correcting output codes. Artificial Intelligence Applications: Natural Resources, Agriculture, and Environmental Science, 11(1):13-28, 1997.
3. N. Alaydie, C.K. Reddy, and F. Fotouhi. Hierarchical multi-label boosting for gene function prediction. In Proceedings of the International Conference on Computational Systems Bioinformatics (CSB), pp. 14-25, Stanford, CA, 2010.
4. E.L. Allwein, R.E. Schapire, and Y. Singer. Reducingmulticlass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research, 1:113-141, 2000.
5. E.L. Allwein, R.E. Schapire, and Y. Singer. Reducing multiclass to binary: A unifying approach for margin classifiers. In Proceedings of the ICML’2000, The Seventeenth International Conference on Machine Learning, pp. 113-141, 2000.
6. E. Alpaydin and C. Kaynak. Cascading classifiers. Kybernetika, 34(4):369-374, 1998.
7. E. Alpaydin and E. Mayoraz. Learning error-correcting output codes fromdata. In ICANN’99, pp. 743-748, Edinburgh, UK, 1999.
8. R. Anand, G. Mehrotra, C.K. Mohan, and S. Ranka. Efficient classification for multiclass problems using modular neural networks. IEEE Transactions on Neural Networks, 6:117-124, 1995.
9. R.E. Banfield, L.O. Hall, K.W. Bowyer, and P. Kegelmeyer. Ensemble diversity measure and their application to thinning. Information Fusion, 6(1):49-62, 2005.
10. R.E. Banfield, L.O. Hall, O. Lawrence, K.W. Bowyer, W. Kevin, and P. Kegelmeyer. A comparison of decision tree ensemble creation techniques. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1):173-180, 2007.
11. E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Machine Learning, 36(1/2):105-139, 1999.
12. M.A. Bautista, X. Baro, O. Pujol, P. Radeva, J. Vitria, and S. Escalera. Compact evolutive design of error-correcting output codes. In O. Okun, M. Re, and G. Valentini, (eds), ECML-SUEMA 2010 Proceedings, pp. 119-128, Barcelona, Spain, 2010.
13. D. Bazell and W.D. Aha. Ensemble of classifiers for morphological galaxy classification. The Astrophysical Journal, 548:219-223, 2001.
14. J. Benediktsson, F. Roli, and J. Kittler. Multiple Classifier Systems, 8th International Workshop, MCS2009, volume 5519 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 2009.
15. J. Benediktsson, J. Sveinsson, O. Ersoy, and P. Swain. Parallel consensual neural networks. IEEE Transactions on Neural Networks, 8:54-65, 1997.
16. J. Benediktsson and P. Swain. Consensus theoretic classification methods. IEEE Transactions on Systems, Man and Cybernetics, 22:688-704, 1992.
17. J.A. Benediktsson, J. Chanussot, and M. Fauvel. Multiple classifier systems in remote sensing:From basics to recent developments. In M. Haindl, J. Kittler, and F. Roli, (eds), Multiple Classifier Systems. Seventh International Workshop, MCS 2007, Prague, Czech Republic, volume 4472 of Lecture Notes in Computer Science, pp. 511-512, Springer, 2007.
18. A. Berger. Error correcting output coding for text classification. In IJCAI’99: Workshop on Machine Learning for Information Filtering, Stockholm, Sweden, 1999.
19. C. M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.
20. L. Breiman. Bagging predictors. Machine Learning, 24(2):123-140, 1996.
21. L. Breiman. Bias, variance and arcing classifiers. Technical Report TR 460, Statistics Department, University of California, Berkeley, CA, 1996.
22. L. Breiman. Arcing classifiers. Annals of Statistics, 26(3):801-849, 1998.
23. L. Breiman. Pasting small votes for classification in large databases and on-line. Machine Learning, 36:85-103, 1999.
24. L. Breiman. Prediction games and arcing classifiers. Neural Computation, 11(7):1493-1517, 1999.
25. L. Breiman. Random forests. Machine Learning, 45(1):5-32, 2001.
26. M. van Breukelen, R.P.W. Duin, D. Tax, and J.E. den Hartog. Combining classifiers for the recognition of handwritten digits. In Ist IAPR TC1 Workshop on Statistical Techniques in Pattern Recognition, pp. 13-18, Prague, Czech Republic, 1997.
27. G.J. Briem, J.A. Benediktsson, and J.R. Sveinsson. Boosting. Bagging and consensus based classification of multisource remote sensing data. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 279-288, Springer-Verlag, 2001.
28. G. Brown, J. Wyatt, R. Harris, and X. Yao. Diversity creation methods: A survey and categorisation. Information Fusion, 6(1):5-20, 2005.
29. R. Bryll, R. Gutierrez-Osuna, and F. Quek. Attribute bagging: Improving accuracy od classifier ensembles by uing random feature subsets. Pattern Recognition, 36:1291-1302, 2003.
30. R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes. Ensemble selction from libraries of models. In 21th International Conference on Machine Learning, ICML 2004, pp. 18, ACM Press, 2004.
31. N. Cesa-Bianchi, C. Gentile, and L. Zaniboni. Hierarchical classification: Combining Bayes with SVM. In Proceedings of the 23rd International Conference on Machine Learning, pp. 177-184, ACM Press, 2006.
32. N. Cesa-Bianchi, M. Re, and G. Valentini. Functional inference in FunCat through the combination of hierarchical ensembles with data fusion methods. In ICML-MLD 2nd International Workshop on Learning from Multi-Label Data, pp. 13-20, Haifa, Israel, 2010.
33. N. Cesa-Bianchi and G. Valentini. Hierarchical cost-sensitive algorithms for genome-wide gene function prediction. Journal of Machine Learning Research, WandC Proceedings, Machine Learning in Systems Biology, 8:14-29, 2010.
34. P. Chan and S. Stolfo. A comparative evaluation of voting and meta-learning on partitioned data. In Proceedings 12th ICML, pp. 90-98, Tahoe City, California, USA, 1995.
35. P. Chan and S. Stolfo. On the accuracy of meta-learning for scalable data mining. Journal of Intelligent Information Systems, 8:5-28, 1997.
36. N.V. Chawla, L.O. Hall, K.W. Bowyer, and W.P. Kegelmeyer. Learning ensembles from bites:A scalable and accurate approach. Journal of Machine Learning Research, 5:421-451, 2004.
37. N.V. Chawla, L.O. Hall, K.W. Bowyer, T.E. Moore, and W.P. Kegelmeyer. Distributed pasting of small votes. In Multiple Classifier Systems. Third International Workshop, MCS2002, Cagliari, Italy, volume 2364 of Lecture Notes in Computer Science, pp. 52-61, Springer-Verlag, 2002.
38. D. Chen. Statistical estimates for Kleinberg’s method of stochastic discrimination. PhD thesis, The State University of New York, Buffalo, USA, 1998.
39. S. Cho and J. Kim. Combining multiple neural networks by fuzzy integral and robust classification. IEEE Transactions on Systems, Man and Cybernetics, 25:380-384, 1995.
40. S. Cho and J. Kim. Multiple network fusion using fuzzy logic. IEEE Transactions on Neural Networks, 6:497-501, 1995.
41. S. Cohen and N. Intrator. A hybrid projection based and radial basis function architecture. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 147-156, Springer-Verlag, 2000.
42. S. Cohen and N. Intrator. Automatic model selection in a hybrid perceptron/radial network. In Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 349-358, Springer-Verlag, 2001.
43. M. Collins, R.E. Schapire, and Y. Singer. Logistic regression, AdaBoost and Bregman distances. Machine Learning, 48:31-44, 2002.
44. K. Crammer and Y. Singer. On the learnability and design of output codes for multiclass problems. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, pp. 35-46, Palo Alto, California, USA, 2000.
45. N.C. de Condorcet. Essai sur l’ application de l’ analyse à la probabilité des decisions rendues à la pluralité des voix. Imprimerie Royale, Paris, 1785.
46. O. Dekel, J. Keshet, and Y. Singer. Large margin hierarchical classification. In Proceedings of the 21st International Conference on Machine Learning, pp. 209-216, Omnipress, 2004.
47. C. Dietrich, G. Palm, and F. Schwenker. Decision templates for the classification of bioacoustic time series. Information Fusion, 4(2):101-109, 2003.
48. T.G. Dietterich. Ensemble methods in machine learning. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 1-15, Springer-Verlag, 2000.
49. T.G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Machine Learning, 40(2):139-158, 2000.
50. T.G. Dietterich and G. Bakiri. Error-correcting output codes:A general method for improving multiclass inductive learning programs. In Proceedings of AAAI-91, pp. 572-577, AAAI Press/MIT Press, 1991.
51. T.G. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, (2):263-286, 1995.
52. P. Domingos. A unified bias-variance decomposition and its applications. In Proceedings of the Seventeenth International Conference on Machine Learning, pp. 231-238, Morgan Kaufmann Stanford, CA, 2000.
53. P. Domingos. A unified bias-variance decomposition for zero-one and squared loss. In Proceedings of the Seventeenth National Conference on Artificial Intelligence, pp. 564-569, Austin, TX, 2000. AAAI Press.
54. P. Domingos and M. Pazzani. On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning, 29:103-130, 1997.
55. H. Drucker and C. Cortes. Boosting decision trees. In D. Touretsky, M. Mozer, and M. Hasselmo (eds), Advances in Neural Information Processing Systems, Vol. 8, pp. 479-485. MIT Press, Cambridge, MA, 1996.
56. H. Drucker, C. Cortes, L. Jackel, Y. LeCun, and V. Vapnik. Boosting and other ensemble methods. Neural Computation, 6(6):1289-1301, 1994.
57. R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification, 2nd edition Wiley and Sons, New York, 2001.
58. R. Duin. The combining classifier: To train or not to train? In Proceedings of the 16th International Conference on Pattern Recognition, ICPR’02, pp. 765-770, Canada, 2002.
59. R.P.W. Duin and D.M.J. Tax. Experiments with classifier combination rules. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 16-29, Springer-Verlag, 2000.
60. S. Dzeroski and B. Zenko. Is combining classifiers with stacking better than selcting the best one? Machine Learning, 54(3):255-273, 2004.
61. B. Efron and R. Tibshirani. An Introduction to the Bootstrap. Chapman and Hall, New York, 1993.
62. N. El Gayar, F. Roli, and Kittler. Multiple Classifier Systems, 9th International Workshop, MCS2010, volume 5997 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 2010.
63. S. Escalera, O. Pujol, and P. Radeva. On the decoding process in ternary error-correcting output codes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1):120-134, 2010.
64. T. Evgeniou, M. Pontil, and A. Elisseeff. Leave one out error, stability, and generalization of voting combinations of classifiers. Machine Learning, 55(1):71-97, 2004.
65. E.D. Feigelson, G. Jogesh Babu, L. Breiman, M. Last, and J. Rice. Random forests: Finding quasars. In E.D. Feigelson and G. Jogesh Babu (eds), Statistical Challenges in Astronomy, pp. 243-254, Springer, New York, 2003.
66. E. Filippi, M. Costa, and E. Pasero. Multi-layer perceptron ensembles for increased performance and fault-tolerance in pattern recognition tasks. In IEEE International Conference on Neural Networks, pp. 2901-2906, Orlando, Florida, 1994.
67. Y. Freund. Boosting a weak learning algorithm by majority. Information and Computation, 121(2):256-285, 1995.
68. Y. Freund and R. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and Systems Sciences, 55(1):119-139, 1997.
69. Y. Freund and R.E. Schapire. Experiments with a new boosting algorithm. In Proceedings of the 13th International Conference onMachine Learning, pp. 148-156, Morgan Kauffman, 1996.
70. J. Friedman and P. Hall. On bagging and nonlinear estimation. Technical Report Technical Report, Statistics Department, University of Stanford, CA, 2000.
71. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Annals of Statistics, 38(2):337-374, 2000.
72. J.H. Friedman. On bias, variance, 0/1 loss and the curse of dimensionality. Data Mining and Knowledge Discovery, 1:55-77, 1997.
73. G. Fumera and F. Roli. Atheoretical and experimental analysis of linear combiners for multiple classifer systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6):942-956, 2005.
74. J. Gamma and P. Brazdil. Cascade generalization. Machine Learning, 41(3):315-343, 2000.
75. Y. Zhang, D. Gao and Y. Zhao. Random forest algorithm for classification of multiwavelength data. Research in Astronomy Astrophysics, 9(2):220-226, 2009.
76. S. Geman, E. Bienenstock, and R. Doursat. Neural networks and the bias-variance dilemma. Neural Computation, 4(1):1-58, 1992.
77. R. Ghaemi, M. Sulaiman, H. Ibrahim, and N. Mustapha. A survey: clustering ensembles techniques. InWorld Academy of Science, Engineering and Technology 50, pp. 636-645, 2009.
78. R. Ghani. Using error correcting output codes for text classification. In ICML2000: Proceedings of the 17th International Conference on Machine Learning, pp. 303-310, Morgan Kaufmann Publishers, San Francisco, US, 2000.
79. S. Ghosh, T.F. Stepinski, and R. Vilalta. Automatic annotation of planetary surfaces with geomorphic labels. IEEE Transactions on Geoscience and Remote Sensing, 48(1):175-185, 2010.
80. G. Giacinto and F. Roli. Dynamic classifier fusion. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 177-189, Springer-Verlag, 2000.
81. G. Giacinto and F. Roli. An approach to the automatic design of multiple classifier systems. Pattern Recognition Letters, 22(1):25-33, 2001.
82. G. Giacinto, F. Roli, and G. Fumera. Design of effectivemultiple classifier systems by clustering of classifiers. In 15th International Conference on Pattern Recognition ICPR 2000, pp. 160-163, Barcelona, Spain, 2000.
83. Y. Guan, C.L. Myers, D.C. Hess, Z. Barutcuoglu, A. Caudy, and O.G. Troyanskaya. Predicting gene function in a hierarchical context with an ensemble of classifiers. Genome Biology, 9(suppl 1):S3, 2008.
84. S. Gunter and H. Bunke. Feature selection algorithms for the generation of multiple classifier systems and their application to handwritten word recognition. Pattern Recognition Letters, 25:1323-1336, 2004.
85. M. Haindl, F. Roli, and Kittler. Multiple Classifier Systems, 7th International Workshop, MCS2007, volume 4472 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 2007.
86. T. Hastie and R. Tibshirani. Classification by pairwise coupling. Annals of Statistics, 26(1):451-471, 1998.
87. T.K. Ho. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8):832-844, 1998.
88. T.K. Ho. Complexity of classification problems ans comparative advantages of combined classifiers. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 97-106, Springer-Verlag, 2000.
89. T.K. Ho. Data complexity analysis for classifiers combination. In J. Kittler and F. Roli, (eds), Multiple Classifier Systems. Second International Workshop, MCS2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 53-67, Springer-Verlag, Berlin, 2001.
90. T.K. Ho. Multiple classifier combinations: Lessons and the next steps. In A. Kandel and K. Bunke, (eds), Hybrid Methods in Pattern Recognition, pp. 171-198, World Scientific, Hackensack, NJ, USA, 2002.
91. T.K. Ho, J.J. Hull, and S.N. Srihari. Decision combination in multiple classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):405-410, 1997.
92. S.M. Hojnacki, J.H. Kastner, G. Micela, E.D. Feigelson, and S.M. LaLonde. An x-ray spectral classification algorithm with application to young stellar clusters. Astrophysical Journal, 659:659, 2007.
93. S. Hojnacki, G. Micela, S. Lalonde, E. Feigelson, and J. Kastner. An unsupervised, ensemble clustering algorithm: A new approach for classification of x-ray sources. Statistical Methodology, 5:350-360, 2008.
94. K. Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 4:251-257, 1991.
95. T. Hothorn and B. Lausen. Bundling classifers by bagging trees. Computational Statistics and Data Analysis, 49:1068-1078, 2005.
96. Y.S. Huang and C.Y. Suen. Combination of multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17:90-94, 1995.
97. R.A. Jacobs. Methods for combining experts probability assessment. Neural Computation, 7:867-888, 1995.
98. R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):125-130, 1991.
99. A. Jain, R. Duin, and J. Mao. Statistical pattern recognition: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22:4-37, 2000.
100. X. Jiang, N. Nariai, M. Steffen, S. Kasif, and E. Kolaczyk. Integration of relational and hierarchical network information for protein function prediction. BMC Bioinformatics, 9:350, 2008.
101. M. Jordan and R. Jacobs. Hierarchies of adaptive experts. In J. Moody, S. Hanson, and R. Lippmann (eds), Advances in Neural Information Processing Systems, Vol. 4, pp. 985-992, Morgan Kauffman, San Mateo, CA, 1992.
102. M.I. Jordan and R.A. Jacobs. Hierarchical mixture of experts and the em algorithm. Neural Computation, 6:181-214, 1994.
103. M.I. Jordan and L. Xu. Convergence results for the EM approach to mixture of experts architectures. Neural Networks, 8:1409-1431, 1995.
104. K. Tumer and C.N. Oza. Input decimated ensembles. Pattern Analysis and Applications, 6:65-77, 2003.
105. J.M. Keller, P. Gader, H. Tahani, J. Chiang, and M. Mohamed. Advances in fuzzy integration for pattern recognition. Fuzzy Sets and Systems, 65:273-283, 1994.
106. F. Kimura and M. Shridar. Handwritten numerical recognition based on multiple algorithms. Pattern Recognition, 24(10):969-983, 1991.
107. J. Kittler. Combining classifiers: A theoretical framework. Pattern Analysis and Applications, 1:18-27, 1998.
108. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(3):226-239, 1998.
109. E.M. Kleinberg. Stochastic discrimination. Annals of Mathematics and Artificial Intelligence, 1:207-239, 1990.
110. E.M. Kleinberg. An overtraining-resistant stochastic modeling method for pattern recognition. Annals of Statistics, 4(6):2319-2349, 1996.
111. E.M. Kleinberg. A mathematically rigorous foundation for supervised learning. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 67-76, Springer-Verlag, 2000.
112. E.M. Kleinberg. On the algorithmic implementation of stochastic discrimination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(5):473-490, 2000.
113. E. Kong and T.G. Dietterich. Error-correcting output coding correct bias and variance. In The XII International Conference on Machine Learning, pp. 313-321, Morgan Kauffman, San Francisco, CA, 1995.
114. L.I. Kuncheva. An application of OWA operators to the aggregation of multiple classification decisions. In The OrderedWeightedAveraging Operators. Theory and Applications, pp. 330-343, Kluwer Academic Publisher, USA, 1997.
115. L.I. Kuncheva. Switching between selection and fusion in combining classifiers: An experiment. IEEE Transactions on Systems, Man and Cybernetics, 32(2):146-156, 2002.
116. L.I. Kuncheva. Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience, New York, 2004.
117. L.I. Kuncheva, J.C. Bezdek, and R.P.W. Duin. Decision templates formultiple classifier fusion:An experimental comparison. Pattern Recognition, 34(2):299-314, 2001.
118. L.I. Kuncheva and L.C. Jain. Designing classifier fusion systems by genetic algorithms. IEEE Trnsactions on Evolutionary Computation, 4(4):327-336, 2000.
119. L.I. Kuncheva and J. Rodriguez. An experimental study on rotation forest ensembles. In M. Haindl, J. Kittler, and F. Roli (eds), Multiple Classifier Systems. Seventh International Workshop, MCS 2007, Prague, Czech Republic, volume 4472 of Lecture Notes in Computer Science, pp. 459-468, Springer, 2007.
120. L.I. Kuncheva, F. Roli, G.L. Marcialis, and C.A. Shipp. Complexity of data subsets generated by the random subspace method: An experimental investigation. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 349-358, Springer-Verlag, 2001.
121. L.I. Kuncheva and C.J. Whitaker. Measures of diversity in classifier ensembles. Machine Learning, 51:181-207, 2003.
122. L. Lam. Classifier combinations: Implementations and theoretical issues. InMultiple Classifier Systems. First International Workshop, MCS2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 77-86, Springer-Verlag, 2000.
123. L. Lam and C. Sue. Optimal combination of pattern classifiers. Pattern Recognition Letters, 16:945-954, 1995.
124. L. Lam and C. Sue. Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE Transactions on Systems, Man and Cybernetics, 27(5):553-568, 1997.
125. W.B. Langdon and B.F. Buxton. Genetic programming for improved receiver operating characteristics. In J. Kittler and F. Roli (eds), Second International Conference onMultiple Classifier System, volume 2096 of LNCS, pp. 68-77, Springer-Verlag, Cambridge, 2001.
126. H. Lapplainen. Nonlinear independent component analysis using ensemble learning: Theory. In Proceedings of the 1st International Workshop on Independent Component Analysis and Blind Signal Separation, p. 7, 1998.
127. A. Lazarevic and Z. Obradovic. Effective pruning of neural network classifiers. In Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN’01, pp. 796-801, IEEE, Washington, DC, USA, 2001.
128. W. Leigh, R. Purvis, and J.M. Ragusa. Forecasting the NYSE composite index with technical analysis, pattern recognizer, neural networks and genetic algorithm: A case study. Decision Support Systems, 32(4):361-377, 2002.
129. M. Li and P. Vitanyi. An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, Berlin, 1993.
130. R. Liu and B. Yuan. Multiple classifier combination by clustering and selection. Information Fusion, 2:163-168, 2001.
131. H. Lu, H. Zhou, J. Wang, T. Wang, X. Dong, Z. Zhuang, and C. Li. Ensemble learning for independent component analysis of normal galaxy spectra. Astronomical Journal, 131:790-805, 2006.
132. G. Martinez-Muniz and A. Suarez. Pruning in ordered bagging ensembles. In 23th International Conference on Machine Learning, ICML 2006, pp. 609-616, ACM Press, 2006.
133. L. Mason, P. Bartlett, and J. Baxter. Improved generalization through explicit optimization of margins. Machine Learning, 38(3):243-255, 2000.
134. F. Masulli and G. Valentini. Effectiveness of error correcting output codes in multiclass learning problems. In Lecture Notes in Computer Science, volume 1857, pp. 107-116, Springer-Verlag, Berlin, Heidelberg, 2000.
135. F. Masulli and G. Valentini. Quantitative evaluation of dependence among outputs in ECOC classifiers using mutual information based measures. In K. Marko and P. Webos (eds), Proceedings of the International Joint Conference on Neural Networks IJCNN’01, volume 2, pp. 784-789, Piscataway, NJ, USA, IEEE, 2001.
136. F. Masulli and G. Valentini. Effectiveness of error correcting output coding decomposition schemes in ensemble and monolithic learning machines. Pattern Analysis and Application, 6:285-300, 2003.
137. F. Masulli and G. Valentini. An experimental analysis of the dependence among codeword bit errors in ecoc learning machines. Neurocomputing, 57:189-214, 2004.
138. E. Mayoraz and M. Moreira. On the decomposition of polychotomies into dichotomies. In The XIV International Conference onMachine Learning, pp. 219-226, Nashville, TN, July 1997.
139. E. Menahem, A. Shabtai, L. Rokach, Y. Elovici, and A. Troiha. Improving malware detection by applying multi-iducer ensemble. Computational Statistics and Data Analysis, 53(4):1483-1494, 2009.
140. C. Merkwirth, H. Mauser, T. Schulz-Gasch, O. Roche, M. Stahl, and T. Lengauer. Ensemble methods for classification in cheminformatics. Journal of Chemical Information and Modeling, 44(6):1971-1978, 2009.
141. M. Moreira and E. Mayoraz. Improved pairwise coupling classifiers with correcting classifiers. In C. Nedellec and C. Rouveirol (eds), Lecture Notes in Artificial Intelligence, Volume 1398, pp. 160-171, Berlin, Heidelberg, New York, 1998.
142. G. Obozinski, G. Lanckriet, C. Grant, M. Jordan, and W.S. Noble. Consistent probabilistic output for protein function prediction. Genome Biology, 9(supp. 1), 2008.
143. O. Okun and G. Valentini (eds.). Supervised and Unsupervised Ensemble Methods and Their Applications, volume 126 of Studies in Computational Intelligence. Springer-Verlag, Berlin, 2008.
144. O. Okun and G. Valentini (eds.). Applications of Supervised and Unsupervised Ensemble Methods, volume 245 of Studies in Computational Intelligence. Springer-Verlag, Berlin, 2009.
145. D.W. Opitz. Feature selection for ensembles. In Proceedings of the 16th National Conference on Artificla Intelligence, AAAI, pp. 379-384, 1999.
146. D.W. Opitz and J.W. Shavlik. Actively searching for an effective neural network ensemble. Connection Science, 8(3/4):337-353, 1996.
147. D.W. Opitz and J.W. Shavlik. Generating accurate and diverse members of a neural-network ensemble. In D. Touretzky, M. Mozer, and M. Hasselmo (eds), Advances in Neural Information Processing Systems, volume 8, pp. 535-541, MIT Press, Cambridge, MA, 1996.
148. N.C. Oza. Aveboost2: Boosting for noisy data. InMultiple Classifier Systems. Fifth International Workshop, MCS 2004, Cagliari, Italy, volume 3077 of Lecture Notes in Computer Science, pp. 31-40, Springer-Verlag, 2004.
149. N.C. Oza, R. Polikar, F. Roli, and Kittler. Multiple Classifier Systems, 6th International Workshop, MCS2005, volume 3541 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 2005.
150. N.C. Oza and K. Tumer. Input decimation ensembles: Decorrelation through dimensionality reduction. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 238-247, Springer-Verlag, 2001.
151. J. Park and I.W. Sandberg. Approximation and radial basis function networks. Neural Computation, 5(2):305-316, 1993.
152. B. Parmanto, P. Munro, and H. Doyle. Improving committe diagnosis with resampling techniques. In D.S. Touretzky, M. Mozer, and M. Hesselmo (eds), Advances in Neural Information Processing Systems, volume 8, pp. 882-888, MIT Press, Cambridge, MA, 1996.
153. B. Parmanto, P. Munro, and H. Doyle. Reducing variance of committee predition with resampling techniques. Connection Science, 8(3/4):405-416, 1996.
154. I. Partalas, G. Tsoumakas, and I. Vlahavas. Focused ensemble selection: A diversity-based method for greedy ensemble selection. In Proceeding of the 2008 Conference on ECAI 2008:18th European Conference on Artificial Intelligence, pp. 117-121, IOS-Press, 2008.
155. I. Partalas, G. Tsoumakas, and I. Vlahavas. An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Machine Learning, 81(3):257-282, 2010.
156. D. Partridge, and W.B Yates. Engineering multiversion neural-net systems. Neural Computation, 8:869-893, 1996.
157. E. Pekalska, M. Skurichina, and R.P.W. Duin. Combining Fisher linear discriminant for dissimilarity representations. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. First International Workshop, MCS 2000, Cagliari, Italy, volume 1857 of Lecture Notes in Computer Science, pp. 230-239, Springer-Verlag, 2000.
158. M.P. Perrone and L.N. Cooper. When networks disagree: Ensemble methods for hybrid neural networks. In R.J. Mammone (ed.), Artificial Neural Networks for Speech and Vision, pp. 126-142, Chapman and Hall, London, 1993.
159. R. Polikar, J. DePasquale, H. Syed Mohammed, G. Brown, and L.I. Kuncheva. Learn++.mf:A random subspace approach for the missing feature problem. Pattern Recognition, 43:3817-3832, 2010.
160. R. Polikar, A. Topalis, D. Parikh, D. Green, J. Frymiare, J. Kounios, and C. Clark. An ensemble based data fusion approach for early diagnosis of alzheimer disease. Information Fusion, 9:83-95, 2008.
161. O. Pujol, P. Radeva, and J. Vitria. Discriminant ECOC: A heuristic method for application dependent design of error correcting output codes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28:1001-1007, 2006.
162. J.R. Quinlan. C4.5 Programs for Machine Learning. Morgan Kauffman, London, UK, 1993.
163. Y. Raviv and N. Intrator. Bootstrapping with noise: An effective regularization technique. Connection Science, 8(3/4):355-372, 1996.
164. M. Re and G. Valentini. Integration of heterogeneous data sources for gene function prediction using decision templates and ensembles of learning machines. Neurocomputing, 73(7-9):1533-1537, 2010.
165. M. Re and G. Valentini. Noise tolerance of multiple classifier systems in data integration-based gene function prediction. Journal of Integrative Bioinformatics, 7(3):139, 2010.
166. M. Re and G. Valentini. Simple ensemble methods are competitive with state-of-the-art data integration methods for gene function prediction. Journal of Machine Learning Research, WandC Proceedings, Machine Learning in Systems Biology, 8:98-111, 2010.
167. S. Reid and G. Grudic. Regularized linear models in stacked generalization. In J. Kittler, J. Benediktsson, and F. Roli, (eds), Multiple Classifier Systems. Eighth International Workshop, MCS 2009, Reykjavik, Iceland, volume 5519 of Lecture Notes in Computer Science, pp. 112-121, Springer, 2009.
168. J. Rodriguez and L.I. Kuncheva. Rotation forest: A new classifier ensemble method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(10):1619-1630, 2006.
169. G. Rogova. Combining the results of several neural neetworks classifiers. Neural Networks, 7:777-781, 1994.
170. L. Rokach. Genetic algorithm-based feature set partitioning for classifiaction problems. Pattern Recognition, 41(5):1676-1700, 2008.
171. L. Rokach. Taxonomy for characterizing ensemble methods in classification asks: A reveiw and annotated bibliography. Computational Statistics and Data Analysis, 53:4046-4072, 2009.
172. F. Roli, G. Giacinto, and G. Vernazza. Methods for designing multiple classifier systems. In J. Kittler and F. Roli (eds), Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 78-87, Springer-Verlag, 2001.
173. F. Roli, J. Kittler, and T. Windeatt. Multiple Classifier Systems, Fifth International Workshop, MCS2004, volume 3077 of Lecture Notes in Computer Science. Springer-Verlag, Berlin, 2004.
174. R. Schapire and Y. Singer. Boostexter: A boosting-based system for text categorization. Machine Learning, 39(2/3):135-168, 2000.
175. R.E. Schapire. The strenght of weak learnability. Machine Learning, 5(2):197-227, 1990.
176. R.E. Schapire. A brief introduction to boosting. In T. Dean (ed.), 16th International Joint Conference on Artificial Intelligence, pp. 1401-1406, Morgan Kauffman, 1999.
177. R.E. Schapire, Y. Freund, P. Bartlett, and W. Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5):1651-1686, 1998.
178. R.E. Schapire and Y. Singer. Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3):297-336, 1999.
179. L. Schietgat, C. Vens, J. Struyf, H. Blockeel, and S. Dzeroski. Predicting gene function using hierarchical multi-label decision tree ensembles. BMC Bioinformatics, 11:2, 2010.
180. B. Scholkopf and A. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
181. B. Settles. Active learning literature survey. Technical Report Computer Sciences Technical Report 1648, University of Wisconsin, Madison, 2010.
182. A. Sharkey. Types of multi-net systems. In F. Roli and J. Kittler (eds), Multiple Classifier Systems, Third International Workshop, MCS2002, volume 2364 of Lecture Notes in Computer Science, pp. 108-117, Springer-Verlag, 2002.
183. A. Sharkey, N. Sharkey, and G. Chandroth. Diverse neural net solutions to a fault diagnosis problem. Neural Computing and Applications, 4:218-227, 1996.
184. A. Sharkey (ed.). Combining Artificial Neural Nets: Ensemble and ModularMulti-Net Systems. Springer-Verlag, London, 1999.
185. M. Skurichina and R.P.W. Duin. Bagging for linear classifiers. Pattern Recognition, 31(7):909-930, 1998.
186. M. Skurichina and R.P.W. Duin. Bagging and the random subspace method for redundant feature spaces. In Multiple Classifier Systems. Second International Workshop, MCS 2001, Cambridge, UK, volume 2096 of Lecture Notes in Computer Science, pp. 1-10, Springer-Verlag, 2001.
187. M. Skurichina and R.P.W. Duin. Bagging, boosting and the random subspace method for linear classifiers. Pattern Analysis and Applications, 5(2):121-135, 2002.
188. S.Y. Sohna and H.W. Shinb. Experimental study for the comparison of classifier combination methods. Pattern Recognition, 40:33-40, 2007.
189. Y. Sun, S. Todorovic, and L. Li. Reducing the overfitting of adaboost by controlling its data distribution skewness. International Journal of Pattern Recognition and Artificial Intelligence, 20(7):1093-1116, 2006.
190. D. Tao, X. Tang, X. Li, and X. Wu. Asymmetric bagging and random subspace for support vector machine-based relevance feedback in image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(7):1088-1099, 2006.
191. G. Tsoumakas, L. Angelis, and I. Vlahavas. Selective fusion of heterogeneous classifiers. Intelligent Data Analysis, 9(6):511-525, 2005.
192. G. Tsoumakas, I. Katakis, and I. Vlahavas. Effective voting of heterogeneous classifiers. In Proceedings of the 15th European Conference on Machine Learning, ECML 2004, pp. 465-476, Pisa, Italy, 2004.
193. A. Tsymbal, M. Pechenizkiy, and P. Cunningham. Diversity in search strategies for ensemble feature selection. Information Fusion, 6:83-98, 2006.
194. A. Tsymbal, S. Puuronen, and D.W. Patterson. Ensemble feature selection with the simple bayesian classifiaction. Information Fusion, 4:87-100, 2003.
195. K. Tumer and J. Ghosh. Error correlation and error reduction in ensemble classifiers. Connection Science, 8(3/4):385-404, 1996.
196. G. Valentini. Gene expression data analysis of human lymphoma using support vector machines and output coding ensembles. Artificial Intelligence in Medicine, 26(3):283-306, 2002.
197. G. Valentini. An experimental bias-variance analysis of SVM ensembles based on resampling techniques. IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, 35(6):1252-1271, 2005.
198. G. Valentini. True path rule hierarchical ensembles for genome-wide gene function prediction. IEEE ACM Transactions on Computational Biology and Bioinformatics, 8(3), 2011.
199. G. Valentini and T.G. Dietterich. Low bias bagged support vector machines. In T. Fawcett and N. Mishra (eds), Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), pp. 752-759, AAAI Press, Washington D.C., USA, 2003.
200. G. Valentini and T.G. Dietterich. Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods. Journal ofMachine Learning Research, 5:725-775, 2004.
201. G. Valentini and F. Masulli. Ensembles of learning machines. InNeural NetsWIRN-02, volume 2486 of Lecture Notes in Computer Science, pp. 3-19, Springer-Verlag, 2002.
202. J. Van Lint. Coding Theory. Springer Verlag, Berlin, 1971.
203. V. N. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
204. C. Vens, J. Struyf, L. Schietgat, S. Dzeroski, and H. Blockeel. Decision trees for hierarchical multi-label classification. Machine Learning, 73(2):185-214, 2008.
205. A. Verikas, A. Lipnickas, K. Malmqvist, M. Bacauskiene, and A. Gelzinis. Soft combination of neural classifiers: A comparative study. Pattern Recognition Letters, 20:429-444, 1999.
206. K.L. Wagstaff and V.G. Laidler. Making the most of missing values: Object clustering with partial data in astronomy. In Astronomical Data Analysis Software and Systems XIV, ASP Conference Series, Vol. 347, Proceedings of the Conference held 24-27 October, 2004 in Pasadena, California, USA, p. 172, 2005.
207. D. Wang, J.M. Keller, C.A. Carson, K.K. McAdoo-Edwards, and C.W. Bailey. Use of fuzzy logic inspired features to improve bacterial recognition through classifier fusion. IEEE Transactions on Systems, Man and Cybernetics, 28B(4):583-591, 1998.
208. D.H. Wolpert. Stacked generalization. Neural Networks, 5:241-259, 1992.
209. K. Woods, W.P. Kegelmeyer, and K. Bowyer. Combination of multiple classifiers using local accuracy estimates. IEEE Transactions on PatternAnalysis and Machine Intelligence, 19(4):405-410, 1997.
210. L. Xu, C. Krzyzak, and C. Suen. Methods of combining multiple classifiers and their applications to handwritting recognition. IEEE Transactions on Systems, Man and Cybernetics, 22(3):418-435, 1992.
211. C. Yeang, S. Ramaswamy, P. Tamayo, S. Mukherjee, R.M. Rifkin, M. Angelo, M. Reich, E. Lander, J. Mesirov, and T. Golub. Molecular classification of multiple tumor types. In ISMB 2001, Proceedings of the 9th International Conference on Intelligent Systems for Molecular Biology, pp. 316-322, Oxford University Press, Copenaghen, Denmark, 2001.
212. C.X. Zhang and J.S. Zhang. A local boosting algorithm for solving classification problems. Computational Statistics and Data Analysis, 52(4):1928-1941, 2008.
213. Y. Zhang, S. Burer, and W.N. Street. Ensemble pruning via semi-definite programming. Journal of Machine Learning Research, 7:1315-1338, 2006.
214. J. Zhou, H. Peng, and C. Suen. Data-driven decompositon for multi-class classification. Pattern Recognition, 41(1):67-76, 2008.
215. X. Zhu and A. Goldberg. Introduction to Semi-Supervised Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan and Claypool, 2009.