Popular Ensemble Methods: An Empirical Study

Journal of Artificial Intelligence Research

Volumn 11, Issue , 1999, Pages 169-198

  Opitz, David ^a   Maclin, Richard ^b  

^a UNIVERSITY OF MONTANA   (United States)

^b UNIVERSITY OF MINNESOTA DULUTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000551189     PISSN: 10769757     EISSN: None     Source Type: Journal    
DOI: 10.1613/jair.614     Document Type: Article

References (49)

1. Ali, K., & Pazzani, M. (1996). Error reduction through learning multiple descriptions. Machine Learning, 24, 173-202.
2. Alpaydin, E. (1993). Multiple networks for function learning. In Proceedings of the 1993 IEEE International Conference on Neural Networks, Vol. I, pp. 27-32 San Francisco.
3. Arbib, M. (Ed.). (1995). The Handbook of Brain Theory and Neural Networks. MIT Press.
4. Asker, L., & Maclin, R. (1997a). Ensembles as a sequence of classifiers. In Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 860-865 Nagoya, Japan.
5. Asker, L., & Maclin, R. (1997b). Feature engineering and classifier selection: A case study in Venusian volcano detection. In Proceedings of the Fourteenth International Conference on Machine Learning, pp. 3-11 Nashville, TN.
6. Bauer, E., & Kohavi, R. (1999). An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 36, 105-139.
7. Baxt, W. (1992). Improving the accuracy of an artificial neural network using multiple differently trained networks. Neural Computation, 4, 772-780.
8. Breiman, L. (1996a). Bagging predictors. Machine Learning, 24(2), 123-140.
9. Breiman, L. (1996b). Bias, variance, and arcing classifiers. Tech. rep. 460, UC-Berkeley, Berkeley, CA.
10. Breiman, L. (1996c). Stacked regressions. Machine Learning, 24(1), 49-64.
11. Clemen, R. (1989). Combining forecasts: A review and annotated bibliography. Journal of Forecasting, 5, 559-583.
12. Drucker, H., & Cortes, C. (1996). Boosting decision trees. In Touretsky, D., Mozer, M., & Hasselmo, M. (Eds.), Advances in Neural Information Processing Systems, Vol. 8, pp. 479-485 Cambridge, MA. MIT Press.
13. Drucker, H., Cortes, C., Jackel, L., LeCun, Y., & Vapnik, V. (1994). Boosting and other machine learning algorithms. In Proceedings of the Eleventh International Conference on Machine Learning, pp. 53-61 New Brunswick, NJ.
14. Efron, B., & Tibshirani, R. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York.
15. Fisher, D., & McKusick, K. (1989). An empirical comparison of ID3 and back-propagation. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 788-793 Detroit, MI.
16. Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. In Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148-156 Bari, Italy.
17. Friedman, J. (1996). On bias, variance, 0/1-loss, and the curse-of-dimensionality. Journal of Data Mining and Knowledge Discovery, 1.
18. Friedman, J., Hastie, T., & Tibshirani, R. (1998). Additive logistic regression: A statistical view of boosting, (http://www-stat.stanford.edu/~jhf).
19. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4, 1-58.
20. Granger, C. (1989). Combining forecasts: Twenty years later. Journal of Forecasting, 8, 167-173.
21. Grove, A., & Schuurmans, D. (1998). Boosting in the limit: Maximizing the margin of learned ensembles. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 692-699 Madison, WI.
22. Hampshire, J., & Waibel, A. (1989). The meta-pi network: Building distributed knowledge representations for robust pattern recognition. Tech. rep. CMU-CS-89-166, CMU, Pittsburgh, PA.
23. Hansen, L., & Salamon, P. (1990). Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 993-1001.
24. Hashem, S. (1997). Optimal linear combinations of neural networks. Neural Networks, 10(4), 599-614.
25. Jacobs, R., Jordan, M., Nowlan, S., & Hinton, G. (1991). Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
26. Kohavi, R., & Wolpert, D. (1996). Bias plus variance decomposition for zero-one loss functions. In Proceedings of the Thirteenth International Conference on Machine Learning, pp. 275-283 Bari, Italy.
27. Kong, E., & Dietterich, T. (1995). Error-correcting output coding corrects bias and variance. In Proceedings of the Twelfth International Conference on Machine Learning, pp. 313-321 Tahoe City, CA.
28. Krogh, A., & Vedelsby, J. (1995). Neural network ensembles, cross validation, and active learning. In Tesauro, G., Touretzky, D., & Leen, T. (Eds.), Advances in Neural Information Processing Systems, Vol. 7, pp. 231-238 Cambridge, MA. MIT Press.
29. Lincoln, W., & Skrzypek, J. (1989). Synergy of clustering multiple back propagation networks. In Touretzky, D. (Ed.), Advances in Neural Information Processing Systems, Vol. 2, pp. 650-659 San Mateo, CA. Morgan Kaufmann.
30. Maclin, R. (1998). Boosting classifiers regionally. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 700-705 Madison, WI.
31. Maclin, R., & Opitz, D. (1997). An empirical evaluation of bagging and boosting. In Proceedings of the Fourteenth National Conference on Artificial Intelligence, pp. 546-551 Providence, RI.
32. Maclin, R., & Shavlik, J. (1995). Combining the predictions of multiple classifiers: Using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pp. 524-530 Montreal, Canada.
33. Mani, G. (1991). Lowering variance of decisions by using artificial neural network portfolios. Neural Computation, 3, 484-486.
34. Mooney, R., Shavlik, J., Towell, G., & Gove, A. (1989). An experimental comparison of symbolic and connectionist learning algorithms. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 775-780 Detroit, MI.
35. Murphy, P. M., & Aha, D. W. (1994). UCI repository of machine learning databases (machine-readable data repository). University of California-Irvine, Department of Information and Computer Science.
36. Nowlan, S., & Sejnowski, T. (1992). Filter selection model for generating visual motion signals. In Hanson, S., Cowan, J., & Giles, C. (Eds.), Advances in Neural Information Processing Systems, Vol. 5, pp. 369-376 San Mateo, CA. Morgan Kaufmann.
37. Opitz, D., & Shavlik, J. (1996a). Actively searching for an effective neural-network ensemble. Connection Science, 8(3/4), 337-353.
38. Opitz, D., & Shavlik, J. (1996b). Generating accurate and diverse members of a neural-network ensemble. In Touretsky, D., Mozer, M., & Hasselmo, M. (Eds.), Advances in Neural Information Processing Systems, Vol. 8, pp. 535-541 Cambridge, MA. MIT Press.
39. Perrone, M. (1992). A soft-competitive splitting rule for adaptive tree-structured neural networks. In Proceedings of the International Joint Conference on Neural Networks, pp. 689-693 Baltimore, MD.
40. Perrone, M. (1993). Improving Regression Estimation: Averaging Methods for Variance Reduction with Extension to General Convex Measure Optimization. Ph.D. thesis, Brown University, Providence, RI.
41. Quinlan, J. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA.
42. Quinlan, J. R. (1996). Bagging, boosting, and c4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725-730. Portland, OR.
43. Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. In Rumelhart, D., & McClelland, J. (Eds.), Parallel Distributed Processing: Explorations in the microstructure of cognition. Volume 1: Foundations, pp. 318-363. MIT Press, Cambridge, MA.
44. Schapire, R. (1990). The strength of weak learnability. Machine Learning, 5(2), 197-227.
45. Schapire, R., Freund, Y., Bartlett, P., & Lee, W. (1997). Boosting the margin: A new explanation for the effectiveness of voting methods. In Proceedings of the Fourteenth International Conference on Machine Learning, pp. 322-330 Nashville, TN.
46. Sollich, P., & Krogh, A. (1996). Learning with ensembles: How over-fitting can be useful. In Touretsky, D., Mozer, M., & Hasselmo, M. (Eds.), Advances in Neural Information Processing Systems, Vol. 8, pp. 190-196 Cambridge, MA. MIT Press.
47. Tresp, V., & Taniguchi, M. (1995). Combining estimators using non-constant weighting functions. In Tesauro, G., Touretzky, D., & Leen, T. (Eds.), Advances in Neural Information Processing Systems, Vol. 7, pp. 419-426 Cambridge, MA. MIT Press.
48. Wolpert, D. (1992). Stacked generalization. Neural Networks, 5, 241-259.
49. Zhang, X., Mesirov, J., & Waltz, D. (1992). Hybrid system for protein secondary structure prediction. Journal of Molecular Biology, 225, 1049-1063.