Pruning extensions to stacking

Intelligent Data Analysis

Volumn 10, Issue 1, 2006, Pages 47-66

  Rooney, Niall ^a   Patterson, David ^a   Nugent, Chris ^a  

^a UNIVERSITY OF ULSTER   (United Kingdom)

Author keywords

ensemble learning; Machine learning; pruning; stacking

Indexed keywords

LEARNING SYSTEMS;

CONSTANT SIZES; ENSEMBLE LEARNING; ENSEMBLE MEMBERS; PRUNING; STACKING; STACKING ALGORITHMS; STATIC APPROACH; TRAINING ACCURACY;

PERSONNEL TRAINING;

EID: 34147111922     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/ida-2006-10104     Document Type: Article

References (36)

1. C.G. Atkeson, A.W. Moore and S. Schaal, Locally weighted learning, Artificial Intelligence Review 11 (1997), 11-73.
2. L. Breiman, Bagging predictors, Machine learning 24 (1996), 123-174.
3. L. Breiman, Stacked regressions, Machine Learning 24 (1996), 49-64.
4. L. Breiman, Randomizing outputs to increase prediction accuracy, Machine Learning 40(3) (2000), 229-242.
5. J. Carney and P. Cunningham, Tuning diversity in bagged ensembles, International Journal of Neural Systems 10(4) (2000), 267-280, World Scientific.
6. S. Christensen, Ensemble Construction via Designed Output Distortion, (Vol. 2709), In Proceedings of the 4th International Workshop on Multiple Classifier Systems, LNCS, 2003, 286-295, Springer-Verlag.
7. C. Clarke, Comparing bayes model averaging and stacking when model approximation error cannot be ignored, Journal of Machine Learning Research 4 (2003), 683-712.
8. P. Cunningham and J. Carney, Diversity versus quality in classification ensembles based on feature selection, (Vol. 1810), In Proceedings of the 11th European Conference on Machine Learning, LNCS, Springer, 2000, 109-116.
9. T. Dietterich, Approximate statistical test for comparing supervised classification learning algorithms, Neural Computation 10(7) (1998), 1895-1923.
10. T. Dietterich, Ensemble Methods in Machine Learning, (Vol 1857), In Proceedings of the 1st International Workshop on Multiple Classifer Systems. LNCS, Springer-Verlag, 2000, 1-10.
11. H. Drucker, Improving Regressors using Boosting Techniques, In Proceedings of the 14th International Conference on Machine Learning, Morgan-Kaufmann, 1997, 107-115.
12. S. Dzeroski and B. ?Senko, Is combining classifiers with stacking better than selecting the best one? Machine Learning 54(3) (2004), 255-273, Kluwer.
13. Y. Freund and R.E. Schapire, Experiments with a new Boosting algorithm, In Proceedings of the 13th International Conference on Machine Learning, Morgan-Kauffman, 1996, 231-238.
14. G. Giacinto and F. Roli, Methods for dynamic classifier selection, In Proceedings of the 10th International Conference on Image Analysis and Processing, IEEE CS Press, 1999, 659-664.
15. G. Giacinto and F. Roli, Dynamic Classifier Selection, (Vol. 1857), In Proceedings of the 1st International Workshop on Multiple Classifier Systems, LNCS, Springer-Verlag, 2000, 177-189.
16. L. Hansen and P. Salomon, Neural network ensembles, IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990), 993-1001.
17. T.K. Ho, The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8) (1998), 832-844.
18. T.K. Ho, Nearest Neighbors in Random Subspaces, In Proceedings of the Joint IAPRWorkshops on Advances in Pattern Recognition, LNCS, Springer-Verlag, 1998, 640-648.
19. A. Krogh and J. Vedelsby, Neural Networks Ensembles, Cross validation, and Active Learning, Advances in Neural Information Processing Systems, MIT Press, 1995, 231-238.
20. M. LeBlanc and R. Tibshirani, Combining estimates in Regression and Classification, Technical Report, Dept. of Statistics, University of Toronto, 1993.
21. C.J. Merz, Dynamical selection of learning algorithms, in: Learning from data, artificial intelligence and statistics, Fisher and H.-J. Lenz, eds, New York: Springer, 1996
22. C. Merz and M. Pazzani, A principal components approach to combining regression estimates, Machine Learning 36 (1999), 9-32, Kluwer.
23. M.P. Perrone and L.N. Cooper, When Networks Disagree: Ensemble Methods for hybrid neural networks, in: Neural Networks for speech and image processing, R.J. Mammone, ed., Chapman and Hall, 1993.
24. R. Quinlan, Learning with continuous classes, In Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, World Scientific, 1992, 343-348.
25. N. Rooney, D. Patterson, S. Anand and A. Tsymbal, Dynamic Integration of regression models, In Proceedings of the 5th International Multiple Classifier Systems Workshop, LNCS, Springer, 2004, 164-173.
26. N. Rooney, D. Patterson and C.D. Nugent, Reduced Ensemble size Stacking 2004, In Proceedings of 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), IEEE CS Press, 2004, 266-271.
27. A.K. Seewald, How to Make Stacking Better and Faster While Also Taking Care of an Unknown Weakness, In Proceedings of the 19th International Conference on Machine Learning, 2002.
28. R. Setiono, W.K. Leow and J. Zurada, Extraction of rules from artificial neural networks for nonlinear regression, IEEE Transactions on Neural Networks 13(3) (2002), 564-577.
29. A.J.C. Sharkey, ed., Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems, Springer-Verlag, 1999.
30. A.J. Smola and B. Scholkopf, A Tutorial on Support Vector Regression. NeuroCOLT2 Technical Report Series-NC2-TR-1998-030, 1998.
31. K.M. Ting and I.H. Witten, Issues in stacked generalization, Journal of Artificial Intelligence Research 10 (1999), 271-289.
32. A. Tsymbal, S. Puuronen and D. Patterson, Ensemble feature selection with the simple Bayesian classification, Information Fusion 4 (2003), 87-100, Elsevier.
33. I. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufmann, 1999.
34. D. Wolpert, Stacked generalization, Neural Networks 5 (1992), 241-259.
35. G. Zenobi and P. Cunningham, Using Diversity in Preparing Ensemble of Classifiers Based on Different Subsets to Minimize Generalization Error, In Proceedings of the 12th European Conference on Machine Learning, Springer-Verlag, 2001, 576-587.
36. Z. Zhou, J.Wu and W. Tang, Ensembling neural networks: many could be better than all, Artificial Intelligence 137(1-2) (2002), 239-263, Elsevier.