Actively Searching for an Effective Neural Network Ensemble

Connection Science

Volumn 8, Issue 3-4, 1996, Pages 337-354

  Opitz, David W ^a,c   Shavlik, Jude W ^b  

^a UNIVERSITY OF MINNESOTA DULUTH   (United States)

^b University of Wisconsin ^*  (United States)

^c UNIVERSITY OF MONTANA   (United States)

Author keywords

Bagging algorithm; Genetic algorithms; KBANN algorithm; Knowledge based neural networks; REGENT algorithm

Indexed keywords

EID: 0030356238     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/095400996116802     Document Type: Article

References (37)

1. Aarts, E. & Korst, J. (1989) Simulated Annealing and Bolzmann Machines. New York: Wiley.
2. Alpaydin, E. (1993) Multiple networks for function learning. In Proceedings of the 1993 IEEE International Conference on Neural Networks, San Francisco, CA, Vol. I, pp. 27-32.
3. Baxt, W. (1992) Improving the accuracy of an artificial neural network using multiple differently trained networks. Neural Computation, 4, 772-780.
4. Breiman, L. (1996a) Bagging predictors. Machine Learning, 24, 123-140.
5. Breiman, L. (1996b) Stacked regressions. Machine Learning, 24, 49-64.
6. Breiman, L., Friedman, J., Olshen, R. & Stone, C. (1984) Classification and Regression Trees. Monterey, CA: Wadsworth and Brooks.
7. Clemen, R. (1989) Combining forecasts: a review and annotated bibliography. International Journal of Forecasting, 5, 559-583.
8. Drucker, H. & Cortes, C. (1996) Boosting decision trees. In D. Touretsky, M. Mozer & M. Hasselmo (Eds), Advances in Neural Information Processing Systems, Vol. 8. Cambridge, MA: MIT Press.
9. Drucker, H., Schapire, R. & Simard, P. (1992) Improving performance in neural networks using a boosting algorithm. In J. Hanson, J. Cowan & C. Giles (Eds), Advances in Neural Information Processing Systems, Vol. 5. Palo Alto, CA: Morgan Kaufmann, pp. 42-49.
10. Drucker, H., Cortes, C., Jackel, L., LeCun, Y. & Vapnik, V. (1994) Boosting and other machine learning algorithms. In Proceedings of the Eleventh International Conference on Machine Learning. New Brunswick, NJ: Morgan Kaufmann, pp. 53-61.
11. Efron, B. & Tibshirani, R. (1993) An Introduction to the Bootstrap. New York: Chapman and Hall.
12. Freund, Y. & Shapire, R. (1995) A decision-theoretic generalization of on-line learning and an application to boosting. In Proceedings of the Second European Conference on Computational Learning.
13. Goldberg, D. (1989) Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley.
14. Granger, C. (1989) Combining forecasts: twenty years later. Journal of Forecasting, 8, 167-173.
15. Hampshire, J. & Waibel, A. (1989) The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Pattern Recognition. Technical Report TR CMU-CS-89-166, CMU, Pittsburgh, PA.
16. Hansen, L. & Salamon, P. (1990) Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 993-1001.
17. Hashem, S., Schmeiser, B. & Yih, Y. (1994) Optimal linear combinations of neural networks: an overview. In Proceedings of the 1994 IEEE International Conference on Neural Networks, Orlando, FL.
18. Holland, J. (1975) Adaptation in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press.
19. Jacobs, R., Jordan, M., Nowlan, S. & Hinton, G. (1991) Adaptive mixtures of local experts. Neural Computation, 3, 79-87.
20. Kibler, D. Langley, P. (1988) Machine learning as an experimental science. In Proceedings of the Third European Working Session on Learning, Edinburgh, UK, pp. 1-12.
21. Koza, J. (1992) Genetic Programming. Cambridge, MA: MIT Press.
22. Krogh, A. & Vedelsby, J. (1995) Neural network ensembles, cross validation, and active learning. In G. Tesauro, D. Touretzky & T. Leen (Eds), Advances in Neural Information Processing Systems, Vol. 7. Cambridge, MA: MIT Press.
23. Lincoln, W. & Skrzypek, J. (1989) Synergy of clustering multiple back propagation networks. In D. Touretzky (Ed.), Advances in Neural Information Processing Systems, Vol. 2. San Mateo, CA: Morgan Kaufmann, pp. 650-659.
24. Maclin, R. & Shavlik, J. (1995) Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, Montreal, Canada.
25. Mani, G. (1991) Lowering variance of decisions by using artificial neural network portfolios. Neural Computation, 3, 484-486.
26. Nowlan, S. & Sejnowski, T. (1992) Filter selection model for generating visual motion signals. In S. Hanson, J. Cowan & C. Giles (Eds), Advances in Neural Information Processing Systems, Vol. 5. San Mateo, CA: Morgan Kaufmann, pp. 369-376.
27. Opitz, D. (1995) An Anytime Approach to Connectionist Theory Refinement: Refining the Topologies of Knowledge-Based Neural Networks. PhD thesis, Computer Sciences Department, University of Wisconsin, Madison, WI.
28. Opitz, D. & Shavlik, J. (1993) Heuristically expanding knowledge-based neural networks. In Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, Chambery, France. San Mateo, CA: Morgan Kaufmann, pp. 1360-1365.
29. Opitz, D. & Shavlik, J. (1994) Using genetic search to refine knowledge-based neural networks. In Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick. San Mateo, CA: Morgan Kaufmann, pp. 208-216.
30. Perrone, M. (1992) A soft-competitive splitting rule for adaptive tree-structured neural networks. In Proceedings of the International Joint Conference on Neural Networks, Baltimore, MD, pp. 689-693.
31. Provost, F. & Danyluk, A. (1995) Learning from bad data. In Workshop on Applying Machine Learning in Practice, held at the Twelth International Conference on Machine Learning, Tahoe City, CA.
32. Shapire, R. (1990) The strength of weak learnability. Machine Learning, 5, 197-227.
33. Sollich, P. & Krogh, A. (1996) Learning with ensembles: how over-fitting can be useful. In D. Touretsky, M. Mozer & M. Hasselmo (Eds), Advances in Neural Information Processing Systems, Vol. 8. Cambridge, MA: MIT Press.
34. Towell, G. & Shavlik, J. (1994) Knowledge-based artificial neural networks. Artificial Intelligence, 70, 119-165.
35. Tresp, V. & Taniguchi, M. (1995) Combining estimators using non-constant weighting functions. In G. Tesauro, D. Touretzky & T. Leen (Eds), Advances in Neural Information Processing Systems, Vol. 7. Cambridge, MA: MIT Press.
36. Wolpert, D. (1992) Stacked generalization. Neural Networks, 5, 241-259.
37. Zhang, X., Mesirov, J. & Waltz, D (1992) Hybrid system for protein secondary structure prediction. Journal of Molecular Biology, 225, 1049-1063.