An Enhancement of Generalization Ability in Cascade Correlation Algorithm by Avoidance of Overfitting/Overtraining Problem

Neural Processing Letters

Volumn 6, Issue 1-2, 1997, Pages 43-50

  Tetko, Igor V ^a,b   Villa, Alessandro E P ^a  

^a UNIVERSITY OF LAUSANNE   (Switzerland)

^b INSTITUTE OF BIOORGANIC CHEMISTRY AND PETROCHEMISTRY   (Ukraine)

Author keywords

Cascade correlation algorithm; Early stopping; Overfitting; Overtraining

Indexed keywords

LEARNING ALGORITHMS; PROBLEM SOLVING;

CASCADE CORRELATION ALGORITHM (CCA); EARLY STOPPING AND ENSEMBLE AVERAGING (ESE); LEAVE ONE OUT TECHNIQUE; OVERFITTING/OVERTRAINING PROBLEM;

NEURAL NETWORKS;

EID: 0031212022     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/a:1009610808553     Document Type: Article

References (14)

1. R. Hecht-Nielsen, "Kolmogorov's mapping neural network existence theorem", Proc. Int. Conf. on Neural Networks, pp. 11-14, 1987.
2. G. Wahba, "Generalization and regularization in nonlinear learning systems", The Handbook of Brain Theory and Neural Networks, M. Arbib (editor) MIT press, pp. 426-430, 1995.
3. S. Fahlman and C. Lebiere, "The cascade-correlation learning architecture", NIPS, Vol. 2, pp. 524-532, 1990.
4. R. Reed and R. Pruning, "Algorithms - a survey", IEEE Trans. Neural Networks, Vol. 4, pp. 740-747, 1993.
5. Y. Le Cun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard and L. Jackel, "Handwritten digit recognition with a backpropagation network, NIPS, Vol. 2, pp. 396-404, 1990.
6. S. Bös, "Avoiding overfitting by finite temperature learning and cross-validation", Proc. ICANN'95 (Paris), Vol. 2, pp. 111-116, 1995.
7. D.J.C. MacKay, "A practical Bayesian framework for backpropagation networks", Neural Computation, Vol. 4, pp. 448-472, 1992.
8. C.M. Bischop, Neural networks for pattern recognition, Oxford: Oxford University Press, 1995.
9. R. Hecht-Nielsen, Neurocomputing, Addison-Wesley, 1989.
10. I.V. Tetko, D.J. Livingstone and A.I. Luik, "Neural network studies. 1. Comparison of overfitting and overtraining", J. Chem. Inf. Comput. Sci., Vol. 35, pp. 826-833, 1995.
11. L. Breimann, "Bagging predictors", Machine Learning, Vol. 24, pp. 123-140, 1996.
12. I.V. Tetko and A.E.P. Villa, "Efficient partition of learning data sets for neural network training", Neural Networks, 1997, in press.
13. S. Geman, E. Bienenstock and R. Dourstat, "Neural networks and the bias/variance dilemma", Neural Computation, Vol. 4, pp. 1-58, 1992.
14. S. Amari, N. Murata, K.-R. Muller, M. Finke and H. Yang, "Asymptomic statistical theory of overtraining and cross-validation", Technical report University of Tokyo, METR-95/06, 1995.