Automatic early stopping using cross validation: Quantifying the criteria

Neural Networks

Volumn 11, Issue 4, 1998, Pages 761-767

  Prechelt, Lutz ^a  

^a UNIVERSITY OF KARLSRUHE   (Germany)

Author keywords

Cross validation; Early stopping; Empirical study; Generalization; Overfitting; Supervised learning

Indexed keywords

APPROXIMATION THEORY; COMPUTER ARCHITECTURE; CONVERGENCE OF NUMERICAL METHODS; LEARNING SYSTEMS;

CROSS VALIDATION; SUPERVISED LEARNING; SUPERVISED NEURAL NETWORKS;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ARTICLE; INFORMATION PROCESSING; LEARNING; MATHEMATICAL ANALYSIS; NERVE CELL NETWORK; PRIORITY JOURNAL; TRAINING;

EID: 0032099978     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(98)00010-0     Document Type: Article

References (22)

1. Baldi P., Chauvin Y. Temporal evolution of generalization during learning in linear networks. Neural Computation. 3:1991;589-603.
2. Cowan, J.D., Tesauro, G. & Alspector, J. (Eds.), 1994. Advances in Neural Information Processing Systems 6, Morgan Kaufman, San Mateo, CA.
3. Cun, Y.L., Denker, J.S. & Solla, S.A., 1990. Optimal brain damage. In: Touretzky, D.S. (Ed.), Advances in Neural Information Processing Systems 2, Morgan Kaufman, San Mateo, CA, pp. 598-605.
4. Fiesler, E., 1994. Comparative bibliography of ontogenic neural networks. In: International Conference on Artificial Neural Networks. Springer, London.
5. Fahlman, S.E., 1988. An empirical study of learning speed in back-propagation networks. Technical Report CMU-CS-88-162, School of Computer Science. Carnegie Mellon University Pittsburgh, PA.
6. Fahlman, S.E. & Lebiere, C., 1990. The cascade-correlation learning architecture. In: Touretzky, D.S. (Ed.), Advances in Neural Information Processing Systems 2, Morgan Kaufman, San Mateo, CA, pp. 524-532.
7. Finnoff W., Hergert F., Zimmermann H.G. Improving model selection by nonconvergent methods. Neural Networks. 6:1993;771-783.
8. Geman S., Bienenstock E., Doursat R. Neural networks and the bias/variance dilemma. Neural Computation. 4:1992;1-58.
9. Hanson, S.J., Gowan, J.D. & Giles, C.L. (Eds.), 1993. Advances in Neural Information Processing Systems 5, Morgan Kaufman, San Mateo, CA.
10. Hassibi, B. & Stork, D.G., 1993. Second order derivatives for network pruning: optimal brain surgeon. In: Advances in Neural Information Processing Systems 5, Morgan Kaufman, San Mateo, CA, pp. 164-171.
11. Krogh, A. & Hertz, J.A., 1992. A simple weight decay can improve generalization. In: Advances in Neural Information Processing Systems 4, Morgan Kaufman, San Mateo, CA, pp. 950-957.
12. Levin, A.U., Leen, T.K. & Moody, J.E., 1994. Fast pruning using principal components. In: Advances in Neural Information Processing Systems 6, Morgan Kaufman, San Mateo, CA.
13. Lippmann, R.P. & Moody, J.E., Touretzky, D.S. (Eds.), 1991. Advances in Neural Information Processing Systems 3, Morgan Kaufman, San Mateo, CA.
14. Moody, J.E., Hanson, S.J. & Lippmann, R.P. (Eds.), 1992. Advances in Neural Information Processing Systems 4, Morgan Kaufman, San Mateo, CA.
15. Morgan, N. & Bourlard, H., 1990. Generalization and parameter estimation in feedforward nets: some experiments. In: Touretzky, D.S. (Ed.), Advances in Neural Information Processing Systems 2, San Mateo, CA, pp. 630-637.
16. Nowlan S.J., Hinton G.E. Simplifying neural networks by soft weight-sharing. Neural Computation. 4(4):1992;473-493.
17. Prechelt, L., 1994. PROBEN1 - a set of benchmarks and benchmarking rules for neural network training algorithms. Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, Germany. Anonymous FTP: /pub/papers/techreports/1994/1994-21.ps.gz on ftp.ira.uka.de.
18. Reed R. Pruning algorithms - a survey. IEEE Transactions on Neural Networks. 4(5):1993;740-746.
19. Riedmiller, M. & Braun, H., 1993. A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In: Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, pp. 586-591.
20. Touretzky, D.S. (Ed.), 1990. Advances in Neural Information Processing Systems 2, Morgan Kaufman, San Mateo, CA.
21. Wang, C., Venkatesh, S.S. & Judd, J.S., 1994. Optimal stopping and effective machine complexity in learning. In: Advances in Neural Information Processing Systems 6, Morgan Kaufman, San Mateo, CA.
22. Weigend, A.S., Rumelhart, D.E. & Huberman, B.A., 1991. Generalization by weight-elimination with application to forecasting. In: Advances in Neural Information Processing Systems 3, Morgan Kaufman, San Mateo, CA, pp. 875-882.