Generalization: The hidden agenda of learning

IEEE Signal Processing Magazine

Volumn 14, Issue 6, 1997, Pages 43-44

  Larsen, Jan ^a   Hansen, Lars Kai ^a  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL METHODS; DATABASE SYSTEMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION;

PRINCIPAL COMPONENT ANALYSIS;

NEURAL NETWORKS;

EID: 0031274657     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/msp.1997.637310     Document Type: Article

References (18)

1. Y. Le Cun, et al., "Back-propagation Applied to Handwritten Zip Code Recognition," Neural Computation, vol. 1, no. 4, pp. 541-551, 1989.
2. S. Geman, E. Bienenstock & R. Doursat, "Neural Networks and the Bias/Variance Dilemma." Neural Computation, vol. 4, pp. 1-59, 1992.
3. H. Akaike, "Fitting Autoregressive Models for Prediction," Ann. Inst. Stat. Mat., vol. 21, pp, 243-247, 1969.
4. P. Craven & G. Wahba, "Smoothing Noisy Data with Spline Functions: Estimating the Correct Degree of Smoothing by the Method of Generalized Cross-Validation," Numerical Methematics, vol. 31, 377-403, 1979.
5. J. Larsen, "A Generalization Error Estimate for Nonlinear Systems," in S.Y. Kung, F. Failside, J. Aa. Sorensen & C.A. Kamm (eds.) Neural Networks for Signal Processing 2: Proceedings of the 1992 IEEE-SP Workshop, Piscataway, New Jersey: IEEE, 1992, pp. 29-38.
6. J. Moody, "Note on Generalization, Regularization, and Architecture Selection in Nonlinear Learning Systems," in B.H. Juang, S.Y. Kung & C.A. Kamm (eds.) Proceedings of the First IEEE Workshop on Neural Networks for Signal Processing, Piscataway, New Jersey: IEEE, pp. 1-10, 1991.
7. J. Moody, "Prediction Risk and Architecture Selection for Neural Networks" in V. Cherkassky, J.H. Friedman & H. Wechsler (eds.) From Statistics to Neural Networks: Theory and Pattern Recognition Applications, Series F, vol. 136, Berlin, Germany: Springer-Verlag, 1994.
8. N. Murata, S. Yoshizawa and & S. Amari, "Network Information Criterion - Determining the Number of Hidden Units for an Artificial Neural Network Model," IEEE Transactions on Neural Networks, vol. 5, no. 6, pp. 865-872, Nov. 1994.
9. L.K. Hansen & J. Larsen, "Unsupervised Learning and Generalization," Proceedings of IEEE International Conference on Neural Networks, Washington DC, vol. 1, pp. 25-30, June 1996.
10. S. Geisser, "The Predictive Sample Reuse Method with Application," Journal of The American Statistical Association, vol. 50, pp. 320-328, 1975.
11. M. Stone, "Cross-validatory Choice and Assessment of Statistical Predictors," Journal of the Royal Statistical Society 5, vol. 36, no. 2, pp. 111-147, 1974.
12. L.K. Hansen & J. Larsen, "Linear Unlearning for Cross-Validation," Advances in Computational Mathematics, vol. 5, pp. 269-280, 1996.
13. B. Efron & R. Tibshirani, An Introduction to the Bootstrap, New York, NY: Chapman & Hall, 1993.
14. B. Efron & R. Tibshirani, "Cross-Validation and the Bootstrap: Estimating the Error Rate of a Prediction Rule," Techn. Report no. 477, Dept. of Statistics, Stanford University, May 1995. To appear in Journ. Amer. Statist.
15. R. Tibshirani, "A Comparison of Some Error Estimates for Neural Network Models," Neural Computation, vol. 8, pp. 152-163, 1996.
16. A.S. Weigend & B. LeBaron, "Evaluating Neural Network Predictors by Bootstrapping," in Proceedings of the International Conference on Neural Information Processing (ICONIP'94 Seoul, Korea, pp. 1207-1212, 1994.
17. C. Svarer, L.K. Hansen & J. Larsen, "On Design and Evaluation of Tapped-Delay Neural Architectures," in Proceedings of the IEEE International Conference on Neural Networks, San Francisco, California, USA, vol. 1, pp. 46-51, 1993.
18. Y. Le Cun, J.S. Denker & S.A. Solla, Optimal Brain Damage. In D.S. Touretzky (ed.), Advances in Neural Information Processing Systems 2, Proceedings of the 1989 Conference., San Mateo, California: Morgan Kaufmann Publishers pp. 598-605, 1990.