Role of function complexity and network size in the generalization ability of feedforward networks

Lecture Notes in Computer Science

Volumn 3512, Issue , 2005, Pages 1-8

  Franco, Leonardo ^a,b   Jerez, José M ^b   Bravo, José M ^b  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b UNIVERSITY OF MÁLAGA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; BOOLEAN ALGEBRA;

GENERALIZATION PROCESS; HIDDEN LAYER NETWORKS; NEURONS;

FEEDFORWARD NEURAL NETWORKS;

EID: 25144436369     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/11494669_1     Document Type: Conference Paper

References (14)

1. Haykin, S. (1994). Neural Networks: A Comprehensive Foundation. Macmillan/IEEE Press.
2. Baum, E.B. & Haussler, D. (1989) What size net gives valid generalization ? Neural Computation, 1, pp. 151-160.
3. Lawrence, S., Giles, C. L., & Tsoi, A. C. (1996). What Size Neural Network Gives Optimal Generalization ? Convergence Properties of Backpropagation. In Technical Report UMIACS-TR-96-22 and CS-TR-3617, Institute for Advanced Computer Studies, Univ. of Maryland.
4. Caruana, R., Lawrence, S., & Giles, C.L. (2001). Overfitting in Neural Networks: Backpropagation, Conjugate Gradient, and Early Stopping. In Leen, T. K., Dietterich, T. G. & Tresp, V. editors, Advances in Neural Information Processing Systems, MIT Press, 13, pp. 402-408.
5. Krogh, A. & Hertz, J.A. (1992) A simple weight decay can improve generalization. In J.E. Moody, S. J. Hanson, & R. P. Lippmann editors, Advances in Neural Information Processing Systems Morgan Kaufmann, San Mateo, CA, 4, pp. 950-957.
6. Prechelt, L. (1998). Automatic Early Stopping Using Cross Validation: Quantifying the Criteria. Neural Networks, 11, pp.761-767.
7. Setiono, R. (2001) Feedforward neural network construction using cross-validation, Neural Computation, 13, pp. 2865-2877.
8. Bartlett, P.L. (1997). For valid generalization the size of the weights is more important than the size of the network. In M.C. Mozer, M. I. Jordan, & T. Petsche, editors, Advances in Neural Information Processing Systems, MIT Press, 9, pp. 134-140.
9. Franco, L. & Anthony, M. (2004). On a generalization complexity measure for Boolean functions. In Proceedings of the 2004 IEEE International Joint Conference on Neural Networks, IEEE Press, pp. 973-978.
10. Wegener, I. (1987) The complexity of Boolean functions. Wiley and Sons Inc.
11. Siu, K.Y., Roychowdhury, V.P., & Kailath, T. (1991) Depth-Size Tradeoffs for Neural Computation IEEE Transactions on Computers, 40, pp. 1402-1412.
12. Franco, L. & Cannas, S.A. (2004). Non glassy ground-state in a long-range anti-ferromagnetic frustrated model in the hypercubic cell Physica A, 332, pp. 337-348.
13. Franco, L. & Cannas, S.A. (2000). Generalization and Selection of Examples in Feedforward Neural Networks. Neural Computation, 12, 10, pp. 2405-2426.
14. Franco, L. & Cannas, S.A. (2001). Generalization Properties of Modular Networks: Implementing the Parity Function. IEEE Transactions on Neural Networks, 12, pp. 1306-1313.