Generalization ability of Boolean functions implemented in feedforward neural networks

Neurocomputing

Volumn 70, Issue 1-3, 2006, Pages 351-361

  Franco, Leonardo ^a  

^a UNIVERSITY OF MÁLAGA   (Spain)

Author keywords

Complexity; Ferromagnetic systems; Learning; Parity; Perceptrons; Sensitivity

Indexed keywords

BOOLEAN FUNCTIONS; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL METHODS; COMPUTER SIMULATION;

NETWORK ARCHITECTURES; RANDOM FUNCTIONS;

FEEDFORWARD NEURAL NETWORKS;

ACCURACY; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER ANALYSIS; COMPUTER NETWORK; COMPUTER SIMULATION; CONTROLLED STUDY; CORRELATION ANALYSIS; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; PRIORITY JOURNAL; RELIABILITY;

EID: 33646505428     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2006.01.025     Document Type: Article

References (48)

1. M. Anthony, Boolean functions and artificial neural networks. CDAM Research Report LSE-CDAM-2003-01, in: Y. Crama, P. Hammer (Eds.), Boolean Functions: Volume II, 2006, to appear.
2. Baum B.E. What size of neural net gives valid generalization?. Neural Comput. 1 (1989) 151-160
3. Bhide S., John N., and Kabuka M.R. A Boolean neural network approach for the travelling salesman problem. IEEE Trans. Comput. 42 (1993) 1271-1278
4. Blum A.L., and Rivest R.L. Training a 3-node neural network is NP-complete. Neural Networks 5 (1992) 117-127
5. Boppana R.B. The average sensitivity of bounded-depth circuits. Inform. Process. Lett. 63 (1997) 257-261
6. R. Caruana, S. Lawrence, C. Lee Giles, Overfitting in neural networks: backpropagation, conjugate gradient, and early stopping, Advances in Neural Information Processing Systems, MIT Press, Denver, 13 (2001) 402-408.
7. A. Chojnacki, L. Jozwiak, An effective and efficient method for functional decomposition of Boolean functions based on information relationships measures, in: Proceedings of Design and Diagnostics of Electronic Circuits and Systems, DDECS'2000, Smolenice, Slovakia.
8. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), Wiley, New York
9. Deolalikar V. Mapping Boolean functions with neural networks having binary weights and zero thresholds. IEEE Trans. Neural Networks 12 (2001) 639-642
10. Dertouzos M.L. Theshold Logic: A Synthesis Approach (1965), MIT Press, Cambridge
11. Feldman J. Minimization of Boolean complexity in human concept learning. Nature 407 (2000) 572-573
12. Franco L., and Cannas S.A. Generalization and selection of examples in feedforward neural networks. Neural Comput. 12 (2000) 2405-2426
13. Franco L., and Cannas S.A. Generalization properties of modular networks implementing the parity function. IEEE Trans. Neural Networks 12 (2001) 1306-1313
14. Franco L., and Cannas S.A. Non glassy ground-state in a long-range antiferromagnetic frustrated model in the hypercubic cell. Physica A 332 (2004) 337-348
15. L. Franco, J.M. Jerez, J.M. Bravo-Montoya, Role of function complexity and network size in the generalization ability of feedforward networks, in: Proceedings of IWANN'05, Vilanova i la Geltrú, Spain, Lectures Notes in Computer Science, vol. 3512, 2005, pp. 1-8.
16. Grassberger P. Information and Complexity Measures in Dynamical Systems. Information Dynamics (1991), Plenum press, New York 15-33
17. Haykin S. Neural Networks: A Comprehensive Foundation (1994), Macmillan, New York
18. Hertz J., Krogh A., and Palmer R.G. Introduction to the Theory of Neural Computation (1991), Addison-Wesley, Redwood City, CA
19. Holden S.B., and Rayner P.J.W. Generalization and PAC learning: some new results for the class of generalized single layer networks. IEEE Trans. Neural Networks 6 (1993) 368-380
20. Hopcroft J., and Mattson R. Synthesis of minimal threshold logic networks. IEEE Trans. Electron. Comput. EC-14 (1965) 552-560
21. Ji C., and Psaltis D. Capacity of two-layer feedforward neural networks with binary weights. IEEE Trans. Inform. Theory 44 (1998) 256-268
22. Judd J.S. Neural Network Design and the Complexity of Learning (1990), MIT Press, Cambridge, MA
23. Kahn J., Kalai G., and Linial N. The influence of variables on Boolean functions. Proceedings 29th IEEE Symposium on the Foundations of Computer Science (FOCS' 88) (1988) 68-80
24. S. Lawrence, C.L. Giles, A.C. Tsoi, What size neural network gives optimal generalization? Convergence properties of backpropagation, Technical Report UMIACSTR -96-22 and CS-TR-3617, Institute for Advanced Computer Studies, University of Maryland, 1996.
25. R.A. Legenstein, W. Maass, Foundations for a circuit complexity theory of sensory processing, in: Advances in Neural Information Processing Systems, MIT Press, Denver, vol. 13, 2001, pp. 94-110.
26. Linial N., Mansour Y., and Nisan N. Constant depth circuits, Fourier transform and learnability. J. ACM 40 (1993) 607-620
27. Lovász L. Information and complexity (how to measure them?). In: Pullman B. (Ed). The Emergence of Complexity in Mathematics, Physics, Chemistry and Biology, Pontifical Academy of Sciences (1996), Princeton University Press, Vatican City 65-80
28. Muroga S. Threshold Logic and its Applications (1971), Wiley, New York
29. Oliveira A.L., and Sangiovanni-Vincentelli A. LSAT. an algorithm for the synthesis of two level threshold gate networks. Proceedings of the ACM/IEEE International Conference on Computer Aided Design. Santa Clara, CA (1991), IEEE Computer Society Press, Silver Spring, MD 130-133
30. A.L. Oliveira, A. Sangiovanni-Vincentelli, Learning complex Boolean functions: algorithms and applications, in: Advances in Neural Information Processing Systems, vol. 6, Morgan Kaufmann, Denver, 1993, pp. 911-918.
31. Orponen P. Computational complexity of neural networks: a survey. Nordic J. Comput. 1 (1994) 94-110
32. Parberry I. Circuit Complexity and Neural Networks (1994), MIT Press, Cambridge, MA
33. Perkowski M., Jozwiak L., and Mohamed S. New approach to learning noisy Boolean functions. Proceedings ICCIMA'98-International Conference on Computational Intelligence and Multimedia Applications. Gippsland, Victoria, Australia (1998), World Scientific, Singapore 693-706
34. Rawski M., Jozwiak L., and Luba T. Functional decomposition with an efficient input support selection for sub-functions based on information relationship measures. J. Syst. Architect. 47 (2001) 137-155
35. Reed R.D., and Marks R.J.II. Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks (1999), MIT Press, Cambridge, MA
36. Rice J.A. Mathematical Statistics and Data Analysis. 2nd ed. (1994), Duxbury Press, Belmont, CA
37. Shannon C.E. The synthesis of two-terminal switching circuits. Bell Syst. Tech. J. 28 (1949) 59-98
38. Sima J. Back-propagation is not efficient. Neural Networks 9 (1996) 1017-1023
39. Sima J., and Orponen P. General-purpose computation with neural networks: a survey of complexity theoretic results. Neural Comput. 15 (2003) 2727-2778
40. Siu K.Y., Roychowdhury V.P., and Kailath T. Depth-size tradeoffs for neural computation. IEEE Trans. Comput. 40 (1991) 1402-1412
41. C. Thornton, Parity: the problem that won't go away, in: Proceedings of AI-96, Toronto, 1996, pp. 362-374.
42. Vapnik V.N., and Chervonenkis A.J. The necessary and sufficient conditions for consistency of the method of empirical risk minimization. Pattern Recognition Image Anal. 1 (1991) 284-305
43. Vapnik V.N. Statistical Learning Theory (1998), Wiley, New York
44. Vidyasagar M. A Theory of Learning and Generalization: With Applications to Neural Networks and Control Systems (Communications and Control Engineering) (1997), Springer, Berlin
45. Wegener I. The Complexity of Boolean Functions (1987), Wiley, New York
46. R.O. Winder, Threshold Logic, Ph.D. Dissertation, Department of Mathematics, Princeton University, 1962.
47. D.H. Wolpert (Ed.), The mathematics of generalization, in: Proceedings of the SFI/CNLS Workshop on Formal Approaches to Supervised Learning, Santa Fe Institute Studies in the Sciences of Complexity, Volume XX, Addison-Wesley, Reading, MA, 1995.
48. Zhu H., and Kinzel W. Antipredictable sequences: harder to predict than random sequences. Neural Comput. 10 (1998) 2219-2230