Tight bounds on the size of neural networks for classification problems

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 1240 LNCS, Issue , 1997, Pages 743-752

  Beiu, Valeriu ^a   De Pauw, Thierry ^b  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

^b UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

Author keywords

Boolean functions; Classification problems; Entropy; Neural networks; Size complexity

Indexed keywords

BOOLEAN FUNCTIONS; CLASSIFICATION (OF INFORMATION); ENTROPY;

DATA SET; SIZE COMPLEXITY; TIGHT BOUND; UPPER BOUND;

NEURAL NETWORKS;

EID: 84902189013     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/bfb0032533     Document Type: Conference Paper

References (30)

1. Abu-Mostafa, Y.S.: Connectivity versus entropy, in D.Z. Anderson (ed.): Neural information processing systems (NIPS*87, Denver, CO), American Institute of Physics, NY (1988) 1-8.
2. Arai, M.: Bounds on the number of hidden units in binary-valued three-layer neural networks, Neural Networks 6(6) (1993) 855-860.
3. Arbib, M.A. (ed.): The handbook of brain theory and neural networks, MIT Press, Cambridge, MA (1995).
4. Baum, E.B.: On the Capabilities of Multilayer Perceptrons, J. of Complexity 4 (1988) 193-215.
5. Baum, E.B.: When are k-nearest neighbour and back propagation accurate for feasible size sets of examples?, in E. Amaldi and E. Mayoraz: Mathematical foundations of artificial neural networks (summer school lecture notes), Swiss Federal Institute of Technology (Lausanne) and Kurt BOsch Academic Institute (Sion), 1992.
6. S.J. Hanson, G.A. Drastal and R.L. Rivest (eds.): Computational learning theory and natural learning systems Vol. 1 Constraints and prospects, Bradford/MIT Press (1994).
7. Beiu, V.: Entropy bounds for classification algorithms, Neural Network Worm 6(4) (1996) 497-505.
8. Beiu, V, : Digital integrated circuit implementations, Chapter E1.4 in [13].
9. Beiu, V.: VLSI complexity of discrete neural networks, Gordon and Breach (1997).
10. Beiu, V., Taylor, J.G.: VLSI-optimal neural network learning algorithm, in D.W. Pearson, N.C. Steele and R.F. Albrecht (eds.): Artificial neural nets and genetic algorithms (ICANNGA'95, Ales, France), Springer-Verlag, Vienna (1995) 61-64.
11. Beiu, V., Taylor, J.G.: Direct synthesis of neural networks, Proc. MicroNeuro'96 (Lausanne, Switzerland), IEEE CS Press, Los Alamitos, CA (1996) 257-264.
12. Bishop, C.M.: Neural networks for pattern recognition, Oxford University Press, Oxford, UK (1995).
13. Bruck, J., Smolensky, R.: Polynomial threshold functions, AC functions and spectral norms, Res. Rep. RJ 7410 (67387), 11/15/89, IBM Yorktown Heights, NY (1989).
14. SIAM J, Computing 21(1) (1992) 33-42.
15. Fiesler, E., Beale, R. (eds.): Handbook of neural computation. Oxford University Press and the Institute of Physics Publishing, NY (1996).
16. Hassoun, M.H.: Fundamentals of artificial neural networks, MIT Press, Cambridge, MA (1995).
17. Hecht-Nielsen, R., Neurocomputing, Addison-Wesley, Reading, MA (1989).
18. Huang, S.-C., Huang, Y.-F.: Bounds on the number of hidden neurons of multilayer perceptrons in classification and recognition, IEEE Trans. on Neural Networks 2(1) (1991) 47-55.
19. Kohonen, T.: Self-organizing and associative memory, Springer-Verlag, Berlin (1988).
20. Krishnamoorthy, A.V., Paturi, R., Blume, M., Linden, G.D., Linden, L.H., Esener, S.C.: Hardware tradeoffs for Boolean concept learning, Proc. Worm Conf. on Neural Networks '94 (WCNN'94, San Diego), Lawrence Erlbaum & INNS Press, Hillsdale (1994) vol. 1, 551-559.
21. Lupanov, O.B.: The synthesis of circuits from threshold elements, Problemy Kibernetiki 20 (1973) 109-140.
22. Neciporuk, E.I.: The synthesis of networks from threshold elements, Problemy Kibernetiki 11 (1964) 49-62.
23. Neciporuk, E.I.: The synthesis of networks from threshold elements English translation in Automation Express 7(1) 35-39.
24. Neciporuk, E.I.: The synthesis of networks from threshold elements English translation in Automation Express and 7(2) 27-32.
25. Parberry, I.: Circuit complexity and neural networks, MIT Press, Cambridge, MA (1994).
26. Pangam-Moisy, H.: Optimisation des rs des neurones artificiels. Ph.D. thesis, Laboratoire de l'Informatique du Parallelisme LIP-IMAG, Ecole Normale Superieure de Lyon, 46 Allee d'Italie, 69364 Lyon, France (1992).
27. Raghavan, P.: Learning in threshold networks: A computational model and applications. Tech. Rep. RC 13859, IBM Res. (1988).
28. Raghavan, P Proc. Ist Workshop on Computational Learning Theory (Cambridge, MA), ACM Press, NY (1988) 19-27.
29. Siu, K.-Y., Roychowdhury, V., Kailath, T.: Depth-size tradeoffs for neural computations, IEEE Trans. on Comp. 40(12) (1991) 1402-1412.
30. Williamson, R.C." E-entropy and the complexity of feedforward neural networks, in R.P. Lippmann, J.E. Moody and D.S. Touretzky (eds.): Neural information processing systems (NIPS*90, Denver, CO), Morgan Kaufmann, San Mateo, CA (1991) 946-952.