A Constructive Approach to Calculating Lower Entropy Bounds

Neural Processing Letters

Volumn 9, Issue 1, 1999, Pages 1-12

  Beiu, Valeriu ^a,d   Draghici, Sorin ^b   De Pauw, Thiery ^c  

^a LOS ALAMOS NATIONAL LABORATORY   (United States)

^b Wayne State University   (United States)

^c UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^d UNIVERSITY POLITEHNICA OF BUCHAREST   (Romania)

Author keywords

Classification problems; Complexity; Constructive algorithms; Limited and integer weights; N dimensional volumes; Number of bits

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTATIONAL COMPLEXITY; COMPUTER ARCHITECTURE; COMPUTER NETWORKS; COMPUTER SYSTEMS PROGRAMMING; INFORMATION THEORY; PROBLEM SOLVING; VLSI CIRCUITS;

CONSTRUCTIVE ALGORITHMS; LOWER ENTROPY BOUNDS;

NEURAL NETWORKS;

EID: 0033075190     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1018659009494     Document Type: Article

References (29)

1. M.A. Arbib (ed.), The Handbook of Brain Theory and Neural Networks, MIT Press: Cambridge, Mass., 1995.
2. E. Fiesler and R. Beale (eds), Handbook of Neural Computation, Oxford Univ. Press and the Inst. of Physics Publ: New York, NY, 1996.
3. V. Beiu, "Entropy bounds for classification algorithms", Neural Network World, Vol. 6, pp. 497-505, 1996.
4. S. Draghici and I.K. Sethi, "On the possibilities of the limited precision weights neural networks in classification problems", in J. Mira, R. Moreno-Díaz and J. Cabestany (eds) Biological and Artificial Computation: From Neuroscience to Technology, Lecture Notes in Computer Science 1240, pp. 753-762, Springer-Verlag: Berlin, June 1997.
5. V. Beiu and T. De Pauw, "Tight bounds on the size of neural networks for classification problems", in J. Mira, R. Moreno-Díaz and J. Cabestany (eds) Biological and Artificial Computation: From Neuroscience to Technology, Lecture Notes in Computer Science 1240, pp. 743-752, Springer-Verlag: Berlin, June 1997.
6. V. Beiu, "Digital integrated circuit implementations", Chapter E1.4 in [2].
7. V. Beiu, VLSI Complexity of Discrete Neural Networks, Gordon and Breach: Newark, N.J., 1998.
8. E.I. Neciporuk, "The synthesis of networks from threshold elements", Soviet Mathematics - Doklady, Vol. 5, pp. 163-166, 1964 (English transl. in Automation Express, Vol. 7, pp. 27-32 and pp. 35-39, 1964).
9. E.I. Neciporuk, "The synthesis of networks from threshold elements", Soviet Mathematics - Doklady, Vol. 5, pp. 163-166, 1964 (English transl. in Automation Express, Vol. 7, pp. 27-32 and pp. 35-39, 1964).
10. O.B. Lupanov, "The synthesis of circuits from threshold elements", Problemy Kibernetiki, Vol. 20, pp. 109-140, 1973.
11. 0 functions and spectral norms", SIAM J. Computing, Vol. 21, pp. 33-42, 1992.
12. K.-Y. Siu, V. Roychowdhury and T. Kailath, "Depth-size tradeoffs for neural computations", IEEE Trans. on Comp., Vol. C-40, pp. 1402-1412, 1991.
13. E.B. Baum, "On the capabilities of multilayer perceptrons", J. Complexity, Vol. 4, pp. 193-215, 1988.
14. S.-C. Huang and Y.-F. Huang, "Bounds on the number of hidden neurons of multilayer perceptrons in classification and recognition", IEEE Trans. on Neural Networks, Vol. NN-2, pp. 47-55, 1991.
15. M. Arai, "Bounds on the number of hidden units in binary-valued three-layer neural networks", Neural Networks, Vol. 6, pp. 855-860, 1993.
16. M.H. Hassoun, Fundamentals of Artificial Neural Networks, MIT Press: Cambridge, Mass., 1995.
17. H. Paugam-Moisy, "Optimisation des réseaux des neurones artificiels", PhD dissertation, Laboratoire de l'Informatique du Parallélisme LIP-IMAG (http://www.ens-lyon.fr/LIP/publis.us.html), École Normale Supérieure de Lyon, 46 Allée d'Italie, 69364 Lyon, France, 1992.
18. Y.S. Abu-Mostafa, "Connectivity versus entropy", in D.Z. Anderson (ed.) Neural Information Processing Systems, pp. 1-8, Inst. of Physics Publ.: New York, NY, 1988.
19. R.C. Williamson, "ε-entropy and the complexity of feedforward neural networks", in R.P. Lippmann, J.E. Moody and D.S. Touretzky (eds) Advances in Neural Information Processing Systems 3, pp. 946-952, Morgan Kaufmann: San Mateo, Calif., 1991.
20. V. Beiu and J.G. Taylor, "VLSI-optimal neural network learning algorithm", in D.W. Pearson, N.C. Steele and R.F. Albrecht (eds) Artificial Neural Nets and Genetic Algorithms, pp. 61-64, Springer-Verlag: Vienna, April 1995.
21. V Beiu and J.G. Taylor, "Direct synthesis of neural networks", in Proc. of the Intl. Conf. on Microelectronics for Neural Networks, pp. 257-264, IEEE CS Press: Los Alamitos, Calif., February 1996.
22. A.H. Khan and E.L. Hines, "Integer-weight neural networks", Electronic Lett., Vol. 30, pp. 1237-1238, 1994; see also: A.H. Khan, Feedforward neural networks with constrained weights, PhD dissertation, Intelligent Sys. Lab., Dept. of Eng., Warwick Univ., Coventry CV4 7AL, UK, 1996 (http://www.eng.warwick.ac.uk/~es2029).
23. A.H. Khan and E.L. Hines, "Integer-weight neural networks", Electronic Lett., Vol. 30, pp. 1237-1238, 1994; see also: A.H. Khan, Feedforward neural networks with constrained weights, PhD dissertation, Intelligent Sys. Lab., Dept. of Eng., Warwick Univ., Coventry CV4 7AL, UK, 1996 (http://www.eng.warwick.ac.uk/~es2029).
24. I. Parberry, Circuit Complexity and Neural Networks, MIT Press: Cambridge, Mass., 1994.
25. P. Raghavan, "Learning in threshold networks: a computational model and applications", IBM Research Center, Tech. Rep. RC 13859, 1988; also in Proc. of the 1st Workshop on Computational Learning Theory, pp. 19-27, ACM Press: New York, NY, 1988.
26. P. Raghavan, "Learning in threshold networks: a computational model and applications", IBM Research Center, Tech. Rep. RC 13859, 1988; also in Proc. of the 1st Workshop on Computational Learning Theory, pp. 19-27, ACM Press: New York, NY, 1988.
27. K.J. Lang and M.J. Witbrock, "Learning to tell two spirals apart," in Proc. of the 1988 Connectionist Models Summer School, pp. 52-59, Morgan Kaufmann: San Mateo, Calif., 1988.
28. V. Beiu, "Reduced complexity constructive learning algorithm", Neural Networks World, Vol. 8, pp. 1-38, 1998.
29. S. Draghici, V. Beiu and I.K. Sethi, "A VLSI optimal constructive algorithm for classification problems", in C.H. Dagli, M. Akay, O. Ersoy, B.R. Fernández and A.E Smith (eds) Smart Engineering Systems: Neural Networks, Fuzzy Logic, Data Mining and Evolutionary Programming, pp. 145-151, ASME Press: Fairfield, N.J., November 1997.