Bounds for the computational power and learning complexity of analog neural nets

Proceedings of the Annual ACM Symposium on Theory of Computing

Volumn Part F129585, Issue , 1993, Pages 335-344

  Maass, Wolfgang ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

BOOLEAN FUNCTIONS; CHEMICAL ACTIVATION; COMPUTER CIRCUITS; PIECEWISE LINEAR TECHNIQUES; RATIONAL FUNCTIONS; THRESHOLD ELEMENTS; APPROXIMATION THEORY; BOOLEAN ALGEBRA; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL METHODS; CONTROL SYSTEMS; FUNCTION EVALUATION; LEARNING SYSTEMS; LOGIC GATES; RECURSIVE FUNCTIONS;

ACTIVATION FUNCTIONS; ASYMPTOTICALLY OPTIMAL; COMPUTATIONAL POWER; HEAVISIDE FUNCTIONS; LEARNING COMPLEXITY; LINEAR THRESHOLD GATES; PIECEWISE LINEAR ACTIVATIONS; PIECEWISE POLYNOMIALS;

NEURAL NETWORKS;

ANALOG NEURAL NETS; LEARNING COMPLEXITY;

EID: 0027224339     PISSN: 07378017     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/167088.167193     Document Type: Conference Paper

References (44)

1. Y. S. Abu-Mostafa, "The Vapnik-Cher von- enkis dimension: information versus com-plexity in learning", Neural Computation, vol. 1, 1989, 312 - 317.
2. P. L. Bartlett, "Lower bounds on the Vapnik-Chervonenkis dimension of multilayer threshold networks", preprint (July 1992).
3. E. B. Baum, D. Haussler, "What size net gives valid generalization?", Neural Computation, vol. 1, 1989, 151 - 160.
4. [BR] A. Blum, R. L. Rivest, "Training a 3-node neural network is NP-complete", Proc. of the 1988 Workshop on Computational Learning Theory, Morgan Kaufmann (San Mateo, 1988), 9 - 18.
5. [CSV] A. K. Chandra, L. Stockmeyer, U. Vishkin, "Constant depth reducibility", SI AM J. Computing, vol. 13 (2), 1984, 423 - 439.
6. [C] T. M. Cover, "Capacity problems for linear machines", in: Pattern Recognition, L. Kanal ed., Thompson Book Co., 1988, 283 - 289.
7. [DS] B. DasGupta, G. Schnitger, "Efficient ap proximations with neural networks: A comparison of gate functions", preprint (Aug. 1992).
8. [DR] R. Durbin, D. E. Rumelhart, "Product units: A computationally powerful and biologically plausible extension to backpropagation networks", Neural Computation, vol. 1, 1989, 133 - 142.
9. [E] H. Edelsbrunner, "Algorithms in Combina torial Geometry", Springer (Berlin, 1987).
10. [EOS] H. Edelsbrunner, J. O'Rourke, R. Seidel, "Constructing arrangements of lines and hyperplanes with applications", SI AM J. Comp., vol. 15, 1986, 341 - 363.
11. [GJ] P. Goldberg, M. Jerrum, "Bounding the Vapnik - Chervonenkis dimension of con-cept classes parameterized by real numbers", preprint (February 1993).
12. [GHR] M. Goldmann, J. Hastad, A. Razborov, "Ma jority gates vs. general weighted threshold gates", Proc. of the 7th Structure in Complexity Theory Conference, 1992, 2 - 13.
13. [HMPST] A. Hajnal, W. Maass, P. Pudlak, M. Szegedy and G. Turan, "Threshold circuits of bounded depth", Proc. of the 28th Annual IEEE Symp. on Foundations of Computer Scicncc, 1987, 99 - 110. Full version to appear in J. Comp. Syst. Sci. 1993.
14. [Has] J. Hastad, "On the size of weights for thresh old gates", preprint (September 1992).
15. [H] D. Haussler, "Decision theoretic generaliza tions of the PAC model for neural nets and other learning applications", Information and Computation, vol. 100, 1992, 78 - 150.
16. [Ho] J. J. Hopfield, "Neurons with graded re sponse have collective computational properties like those of two-state neurons", Proc. Nat. Acad, of Sciences USA, 1984, 3088 - 3092.
17. [J] D. S. Johnson, "A catalog of complexity classes", in: Handbook of Theoretical Computer Science vol. A, J. van Leeuwen ed., MIT Press (Cambridge, 1990).
18. [KV] M. Kearns, L. Valiant, "Cryptographic limi tations on learning boolean formulae and finite automata", Proc. of the 21st ACM Symposium on Theory of Computing, 1989, 433 - 444.
19. [L] R. P. Lippmann, "An introduction to com puting with neural nets", IEEE ASSP Magazine, 1987, 4 - 22.
20. [Lu] O. B. Lupanov, "On circuits of threshold el ements", Dokl. Akad. Nauk SSSR, vol. 202, 1288 - 1291; engl. translation in: Sov. Phys. Dokl, vol. 17, 1972, 91 - 93.
21. [M] W. Maass, "Bounds for the computational power and learning complexity of analog neural nets", IIG - Report 349 of the Tech- nische Universitat Graz, (October 1992).
22. [MSS] W. Maass, G. Schnitger, E. D. Sontag, "On the computational power of sigmoid versus boolean threshold circuits", Proc. of the 32nd Annual IEEE Symp. on Foundations of Computer Science, 1991, 767 - 776.
23. [MT] W. Maass, G. Turan, "How fast can a thresh old gate learn?", in: Computational Learning Theory and Natural Learning Systems: Constraints and Prospects, G. Drastal, S. J. Hanson and R. Rivest eds., MIT Press, to appear.
24. [MS] A. Macintyre, E. D. Sontag, "Finiteness re sult for sigmoidal "neural" networks", Proc. of the 23th ACM Symposium on Theory of Computing, 1993.
25. [MR] J. L. McClelland, D. E. Rumelhart "Parallel Distributed Processing", vol. 2, MIT Press (Cambridge, 1986).
26. [Me] N. Megiddo, "Linear Programming in linear time when the dimension is fixed", J. of the ACM, vol. 31, 1984, 114 - 127.
27. [MP] M. Minsky. S. Papert. "Perceptrons: An Introduction to Computational Geometry", Expanded Edition, MIT Press (Cambridge, 1988).
28. [MD] J. Moody, C. J. Darken, "Fast learning in networks of locally-Tuned processing units", Neural Computation, vol. 1, 1989, 281 - 294.
29. [Mu] S. Muroga, "Threshold Logic and its Appli cations", Wiley (New York, 1971).
30. [N] E.I. Neciporuk, "The synthesis of networks from threshold elements", Probl. Kibern. No. 11, 1964, 49 - 62; engl. translation in: Autom. Expr., vol. 7, No. 1, 1964, 35 - 39.
31. [Ni] N. J. Nilsson, Learning Machines, McGraw- Hill (New York, 1971).
32. [PS] I. Parbcrry, G. Schnitger, "Parallel computation with threshold functions", Lecture Notes in Computer Science vol. 223, Springer (Berlin, 1986), 272 - 290.
33. [PG] T. Poggio, F. Girosi, "Networks for approximation and learning", Proc. of the IEEE, vol. 78(9), 1990, 1481 - 1497.
34. [R] F. Rosenblatt, "Principles of Neurodynamics", Spartan Books (New York, 1962).
35. [RM] D. E. Rumelhart, J. L. McClelland, "Parallel Distributed Processing", vol. 1, MIT Press (Cambridge, 1986).
36. [Sch] A. Schrijver, "Theory of Linear and Integer Programming", Wiley (New York, 1986).
37. [SS] H. T. Siegelmann, E. D. Sontag, "Neural networks with real weights: Analog com-putational complexity", Report SYCON-92- 05, Rutgers Center for Systems and Control (Oct. 1992).
38. K. Y. Siu, J. Bruck, T. Kailath, T. Hofmcis-Ter, "Depth efficient neural networks for di-vision and related problems", to appear in IEEE Transactions on Inf. Theory.
39. [SR] K. Y. Siu, V. Roychowdhury, "On optimal depth threshold circuits for multiplication and related problems", Tech. Report ECE - 92-05, University of California, Irvine (March 1992).
40. [S1] E. D. Sontag, "Remarks on interpolation and recognition using neural nets", in: Advances in Neural Information Processing Systems 3, R. P. Lippmann, J. Moody, D. S. Touretzky, cds., Morgan Kaufmann (San Mateo, 1991), 939 - 945.
41. [S3] E. D. Sontag, "Feedforward nets for interpolation and classification", J. Comp. Syst. Set., vol. 45, 1992, 20 - 48.
42. [S3] E. D. Sontag, private communication (July 1992).
43. [T] G. Turan, private notes (1989).
44. [V] L. G. Valiant, "A theory of the learnable", Comm. of the ACM, vol. 27, 1984, 1134 - 1142.