General-Purpose Computation with Neural Networks: A Survey of Complexity Theoretic Results

Neural Computation

Volumn 15, Issue 12, 2003, Pages 2727-2778

  Šíma, Jiří ^a   Orponen, Pekka ^b  

^a INSTITUTE OF COMPUTER SCIENCE   (Czech Republic)

^b AALTO UNIVERSITY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; CLASSIFICATION; MATHEMATICAL COMPUTING; NERVE CELL; PHYSIOLOGY; REVIEW; STATISTICAL MODEL; TIME;

ALGORITHMS; CLASSIFICATION; MATHEMATICAL COMPUTING; MODELS, NEUROLOGICAL; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NEURONS; TIME FACTORS;

EID: 10744230566     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976603322518731     Document Type: Review

References (156)

1. Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9(1), 147-169.
2. Allender, E. (1989). A note on the power of threshold circuits. In Proceedings of the 30th Annual Symposium on Foundations of Computer Science (FOCS '89), Research Triangle Park, North Carolina (pp. 580-584). New York: IEEE Computer Society Press.
3. Alon, N. (1985). Asynchronous threshold networks. Graphs and Combinatorics, 1, 305-310.
4. Alon, N., & Bruck, J. (1994). Explicit construction of depth-2 majority circuits for comparison and addition. SIAM Journal on Discrete Mathematics, 7(1), 1-8.
5. Alon, N., Dewdney, A. K., & Ott, T. J. (1991). Efficient simulation of finite automata by neural nets. Journal of the ACM, 38(2), 495-514.
6. Balcázar, J. L., Díaz, J., & Gabarró, J. (1995). Structural complexity I (2nd ed.). Berlin: Springer-Verlag.
7. Balcázar, J. L., Gavaldà, R., & Siegelmann, H. T. (1997). Computational power of neural networks: A characterization in terms of Kolmogorov complexity. IEEE Transactions on Information Theory, 43(4), 1175-1183.
8. Balcázar, J. L., & Hermo, M. (1998). The structure of logarithmic advice complexity classes. Theoretical Computer Science, 207(1), 217-244.
9. Barahona, F. (1982). On the computational complexity of Ising spin glass models. Journal of Physics A: Mathematical and General, 15(10), 3241-3253.
10. Ben-Hur, A., Siegelmann, H. T., & Fishman, S. (2002). A theory of complexity for continuous time systems. Journal of Complexity, 18(1), 51-86.
11. Bertoni, A., & Campadelli, P. (1994). On the approximability of the energy function. In Proceedings of the 4th International Conference on Artificial Neural Networks (ICANN'94) (pp. 1157-1160). Berlin: Springer-Verlag.
12. Bertoni, A., Campadelli, P., Gangai, C., & Posenato, R. (1997). Approximability of the ground state problem for certain Ising spin glasses. Journal of Complexity, 13(3), 323-339.
13. Bieche, I., Maynard, R., Rammal, R., & Uhry, J. P. (1980). On the ground states of the frustration model of a spin glass by a matching method of graph theory. Journal of Physics A: Mathematical and General, 13(8), 2553-2576.
14. Bovet, D. P., & Crescenzi, P. (1994). Introduction to the theory of complexity. Hemel Hempstead: Prentice Hall International.
15. Bruck, J., & Goodman, J. W. (1988). A generalized convergence theorem for neural networks. IEEE Transactions on Information Theory, 34(5), 1089-1092.
16. Casey, M. (1996). The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation, 8(6), 1135-1178.
17. Chandra, A. K., Stockmeyer, L. J., & Vishkin, U. (1984). Constant depth reducibility. SIAM Journal on Computing, 13(2), 423-439.
18. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 13(5), 815-826.
19. Cover, T. M. (1965). Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, 14(3), 326-334.
20. Cover, T. M. (1968). Capacity problems for linear machines. In L. Kanal (Ed.), Pattern recognition (pp. 283-289). Washington, DC: Thompson Book Co.
21. DasGupta, B., & Schnitger, G. (1993). The power of approximating: A comparison of activation functions. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems (NIPS'92), 5 (pp. 615-622). San Mateo, CA: Morgan Kaufmann.
22. DasGupta, B., & Schnitger, G. (1996). Analog versus discrete neural networks. Neural Computation, 8(4), 805-818.
23. Floréen, P. (1991). Worst-case convergence times for Hopfield memories. IEEE Transactions on Neural Networks, 2(5), 533-535.
24. Floréen, P., & Orponen, P. (1989). On the computational complexity of analyzing Hopfield nets. Complex Systems, 3(6), 577-587.
25. Floréen P., & Orponen, P. (1993). Attraction radii in Hopfield nets are hard to compute. Neural Computation, 5(5), 812-821.
26. Floréen, P., & Orponen, P. (1994). Complexity issues in discrete Hopfield networks (Research Rep. No. A-1994-4). Helsinki: Department of Computer Science, University of Helsinki.
27. Fogelman, F., Goles, E., & Weisbuch, G. (1983). Transient length in sequential iterations of threshold functions. Discrete Applied Mathematics, 6(1), 95-98.
28. Fogelman-Soulié, F., Mejia, C., Goles, E., & Martinez, S. (1989). Energy functions in neural networks with continuous local functions. Complex Systems, 3(3), 269-293.
29. Furst, M., Saxe, J. B., & Sipser, M. (1984). Parity, circuits and the polynomial-time hierarchy. Mathematical Systems Theory, 17(1), 13-27.
30. Garey, M. R., & Johnson, D. S. (1979). Computers and intractability: A guide to the theory of NP-completeness. New York: Freeman.
31. Godbeer, G. H., Lipscomb, J., & Luby, M. (1988). On the computational complexity of finding stable state vectors in connectionist models (Hopfield nets) (Technical Rep. No. 208/88). Toronto: Department of Computer Science, University of Toronto.
32. Goemans, M. X., & Williamson, D. P. (1995). Improved approximate algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM, 42(6), 1115-1145.
33. Goldmann, M., Håstad, J., & Razborov, A. (1992). Majority gates vs. general weighted threshold gates. Computational Complexity, 2(4), 277-300.
34. Goldmann, M., & Karpinski, M. (1998). Simulating threshold circuits by majority circuits. SIAM Journal on Computing, 27(1), 230-246.
35. Goldschlager, L. M., & Parberry, I. (1986). On the construction of parallel computers from various bases of Boolean functions. Theoretical Computer Science, 43(1), 43-48.
36. Goles, E. (1985). Dynamics of positive automata networks. Theoretical Computer Science, 41(1), 19-32.
37. Goles, E. (1987). Lyapunov functions associated to automata networks. In F. Fogelman-Soulié, Y. Robert, & M. Tchuente (Eds.), Automata networks in computer science - theory and applications (pp. 58-81). Manchester: Manchester University Press.
38. Goles, E., & Martínez, S. (1989). Exponential transient classes of symmetric neural networks for synchronous and sequential updating. Complex Systems, 3(6), 589-597.
39. Goles, E., & Martínez, S. (1990). Neural and automata networks: Dynamical behavior and applications. Dordrecht: Kluwer.
40. Goles, E., & Olivos, J. (1981a). Comportement periodique des fonctions a seuil binaires et applications. Discrete Applied Mathematics, 3(2), 93-105.
41. Goles, E., & Olivos, J. (1981b). The convergence of symmetric threshold automata. Information and Control, 51(2), 98-104.
42. Goles-Chacc, E., Fogelman-Soulié, F., & Pellegrin, D. (1985). Decreasing energy functions as a tool for studying threshold networks. Discrete Applied Mathematics, 12(3), 261-277.
43. Gori, M., & Meer, K. (2002). A step towards a complexity theory for analog systems. Mathematical Logic Quarterly, 48(1), 45-58.
44. Hajnal, A., Maass, W., Pudlák, P., Szegedy, M., & Turán, G. (1993). Threshold circuits of bounded depth. Journal of Computer and System Sciences, 46(2), 129-154.
45. Haken, A. (1989). Connectionist networks that need exponential time to stabilize. Unpublished manuscript, Department of Computer Science, University of Toronto.
46. Haken, A., & Luby, M. (1988). Steepest descent can take exponential time for symmetric connectionist networks. Complex Systems, 2(2), 191-196.
47. Hammer, P. L., Ibaraki, T., & Peled, U. N. (1981). Threshold numbers and threshold completions. In P. Hansen (Ed.), Studies on graphs and discrete programming, Annals of discrete mathematics, 11, Mathematics studies, 59 (pp. 125-145). Amsterdam: North-Holland.
48. Hartley, R., & Szu, H. (1987). A comparison of the computational power of neural network models. In Proceedings of the IEEE First International Conference on Neural Networks, San Diego (pp. 15-22). New York: IEEE Press.
49. Håstad, J. (1989). Almost optimal lower bounds for small depth circuits. In S. Micali (Ed.), Advances in computing research, randomness and computation, 5 (pp. 143-170). Greenwich, CT: JAI Press.
50. Håstad, J. (1994). On the size of weights for threshold gates. SIAM Journal on Discrete Mathematics 7(3), 484-492.
51. Hegedüs, T., & Megiddo, N. (1996). On the geometric separability of Boolean functions. Discrete Applied Mathematics 66(3), 205-218.
52. Hofmeister, T. (1994). Depth-efficient threshold circuits for arithmetic functions. In V. P. Roychowdhury, K.-Y. Siu, & A. Orlitsky (Eds.), Theoretical advances in neural computation and learning (pp. 37-84). Boston: Kluwer.
53. Hofmeister, T., Hohberg, W., & Köhling, S. (1991). Some notes on threshold circuits, and multiplication in depth 4. Information Processing Letters, 39(4), 219-225.
54. 0 (Res. Rep. No. 447). Dortmund: Dortmund University.
55. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554-2558.
56. Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the National Academy of Sciences USA, 81, 3088-3092.
57. Hopfield, J. J., & Tank, D. W. (1985). "Neural" computation of decision in optimization problems. Biological Cybernetics, 52(3), 141-152.
58. Horne, B. G., & Hush, D. R. (1994). On the node complexity of neural networks. Neural Networks, 7(9), 1413-1426.
59. Horne B. G., & Hush, D. R. (1996). Bounds on the complexity of recurrent neural network implementations of finite state machines. Neural Networks, 9(2), 243-252.
60. Indyk, P. (1995). Optimal simulation of automata by neural nets. In Proceedings of the 12th Annual Symposium on Theoretical Aspects of Computer Science (STACS'95), LNCS 900 (pp. 337-348). Berlin: Springer-Verlag.
61. Irmatov, A. A. (1996). Bounds for the number of threshold functions. Discrete Mathematics and Applications, 6(6), 569-583.
62. Johnson, D. S., Papadimitriou, C. H., & Yannakakis, M. (1988). How easy is local search? Journal of Computer and System Sciences, 37(1), 79-100.
63. Kahn, J., Komlós, J., & Szemerédi, E. (1995). On the probability that a random {±1}n-matrix is singular. Journal of the American Mathematical Society, 8(1), 223-240.
64. Kilian, J., & Siegelmann, H. T. (1996). The dynamic universality of sigmoidal neural networks. Information and Computation, 125(1), 48-56.
65. Kleene, S. C. (1956). Representation of events in nerve nets and finite automata. In C. E. Shannon, & J. McCarthy (Eds.), Automata studies, Annals of mathematics studies, 34 (pp. 3-41). Princeton, NJ: Princeton University Press.
66. Kohonen, T. (2001). Self-organizing maps (3rd ext. ed.). Springer series in information sciences, 30. Berlin: Springer-Verlag.
67. Koiran, P. (1994). Dynamics of discrete time, continuous state Hopfield networks. Neural Computation, 6(3), 459-468.
68. Koiran, P. (1996). A family of universal recurrent networks. Theoretical Computer Science, 168(2), 473-480.
69. Komlós, J., & Paturi, R. (1988). Convergence results in an associative memory model. Neural Networks, 1(3), 239-250.
70. Legenstein, R. A., & Maass, W. (2001). Foundations for a circuit complexity theory of sensory processing. In T. K. Leen, T. G. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems (NIPS 2000), 13 (pp. 259-265). Cambridge, MA: MIT Press.
71. Lepley, M., & Miller, G. (1983). Computational power for networks of threshold devices in asynchronous environment (Tech. Rep.). Cambridge, MA: Department of Mathematics, MIT.
72. Lipscomb, J. (1987). On the computational complexity of finding a connectionist model's stable state vectors. Unpublished master's thesis, Dept. of Computer Science, University of Toronto.
73. Lupanov, O. B. (1961). Implementing the algebra of logic functions in terms of bounded depth formulas in the basis +,*,-. Soviet Physics Doklady, 6(2), 107-108.
74. Lupanov, O. B. (1972). Circuits using threshold elements. Soviet Physics Doklady, 17(2), 91-93.
75. Maass, W. (1995). On the computational complexity of networks of spiking neurons. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems (NIPS'94), 7 (pp. 183-190). Cambridge, MA: MIT Press.
76. Maass, W. (1996a). On the computational power of noisy spiking neurons. In D. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems (NIPS'95), 8 (pp. 211-217). Cambridge, MA: MIT Press.
77. Maass, W. (1996b). Lower bounds for the computational power of networks of spiking neurons. Neural Computation, 8(1), 1-40.
78. Maass, W. (1997a). Bounds for the computational power and learning complexity of analog neural nets. SIAM Journal on Computing, 26(3), 708-732.
79. Maass, W. (1997b). Fast sigmoidal networks via spiking neurons. Neural Computation, 9(2), 279-304.
80. Maass, W. (1997c). Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9), 1659-1671.
81. Maass, W. (2000). On the computational power of Winner-Take-All. Neural Computation, 12(11), 2519-2536.
82. Maass, W., & Bishop, C. M. (Eds.) (1998). Pulsed neural networks. Cambridge, MA: MIT Press.
83. Maass, W., & Natschläger, T. (1997). Networks of spiking neurons can emulate arbitrary Hopfield nets in temporal coding. Network: Computation in Neural Systems, 8(4), 355-371.
84. Maass, W., & Natschläger, T. (2000). A model for fast analog computation based on unreliable synapses. Neural Computation, 12(7), 1679-1704.
85. Maass, W., & Orponen, P. (1998). On the effect of analog noise in discrete-time analog computations. Neural Computation, 10(5), 1071-1095.
86. Maass, W., & Ruf, B. (1999). On computation with pulses. Information and Computation, 148(2), 202-218.
87. Maass, W., & Schmitt, M. (1999). On the complexity of learning for spiking neurons with temporal coding. Information and Computation, 153(1), 26-46.
88. Maass, W., Schnitger, G., & Sontag, E. D. (1991). On the computational power of sigmoid versus Boolean threshold circuits. In Proceedings 32nd Annual Symposium on Foundations of Computer Science (FOCS'91), San Juan, Puerto Rico (pp. 767-776). New York: IEEE Press.
89. Maass, W., & Sontag, E. D. (1999). Analog neural nets with gaussian or other common noise distribution cannot recognize arbitrary regular languages. Neural Computation, 11(3), 771-782.
90. Mahajan, S., & Ramesh, H. (1999). Derandomizing approximation algorithms based on semidefinite programming. SIAM Journal on Computing, 28(5), 1641-1663.
91. McEliece, R. J., Posner, E. C., Rodemich, E. R., & Venkatesh, S. S. (1987). The capacity of the Hopfield associative memory. IEEE Transactions on Information Theory, 33(4), 461-482.
92. Minsky, M. L. (1967). Computation: Finite and infinite machines. Englewood Cliffs, NJ: Prentice Hall.
93. Minsky, M. L., & Papert, S. A. (1969). Perceptrons. Cambridge, MA: MIT Press.
94. Moore, C. (1998). Finite-dimensional analog computers: flows, maps, and recurrent neural networks. In Proceedings of the 1st International Conference on Unconventional Models of Computation (pp. 59-71). Berlin: Springer-Verlag.
95. Muroga, S. (1971). Threshold logic and its applications. New York: Wiley Interscience.
96. Muroga, S., Toda, I., & Takasu, S. (1961). Theory of majority decision elements. Journal of the Franklin Institute, 271, 376-418.
97. Nechiporuk, E. I. (1964). The synthesis of networks from threshold elements. Problemy Kibernetiki, 11, 49-62.
98. O'Neil, P. E. (1971). Hyperplane cuts of an n-cube. Discrete Mathematics, 1(2), 193-195.
99. Orponen, P. (1994). Computational complexity of neural networks: A survey. Nordic Journal of Computing, 1(1), 94-110.
100. Orponen, P. (1996). The computational power of discrete Hopfield nets with hidden units. Neural Computation, 8(2), 403-415.
101. Orponen, P. (1997a). A survey of continuous-time computation theory. In D.-Z. Du, & K.-I. Ko (Eds.), Advances in algorithms, languages, and complexity (pp. 209-224). Dordrecht: Kluwer.
102. Orponen, P. (1997b). Computing with truly asynchronous threshold logic networks. Theoretical Computer Science, 174(1-2), 123-136.
103. Orponen, P. (1997c). The computational power of continuous time neural networks. In Proceedings of the 24th Seminar on Current Trends in Theory and Practice of Informatics (SOFSEM'97), Milovy, Czech Republic, LNCS 1338 (pp. 86-103). Berlin: Springer-Verlag.
104. Orponen, P. (2000). An overview of the computational power of recurrent neural networks. In H. Hyötyniemi (Ed.), Proceedings of the 9th Finnish AI Conference STeP 2000-Millennium of AI, Espoo, Finland, "AI of Tomorrow": Symposium on Theory (Vol. 3, pp. 89-96). Vaasa, Finland: Finnish AI Society.
105. Papadimitriou, C. H. (1994). Computational complexity. Reading, MA: Addison-Wesley.
106. Parberry, I. (1990). A primer on the complexity theory of neural networks. In R. B. Banerji (Ed.), Formal techniques in artificial intelligence: A sourcebook, Studies in computer science and artificial intelligence, 6 (pp. 217-268). Amsterdam: Elsevier-North-Holland.
107. Parberry, I. (1994). Circuit complexity and neural networks. Cambridge, MA: MIT Press.
108. Parberry, I., & Schnitger, G. (1989). Relating Boltzmann machines to conventional models of computation. Neural Networks, 2(1), 56-67.
109. Poljak, S., & Sůra, M. (1983). On periodical behaviour in societies with symmetric influences. Combinatorica, 3(1), 119-121.
110. Porat, S. (1989). Stability and looping in connectionist models with asymmetric weights. Biological Cybernetics, 60, 335-344.
111. Powell, M. J. D. (1985). Radial basis functions for multivariable interpolation: A review. In J. C. Mason & M. G. Cox (Eds.), Proceedings of the IMA Conference on Algorithms for the Approximation of Functions and Data (pp. 143-167). Oxford: Oxford Science Publications.
112. Rabin, M. (1963). Probabilistic automata. Information and Control, 6(3), 230-245.
113. Razborov, A. A. (1992). On small depth threshold circuits. In O. Nurmi & E. Ukkonen (Eds.), Proceedings of the 3rd Scandinavian Workshop on Algorithm Theory (SWAT'92), Helsinki, Finland, LNCS 621 (pp. 42-52). Berlin: Springer-Verlag.
114. Reif, J. H., & Tate, S. R. (1992). On threshold circuits and polynomial computations. SIAM Journal on Computing, 21(5), 896-908.
115. Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386-408.
116. Roychowdhury, V. P., Siu, K.-Y., & Orlitsky, A. (Eds.). (1994). Theoretical advances in neural computation and learning. Boston: Kluwer.
117. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323, 533-536.
118. Savage, J. E. (1972). Computational work and time on finite machines. Journal of the ACM, 19(4), 660-674.
119. Savage, J. E. (1998). Models of computation: Exploring the power of computing. Reading, MA: Addison-Wesley.
120. Schäffer, A. A., & Yannakakis, M. (1991). Simple local search problems that are hard to solve. SIAM Journal on Computing, 20(1), 56-87.
121. Schläfli, L. (1901). Theorie der vielfachen Kontinuität. Zürich: Zürcher & Furrer.
122. Schmitt, M. (1998). On computing Boolean functions by a spiking neuron. Annals of Mathematics and Artificial Intelligence, 24(1-4), 181-191.
123. Schmitt, M. (2002). Descartes' rule of signs for radial basis function neural networks. Neural Computation, 14(12), 2997-3011.
124. Siegelmann, H. T. (1994). On the computational power of probabilistic and faulty neural networks. In S. Abiteboul & E. Shamir (Eds.), Proceedings of the 21st International Colloquium on Automata, Languages, and Programming (ICALP'94), LNCS 820 (pp. 23-34). Berlin: Springer-Verlag.
125. Siegelmann, H. T. (1996). Recurrent neural networks and finite automata. Journal of Computational Intelligence, 12(4), 567-574.
126. Siegelmann, H. T. (1999a). Neural networks and analog computation: Beyond the Turing limit. Boston: Birkhäuser.
127. Siegelmann, H. T. (1999b). Stochastic analog networks and computational complexity. Journal of Complexity, 15(4), 451-475.
128. Siegelmann, H. T., Roitershtein, A., & Ben-Hur, A. (2000). Noisy neural networks and generalizations. In S. A. Solla, T. K. Leen, & K.-R. Müller (Eds.), Advances in neural information processing systems (NIPS'92), 12 (pp. 335-341). Cambridge, MA: MIT Press.
129. Siegelmann, H. T., & Sontag, E. D. (1994). Analog computation via neural networks. Theoretical Computer Science, 131(2), 331-360.
130. Siegelmann, H. T., & Sontag, E. D. (1995). Computational power of neural networks. Journal of Computer System Science, 50(1), 132-150.
131. Šíma, J. (1995). Hopfield languages. In Proceedings of the 22nd Seminar on Current Trends in Theory and Practice of Informatics (SOFSEM'95), Milovy, Czech Republic, LNCS 1012 (pp. 461-468). Berlin: Springer-Verlag.
132. Šíma, J. (1997). Analog stable simulation of discrete neural networks. Neural Network World, 7(6), 679-686.
133. Šíma, J. (2001). The computational capabilities of neural networks (extended abstract). In Proceedings of the 5th International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA'01), Prague, Czech Republic (pp. 22-26). Vienna: Springer-Verlag.
134. Šíma, J., & Orponen, P. (2000). A continuous-time Hopfield net simulation of discrete neural networks. In Proceedings of the 2nd International ICSC Symposium on Neural Computation (NC 2000), Berlin, Germany (pp. 36-42). Wetaskiwin, Canada: ICSC Academic Press.
135. Šíma, J., & Orponen, P. (2001). Exponential transients in continuous-time symmetric Hopfield nets. In Proceedings of the 11th International Conference on Artificial Neural Networks (ICANN'01), Vienna, Austria, LNCS 2130 (pp. 806-813). Berlin: Springer-Verlag.
136. Šíma, J., & Orponen, P. (2003). Continuous-time symmetric Hopfield nets are computationally universal. Neural Computation, 15(3), 693-733.
137. Šíma, J., Orponen, P., & Antti-Poika, T. (2000). On the computational complexity of binary and analog symmetric Hopfield nets. Neural Computation, 22(12), 2965-2989.
138. Šíma, J., & Wiedermann, J. (1998). Theory of neuromata. Journal of the ACM, 45(1), 155-178.
139. Siu, K.-Y, Bruck, J., Kailath, T., & Hofmeister, T. (1993). Depth efficient neural networks for division and related problems. IEEE Transactions on Information Theory, 39(3), 946-956.
140. Siu, K.-Y., & Roychowdhury, V. P. (1994). On optimal depth threshold circuits for multiplication and related problems. SIAM Journal on Discrete Mathematics, 7(2), 284-292.
141. Siu, K.-Y., Roychowdhury, V. P., & Kailath, T. (1991). Depth-size tradeoffs for neural computation. IEEE Transactions on Computers, 40(12), 1402-1412.
142. Siu, K.-Y., Roychowdhury, V. P., & Kailath, T. (1993). Computing with almost optimal size neural networks. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems (NIPS'92), 5 (pp. 19-26). San Mateo, CA: Morgan Kaufmann.
143. Siu, K.-Y, Roychowdhury, V. P., & Kailath, T. (1994). Rational approximation techniques for analysis of neural networks. IEEE Transactions on Information Theory, 40(2), 455-466.
144. Siu, K.-Y., Roychowdhury, V. P., & Kailath, T. (1995a). Discrete neural computation: A theoretical foundation. Englewood Cliffs, NJ: Prentice Hall.
145. Siu, K.-Y., Roychowdhury, V. P., & Kailath, T. (1995b). Toward massively parallel design of multipliers. Journal of Parallel and Distributed Computing, 24(1), 86-93.
146. Šorel, M., & Šíma, J. (2000). Robust implementation of finite automata by recurrent RBF networks. In Proceedings of the 27th Seminar on Current Trends in Theory and Practice of Informatics (SOFSEM 2000), Milovy, Czech Republic, LNCS 1963 (pp. 431-439). Berlin: Springer-Verlag.
147. Tanaka, F., & Edwards, S. F. (1980). Analytic theory of the ground state properties of a spin glass: I. Ising spin glass. Journal of Physics F: Metal Physics, 10, 2769-2778.
148. Tchuente, M. (1986). Sequential simulation of parallel iterations and applications. Theoretical Computer Science, 48(2-3), 135-144.
149. Vollmer, H. (1999). Introduction to circuit complexity. Berlin: Springer-Verlag.
150. von Neumann, J. (1956). Probabilistic logics arid the synthesis of reliable organisms from unreliable components. In C. E. Shannon & J. McCarthy (Eds.), Automata studies, Annals of mathematics studies, 34 (pp. 43-98). Princeton, NJ: Princeton University Press.
151. Wegener, I. (1987). The complexity of Boolean functions. Chichester: Wiley/Teubner. Available on-line at http://www.eccc.uni-trier.de/eccc-local/ ECCC-Books/wegener_book_readme.html.
152. Wegener, I. (1993). Optimal lower bounds on the depth of polynomial-size threshold circuits for some arithmetic functions. Information Processing Letters, 46(2), 85-87.
153. Werbos, P. J. (1974). Beyond regression: New tools for prediction and analysis in the behavioral sciences. Unpublished doctoral dissertation, Harvard University.
154. Wiedermann, J. (1994). Complexity issues in discrete neurocomputing. Neural Network World, 4(1), 99-119.
155. Yao, A. C.-C. (1985). Separating the polynomial-time hierarchy by oracles. Proceedings of the 26th Annual Symposium on the Foundations of Computer Science (FOCS'85), Portland, Oregon (pp. 420-428). New York: IEEE Computer Society.
156. Yuille, A. L., & Geiger, D. (2003). Winner-take-all networks. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (2nd ed., pp. 1228-1231). Cambridge, MA: MIT Press.