Covering Cubes by Random Half Cubes, with Applications to Binary Neural Networks

Journal of Computer and System Sciences

Volumn 56, Issue 2, 1998, Pages 223-252

  Kim, Jeong Han ^a   Roche, James R ^b  

^a MICROSOFT RESEARCH   (United States)

^b CENTER FOR COMMUNICATIONS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MATHEMATICAL MODELS; NEURAL NETWORKS; STATISTICAL MECHANICS; VECTORS;

RANDOM HALF CUBES;

RANDOM PROCESSES;

EID: 0032047151     PISSN: 00220000     EISSN: None     Source Type: Journal    
DOI: 10.1006/jcss.1997.1560     Document Type: Article

References (38)

1. N. Alon and J. Spencer, "The Probabilistic method," Wiley, New York, 1992.
2. E. Barkai and I. Kanter, Storage capacity of a multilayer neural network with binary weights, Europhys. Lett. 14 (1991), 107-112.
3. B. Bollobás, "Random Graphs," Academic Press, London, 1985.
4. L. Breiman, "Probability," Addison-Wesley, Reading, MA, 1968.
5. T. M. Cover, Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE Trans. Electronic Comput. 14 (1965), 326-331.
6. S. Fang and S. Venkatesh, On the average tractability of binary integer programming and the curious transition to perfect generalization in learning majority functions, in "Proceedings of the 6th Workshop on Computational Learning Theory," Morgan Kaufman, San Mateo, CA, 1993.
7. S. Fang and S. Venkatesh, The capacity of majority rule, Random Structures and Algorithms 12 (1998), 83-109.
8. S. Fang and S. Venkatesh, Learning binary perceptrons perfectly efficiently, J. Comput. System Sci. 52 (1996), 374-389.
9. Z. Füredi, Random polytopes in the d-dimensional cube, Discrete Comput. Geom. 1 (1986), 315-319.
10. E. Gardner, Maximum storage capacity in neural networks, Europhys. Lett. 4 (1987), 481-485.
11. E. Gardner, The space of interactions in neural network models, J. Phys. A 21 (1988), 257-270.
12. E. Gardner and B. Derrida, Optimal storage properties of neural network models, J. Phys. A 21 (1988), 271-284.
13. M. Golea and M. Marchand, Average case analysis of the clipped Hebb rule for nonoverlapping perceptron networks, in "Proceedings of the 6th Workshop on Computational Learning Theory," pp. 151-157, Morgan Kaufman, San Mateo, CA, 1993.
14. G. Gyorgyi, First-order transition to perfect generalization in a neural network with binary synapses, Phys. Rev. A 41 (1990), 7097-7100.
15. D. Haussler, M. Kearns, H. S. Seung, and N. Tishby, Rigorous learning curve bounds from statistical mechanics, in "Proceedings of the 7th Workshop on Computational Learning Theory," pp. 76-87, Morgan Kaufman, San Mateo, CA, 1994.
16. D. O. Hebb, "The Organization of Behavior," Wiley, New York, 1949.
17. G. E. Hinton and J. A. Anderson (Eds.), "Parallel Models of Associative Memory," Erlbaum, Hillsdale, NJ, 1981.
18. J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Nat. Acad. Sci. U.S.A. 79 (1982), 2554-2558.
19. J. Kahn, J. Komlós, and E. Szemerédi, Singularity probability for random ± 1 matrices, preprint, 1993.
20. H. Köhler, S. Diederich, W. Kinzel, and M. Opper, Learning algorithm for a neural network with binary synapses, J. Phys. B 78 (1990), 333-342.
21. T. Kohonen, "Self-Organization and Association Memory," Springer-Verlag, New York, 1984.
22. T. Kohonen, State of the art in neural computing, in "IEEE First International Conference on Neural Networks I," pp. 77-90, 1987.
23. J. Komlós, On the determinant of (0, 1) matrices, Studia Sci. Math. Hung. 2 (1967), 7-21.
24. J. Komlós and R. Paturi, Convergence results in an associative memory model, Neural Networks 1 (1988), 239-250.
25. W. Krauth and M. Mézard, Storage capacity of memory networks with binary couplings, J. Physique 50 (1989), 3057-3066.
26. W. Krauth and M. Opper, Critical storage capacity of the j = ± 1 neural network, J. Phys. A: Math. Gen. 22 (1989), L519-L523.
27. Y. S. Lashley, "The Neurophysiology of Lashley: Selected Papers of K. S. Lashley," McGraw-Hill, New York, 1960.
28. W. A. Little, The existence of persistent states in the brain, Math. Biosci. 19 (1974), 101-119.
29. W. A. Little and G. L. Shaw, Analytic study of the memory storage capacity of a neural network, Math. Biosci. 39 (1978), 281-290.
30. G. Mato and N. Parga, Generalization properties of multilayered neural networks, J. Phys. A: Math. Gen. 25 (1992), 5047-5054.
31. W. S. McCulloch and W. H. Pitts, A logical calculus for ideas immanent in nervous activity, Bull. Math. Biophys. 5 (1943), 115-133.
32. P. Moran, "An Introduction to Probability Theory," Oxford Univ. Press, Oxford, UK, 1968.
33. H. S. Seung, H. Sompolinsky, and N. Tishby, Statistical mechanics of learning from examples, Phys. Rev. A 45 (1992), 6056-6091.
34. S. Venkatesh, Computation and learning in the context of neural network capacity, in "Neural Networks for Perception" (H. Wechsler, Ed.), Academic Press, New York, 1991.
35. S. Venkatesh, On learning binary weights for majority functions, in "Proceedings of the 4th Workshop on Computational Learning Theory," pp. 257-266, Morgan Kaufman, San Mateo, CA, 1991.
36. S. Venkatesh, The science of making ERORS: What error tolerance implies for capacity in neural networks, IEEE Trans. Knowledge and Data Eng. 4 (1992), 135-144.
37. J. Wendel, A problem in geometric probability, Math. Scand. 11 (1962), 109-111.
38. J. M. Wozencraft and I. M. Jacobs, "Principles of Communication Engineering," Wiley, New York, 1965.