A simple derivation of a bound on the perceptron margin using singular value decomposition

Neural Computation

Volumn 23, Issue 8, 2011, Pages 1935-1943

  Barak, Omri ^a   Rigotti, Mattia ^a,b  

^a NewYork Presbyterian Hospital   (United States)

^b NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 79959677882     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00152     Document Type: Article

References (23)

1. Abbott, L., & Kepler, T. (1989a). Optimal learning in neural network memories. Journal of Physics A: Mathematical and General, 22, 2031-2038.
2. Abbott, L., & Kepler, T. (1989b). Universality in the space of interactions for network models. Journal of Physics A: Mathematical and General, 22, L711-L717.
3. Anlauf, J. K., & Biehl, M. (1989). The adatron: An adaptive perceptron algorithm. EPL (Europhysics Letters), 10, 687-692.
4. Brader, J. M., Senn, W., & Fusi, S. (2007). Learning real-world stimuli in a neural networkwith spike-driven synaptic dynamics. Neural Comput., 19(11), 2881-2912.
5. Brunel, N., Hakim, V., Isope, P., Nadal, J.-P., & Barbour, B. (2004). Optimal information storage and the distribution of synaptic weights: Perceptron versus Purkinje cell. Neuron, 43(5), 745-757.
6. Fletcher, R. (1988). Practical methods of optimization. Hoboken, NJ: Wiley.
7. Forrest, B. M. (1988). Content-addressability and learning in neural networks. J. Phys. A: Math. Gen., 21, 245-255.
8. Freund, Y., & Schapire, R. (1999). Large margin classification using the perceptron algorithm. Machine Learning, 37(3), 277-296.
9. Gardner, E. (1988). The space of interactions in neural network models. J. Phys. A, 21, 257-270.
10. Gardner, E., & Derrida, B. (1988). Optimal storage properties of neural network models. J. Phys. A: Math. Gen., 21, 271-284.
11. Gerl, F., & Krey, U. (1994). Storage capacity and optimal learning of Potts-model perceptrons by a cavity method. Journal of Physics A:Mathematical and General, 27, 7353-7372.
12. Kanter, I., & Sompolinsky, H. (1987). Associative recall of memory without errors. Phys. Rev. A, 35(1), 380-392.
13. Kepler, T., & Abbott, L. (1988). Domains of attraction in neural networks. Journal de Physique, 49(10), 1657-1662.
14. Korzen, M., & Klesk, P. (2008). Maximal margin estimation with perceptron-like algorithm. In Artificial Intelligence and Soft Computing ICAISC 2008 (pp. 597-608). Berlin: Springer.
15. Krauth,W.,&Mézard, M. (1987).Learning algorithms with optimal stability in neural networks. Journal of Physics A: Mathematical and General, 20(11), L745-L752.
16. Mézard, M., Parisi, G., & Virasoro, M. (1987). Spin glass theory and beyond. Singapore: World Scientific.
17. Minsky, M., & Papert, S. (1969). Perceptrons. Cambridge, MA: MIT Press.
18. Novikoff, A. (1962). On convergence proofs on perceptrons. In Proceedings of the Symposium on the Mathematical Theory of Automata (pp. 615-622). Hoboken, NJ: Wiley.
19. Personnaz, L., Guyon, I., & Dreyfus, G. (1985). Information storage and retrieval in spin-glass like neural networks. J. Physique Lett., 46(8), 359-365.
20. Rigotti, M., Ben Dayan Rubin, D. D., Wang, X.-J., & Fusi, S. (2010). Internal representation of task rules by recurrent dynamics: The importance of the diversity of neural responses. Frontiers in Computational Neuroscience, 4(24), 29.
21. Rosenblatt, F. (1962). Principles of neurodynamics. New York: Spartan Books.
22. Tarkowski,W., Komarnicki, M., & Lewenstein, M. (1991). Optimal storage of invariant sets of patterns in neural network memories. Journal of Physics A:Mathematical and General, 24, 4197-4217.
23. Tarkowski, W., & Lewenstein, M. (1992). Estimates of optimal storage conditions in neural network memories based on random matrix theory. Journal of Physics A: Mathematical and General, 25, 6251-6264.