Sparse neural networks with large learning diversity

IEEE Transactions on Neural Networks

Volumn 22, Issue 7, 2011, Pages 1087-1096

  Gripon, Vincent ^a,b   Berrou, Claude ^a,b  

^a INSTITUT MINES TELECOM   (France)

^b LAB STICC   (France)

Author keywords

Associative memory; capacity; classification; clique; diversity; error correcting code; learning machine; recurrent neural network; sparse coding

Indexed keywords

ASSOCIATIVE MEMORIES; CAPACITY; CLIQUE; DIVERSITY; ERROR CORRECTING CODE; LEARNING MACHINES; SPARSE CODING;

ASSOCIATIVE PROCESSING; ASSOCIATIVE STORAGE; RECURRENT NEURAL NETWORKS;

NETWORK CODING;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION; HUMAN; LEARNING; NERVE CELL; PHYSIOLOGY; RECALL;

COMPUTER SIMULATION; HUMANS; LEARNING; MENTAL RECALL; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS;

EID: 79960111862     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2146789     Document Type: Article

References (19)

1. J. Bruck and M. Blaum, "Neural networks, error-correcting codes, and polynomials over the binary n-cube," IEEE Trans. Inf. Theory, vol. 35, no. 5, pp. 976-987, Sep. 1989. (Pubitemid 20630579)
2. T. G. Dietterich and G. Bakiri, "Solving multiclass learning problems via error-correcting output codes," J. Artif. Intell. Res., vol. 2, no. 1, pp. 263-286, Aug. 1994.
3. C. Berrou and V. Gripon, "Coded Hopfield networks," in Proc. 6th Int. Symp. Turbo Codes Iterat. Inf. Process., Brest, France, Sep. 2010, pp. 1-5.
4. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proc. Nat. Acad. Sci., Biophys., vol. 79, no. 8, pp. 2554-2558, Apr. 1982.
5. C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1," in Proc. IEEE Int. Conf. Commun., vol. 2. Geneva, Switzerland, May 1993, pp. 1064-1070. (Pubitemid 23674800)
6. R. Gallager, "Low-density parity-check codes," IEEE Trans. Inf. Theory, vol. 8, no. 1, pp. 21-28, Jan. 1962.
7. D. J. MacKay and R. M. Neal, "Good codes based on very sparse matrices," in Proc. 5th IMA Conf. Cryptography Coding, Berlin, Germany, 1995, pp. 100-111. (Pubitemid 126019851)
8. B. A. Olshausen and D. J. Field, "Sparse coding of sensory inputs," Current Opin. Neurobiol., vol. 14, no. 4, pp. 481-487, Aug. 2004. (Pubitemid 39078497)
9. G. J. Rinkus, "A cortical sparse distributed coding model linking miniand macrocolumn-scale functionality," Frontiers Neuroanat., vol. 4, no. 17, pp. 1-13, Jun. 2010.
10. R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, "The capacity of the Hopfield associative memory," IEEE Trans. Inf. Theory, vol. 33, no. 4, pp. 461-482, Jul. 1987.
11. A. Storkey, "Increasing the capacity of a Hopfield network without sacrificing functionality," in Proc. Int. Conf. Artif. Neural Netw., Lausanne, Switzerland, Oct. 1997, pp. 451-456. (Pubitemid 127140276)
12. D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzmann machines," Cognit. Sci., vol. 9, no. 1, pp. 147-169, Jan.-Mar. 1985.
13. F. Schwenker, F. T. Sommer, and G. Palm, "Iterative retrieval of sparsely coded associative memory patterns," Neural Netw., vol. 9, no. 3, pp. 445-455, Apr. 1996. (Pubitemid 26170414)
14. L. Lin, R. Osan, and J. Z. Tsien, "Organizing principles of real-time memory encoding: Neural clique assemblies and universal neural codes," Trends Neurosci., vol. 29, no. 1, pp. 48-57, Jan. 2006. (Pubitemid 43063130)
15. F. J. MacWilliams and N. J. A. Sloane, The Theory of Error-Correcting Codes. Amsterdam, The Netherlands: North Holland, 1979, pp. 526-527.
16. J.-F. Yang and C.-M. Chen, "Winner-take-all neural networks using the highest threshold," IEEE Trans. Neural Netw., vol. 11, no. 1, pp. 194-199, Jan. 2000. (Pubitemid 32132899)
17. T. Kohonen, "Self-organized formation of topologically correct feature maps," in Neurocomputing: Foundations of Research. Cambridge, MA: MIT Press, 1988, pp. 509-521.
18. M. E. J. Newman, "The structure and function of complex networks," SIAM Rev., vol. 45, no. 2, pp. 167-256, 2003.
19. J. Chen and N. S. Chaudhari, "Segmented-memory recurrent neural networks," IEEE Trans. Neural Netw., vol. 20, no. 8, pp. 1267-1280, Aug. 2009.