Large memory capacity in chaotic artificial neural networks: A view of the anti-integrable limit

IEEE Transactions on Neural Networks

Volumn 20, Issue 8, 2009, Pages 1340-1351

  Lin, Wei ^a,b   Chen, Guanrong ^c  

^a FUDAN UNIVERSITY   (China)

^b MAX PLANCK INSTITUTE   (Germany)

^c CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Anti integrable limit; Artificial neural network; Chaos; Periodic activation function

Indexed keywords

ACTIVATION FUNCTIONS; ANTI-INTEGRABLE LIMIT; ARTIFICIAL NEURAL NETWORK; ARTIFICIAL NEURAL NETWORK MODELS; CHAOS; CHAOTIC DYNAMICS; CHAOTIC MODEL; CONTROL SCHEMES; MEMORY CAPACITY; NEURAL NETWORK MODEL; PERIODIC ACTIVATION FUNCTION; SIGMOIDAL FUNCTIONS; SINUSOIDAL FUNCTIONS; STORED PATTERN;

BACKPROPAGATION;

NEURAL NETWORKS;

ACTION POTENTIAL; ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; NERVE CELL; PERIODICITY; PHYSIOLOGY; TIME;

ACTION POTENTIALS; ALGORITHMS; COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER); NEURONS; PERIODICITY; TIME FACTORS;

EID: 68949205553     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2024148     Document Type: Article

References (61)

1. J. J. Hopfield, "Neural networks and physical systems with emergent collective computation abilities," Proc. Nat. Acad. Sci. USA, vol. 79, pp. 2554-2558, 1982.
2. J. J. Hopfield, D. I. Feinstein, and R. G. Palmer, "'Unlearning' has a stabilizing effect in collective memories," Nature, vol. 304, pp. 158-159, 1983.
3. T. Kohonen, "Self-organized formation of topologically correct feature maps," Biol. Cybern., vol. 43, pp. 59-69, 1982.
4. M. H. Hassoun, Fudamentals of Artificial Neural Networks. Cambridge, MA: MIT Press, 1995.
5. H. G. Schuster, Complex Adaptive Systems. Saabrücken, Germany: Springer-Verlag, 2001.
6. S. Amari and K. Maginu, "Statistical neurodynamics of associative memory," Neural Netw., vol. 2, pp. 63-73, 1988.
7. S. Amari, "Characteristics of sparsely encoded associative memories," Neural Netw., vol. 3, pp. 451-457, 1989.
8. D. J. Amit, H. Gutfreund, and H. Sompolinsky, "Storing infinite number of patterns in a spin-glass odel of neural networks," Phys. Rev. Lett., vol. 55, pp. 1530-1533, 1985.
9. T. Kohonen, Self-Organization and Associative Memory. Berlin, Germany: Springer-Verlag, 1988.
10. S. Nara, P. Davis, M. Kawachi, and H. Totuji, "Memory search using complex dynamics in a recurrent neural network model," Neural Netw. vol. 6, pp. 963-973, 1993.
11. R. J. McEliece, E. C. Posner, E. R. Rodemich, and S. S. Venkatesh, "The capacity of the Hopfield associative memory," IEEE Trans. Inf. Theory, vol. IT-33, no. 4, pp. 461-482, Jul. 1987.
12. M. Morita, "Associative memory with nonmonotone dynamics," Neural Netw., vol. 6, pp. 115-126, 1993.
13. S. Yoshizawa, M. Morita, and S. Amari, "Capacity of associative memory using a nonmonotonic neuron model," Neural Netw., vol. 6, pp. 167-176, 1993.
14. H.-F. Yanai and S. Amari, "Auto-associative memory with two-stage dynamics of nonmonotonic neurons," IEEE Trans. Neural Netw., vol. 7, no. 4, pp. 803-815, Jul. 1996.
15. I. Kanter and E. Eisenstein, "Neural networks and the solution of nonlinear equations," Phys. Rev. Lett., vol. 65, pp. 520-523, 1990.
16. P. N. McGraw and M. Menzinger, "Bistable gradient networks. I. Attractors and pattern retrieval at low loading in the thermodynamic limit," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 67, 2003, 016118.
17. H. Zhao, "Designing asymmetric neural networks with associative memory," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 70, 2004, 066137.
18. H. Degn, A. V. Holden, and L. F. Olsen, Eds., Chaos in Biological Systems. New York: Plenum, 1987.
19. S. H. Strogatz, Nonlinear Dynamics and Chaos, with Applications to Physics, Biology, Chemistry and Engineering. Cambridge, MA: Perseus Books, 1994.
20. D. J. Albers and J. C. Sprott, "Routes to chaos in high-dimensional dynamical systems: A qualitative numerical study," Physica D, vol. 223, pp. 194-207, 2006.
21. D. J. Albers and J. C. Sprott, "Structural stability and hyperbolicity violation in high-dimensional dynamical systems," Nonlinearity, vol. 19, pp. 1801-1847, 2006.
22. D. J. Albers, J. C. Sprott, and J. P. Crutchfield, "Persistent chaos in high dimensions," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 74, 2006, 057201.
23. B. Cessac, B. Doyon, M. Quoy, and M. Samuelides, "Mean-field equations, bifurcation map and route to chaos in discrete time neural networks," Physica D, vol. 74, pp. 24-44, 1994.
24. H. Sompolinsky, A. Crisanti, and H. J. Sommers, "Chaos in random neural networks," Phys. Rev. Lett., vol. 61, pp. 259-262, 1988.
25. K. Aihara, T. Takabe, and M. Toyoda, "Chaotic neural networks," Phys. Lett. A, Gen. Phys., vol. 144, pp. 333-340, 1990.
26. M. Adachi and K. Aihara, "Associative dynamics in a chaotic neural networks," Neural Netw., vol. 10, pp. 83-98, 1997.
27. I. Tsuda, "Dynamic link of memory: Chaotic memory map in nonequilibrium neural networks," Neural Netw., vol. 5, pp. 313-326, 1992.
28. I. Tsuda, "Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems," Behav. Brain Sci., vol. 24, pp. 793-847, 2001.
29. S. Nara and P. Davis, "Chaoticwandering and search in a cycle memory neural network," Prog. Theor. Phys., vol. 88, pp. 845-855, 1992.
30. L. Chen and K. Aihara, "Chaos and asymptotical stability in discretetime neural networks," Physica D, vol. 104, pp. 286-325, 1997.
31. L. Chen and K. Aihara, "Strange attractors in chaotic neural networks," IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol. 47, no. 10, pp. 1455-1468, Oct. 2000.
32. L. Chen and K. Aihara, "Chaotic simulated annealing by a neural network model with transient chaos," Neural Netw., vol. 8, pp. 199-223, 1995.
33. I. Tokuda, T. Nagashima, and K. Aihara, "Global bifurcation structure of chaotic neural networks and its application to traveling salesman problems," Neural Netw., vol. 10, pp. 1675-1690, 1997.
34. L. Chen and K. Aihara, "Global searching ability of chaotic neural networks," IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol. 48, no. 8, pp. 974-993, Aug. 1999.
35. S. Nara, "Can potentially useful dynamics to solve complex problems emerge from constrained chaos and/or chaotic itinerancy?," Chaos, vol. 13, pp. 1110-1121, 2003.
36. Y. Li and S. Nara, "Novel tracking function of moving target using chaotic dynamics in a recurrent neural network model," Cogn. Neurodyn., vol. 2, pp. 39-48, 2008.
37. M. Kushibe, Y. Liu, and J. Ohtsubo, "Associative memory with spatiotemporal chaos control," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 53, pp. 4502-4508, 1996.
38. Z. Tan and M. K. Ali, "Pattern recognition in a neural network with chaos," Phys. Rev. E, Stat. Phys. Plasmas Fluids Relat. Interdiscip. Top., vol. 58, pp. 3649-3652, 1998.
39. G. He, Z. Cao, H. Chen, and P. Zhu, "Controlling chaos in a chaotic neural network based on phase space constraint," Int. J. Mod. Phys. B, vol. 17, pp. 4209-4214, 2003.
40. G. He, M. D. Shrimali, and K. Aiharaa, "Threshold control of chaotic neural network," Neural Netw., vol. 21, pp. 114-121, 2008.
41. M. Nakagawa, "A parameter controlled chaos neural network," J. Phys. Soc. Jpn., vol. 65, pp. 1859-1867, 1996.
42. M. Nakagawa, "A novel chaos neuron model with a periodic mapping," J. Phys. Soc. Jpn., vol. 66, pp. 263-267, 1997.
43. M. Nakagawa, "Statistical properties of chaos associative memory," J. Phys. Soc. Jpn., vol. 71, pp. 2316-2325, 2002.
44. S. Aubry and A. Abramovici, "Chaotic trajectories in the standard map, the concept of anti-integrability," Physica D, vol. 43, pp. 199-219, 1990.
45. R. S. MacKay and J. D. Meiss, "Cantori for symplectic maps near the anti-integrable limit," Nonlinearity, vol. 5, pp. 149-160, 1992.
46. R. S. Mackay and S. Aubry, "Proof of existence of breather for time-reversible or Hamiltonian networks of weakly coupled oscillators," Nonlinearity, vol. 7, pp. 1623-1643, 1994.
47. S. Aubry, "Breathers in nonlinear lattices: Existence, linear stability and quantization," Physica D, vol. 103, pp. 201-250, 1997.
48. D. Sterling and J. D. Meiss, "Computing periodic orbits using the antiintegrable limit," Phys. Lett. A, Gen. Phys., vol. 241, pp. 46-52, 1998.
49. D. Sterling, H. R. Dullin, and J. D. Meiss, "Homoclinic bifurcations for the hénon map," Physica D, vol. 134, pp. 153-184, 1999.
50. Y.-C. Chen, "Bernoulli shift for second order recurrence relations near the anti-integrable limit," Discrete Continuous Dyn. Syst. B, vol. 5, pp. 587-598, 2005.
51. Y. Zheng, G. Chen, and Z. Liu, "On chaotification of discrete systems," Int. J. Bifur. Chaos, vol. 13, pp. 3443-3447, 2003.
52. E. Zeidler, "Nonlinear Functional Analysis and Its Applications," in Fixed-Point Theorems. Berlin, Germany: Springer-Verlag, 1986, vol. I.
53. C. Robinson, Dynamical Systems: Stability, Symbolic Dynamics, and Chaos. Boca Raton, FL: CRC Press, 1998.
54. D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding. Cambridge, U.K.: Cambridge Univ. Press, 1995.
55. W. Lin, "Some theoretical problems in complex systems and their applications," Ph.D. dissertation, Inst. Math., Fudan Univ, Shanghai, China, 2002.
56. W. Lin and G. Chen, "Heteroclinical repellers imply chaos," Int. J. Bifur. Chaos, vol. 16, pp. 1471-1489, 2006.
57. W. Lin, J.Wu, and G. Chen, "Generalized snap-back repeller and semiconjugacy to shift operators of piecewise continuous transformations," Discrete Continuous Dyn. Syst., vol. 19, pp. 103-119, 2007.
58. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to The Theory of Neural Computation. Reading, MA: Addison-Wesley, 1991.
59. A. Engel and C. Van den Broeck, Statistical Mechanics of Learning. Cambridge, U.K.: Cambridge Univ. Press, 2001.
60. P. Dayan and L. F. Abbott, Theoretical Neuroscience. Cambridge, MA: MIT Press, 2005.
61. P. P. Mitra and H. Bokil, Observed Brain Dynamics. Oxford, U.K.: Oxford Univ. Press, 2008.