Noise-enhanced categorization in a recurrently reconnected neural network

Physical Review E - Statistical, Nonlinear, and Soft Matter Physics

Volumn 71, Issue 3, 2005, Pages

  Monterola, Christopher ^a   Zapotocky, Martin ^a  

^a MAX PLANCK INSTITUT FÜR PHYSIK KOMPLEXER SYSTEME   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATIVE MEMORY NETWORK; NOISE-ENHANCED CATEGORIZATION; RECONNECTED NETWORK (RNN); STOCHASTIC RESONANCE;

DYNAMICS; MAPPING; NEURAL NETWORKS; RANDOM PROCESSES; RESONANCE;

SPURIOUS SIGNAL NOISE;

EID: 41349111222     PISSN: 15393755     EISSN: 15502376     Source Type: Journal    
DOI: 10.1103/PhysRevE.71.036134     Document Type: Article

References (36)

1. L. Gammaitoni, P. Hänggi, P. Jung, and F. Marchesoni, Rev. Mod. Phys. 70, 223 (1998).
2. P. Dayan and L. F. Abbot, Theoretical Neuroscience: Computational and mathematical modeling of neural systems (MIT Press, Cambridge, MA, 2001).
3. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
4. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
5. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
6. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
7. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
8. G. Wenning and K. Obermayer, Phys. Rev. Lett. 90, 120602 (2003); S. Laughlin and T. Sejnowski, Science 301, 1870 (2003); T. Hoch, G. Wenning and C. Obermayer, Phys. Rev. E 68, 011911 (2003); B. Kosko and S. Mitaim, Neural Networks 16, 755 (2003); M. Chacron, A. Longtin, and L. Maler, Network Comput. Neural Syst. 14, 803 (2003); I. Goychuk and P. Hänggi, Phys. Rev. E 69, 021104 (2004).
9. C. Monterola and C. Saloma, Phys. Rev. Lett. 89, 188102 (2002).
10. S. K. Han, W. S. Kim, and H. Kook, Phys. Rev. E 58, 2325 (1998); T. Kanamaru and Y. Okabe, ibid. 62, 2629 (2000).
11. S. K. Han, W. S. Kim, and H. Kook, Phys. Rev. E 58, 2325 (1998); T. Kanamaru and Y. Okabe, ibid. 62, 2629 (2000).
12. Z. Gingl, L. B. Kiss, and F. Moss, Europhys. Lett. 23, 191 (1995).
13. L. Gammaitoni, Phys. Rev. E 52, 4691 (1995).
14. 1/2N [9]. The capacity per neuron, C/N(J+1), reaches the maximum value of 3.8 N between J=2 and J=3.)
15. R. Urbanczik, J. Phys. A 30, L387 (1997).
16. H=1/[1 +exp(-x)] are given at http://www.mpipks-dresden.mpg.de/~chris. In this case, the period-two limit cycle of the RNN (Sec. III A.) is replaced by a fixed point x -0.
17. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. (Prentice-Hall, New York, 1999).
18. 2 where 〉 ···〉 designates averaging over the state (note that F=2Q-C) [13].
19. F. Huck et al., J. Opt. Soc. Am. A 2, 1644(1985).
20. jk} is listed at http://www.mpipks-dresden.mpg.de/~chris.
21. t at the previous time step was uniform. This is a property of the evolved weights.
22. 6000) taken from the GNU scientific library with seed randomized by the CPU clock. Results analogous to those reported for UWN noise were also obtained for Gaussian-distributed noise.
23. The behavior described in the main text was obtained in 93% of our runs for weak noise [α<0.16 (Δα=0.01)], 81% of runs for moderate noise [0.16≤α<0.29 (Δα=0.01)], and 87% of runs for strong noise [0.29 ≤α≤1.0 (Δα=0.05)]. Ten runs were performed at each noise strength.
24. k and the corresponding eigenvectors (in a Mathematica 5.0 notepad) can be viewed at http://www.mpipks-dresden.mpg.de~chris.
25. H. Tong, Non-linear Time Series: A dynamical system approach (Clarendon Press, Oxford, 1990).
26. 25 have settled to fixed points other than the states listed above.
27. K. Mori, H. Nagao, and Y. R Sasaki, Network 9, R79 (1998).
28. J. P. Rospars et al., Brain Res. 662, 31 (1994).
29. This will also be the case when the sensory stimulus is extended in time and the sensory neurons rapidly adapt.
30. While we evaluated the success rate of recovery of weak signals in Fig. 4(b) after 1000 iterations, in most cases fewer than 20 iterations are required to converge to the correct memory state.
31. D. Krone et al., Phytother Res. 15, 135 (2001).
32. C. Brody and J. J. Hopfield, Neuron 37, 843 (2003).
33. T. Watkin and A. Rau, Rev. Mod. Phys. 65, 499 (1993).
34. J. Hertz, A. Krogh, and R. Palmer, Introduction to the Theory of Neural Computation (Addison-Wesley, Cambridge, MA, 1991).
35. Z. Lou, Neural Comput. 3, 226 (1991); C. Monterola et al., J. Forecast 21, 435 (2002).
36. Z. Lou, Neural Comput. 3, 226 (1991); C. Monterola et al., J. Forecast 21, 435 (2002).