Selective positive-negative feedback produces the winner-take-all competition in recurrent neural networks

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 2, 2013, Pages 301-309

  Li, Shuai ^a   Liu, Bo ^b   Li, Yangming ^c  

^a Stevens Institute of Technology   (United States)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Competition; Nonlinear; Recurrent neural networks; Selective positive negative feedback; Winner take all (WTA)

Indexed keywords

COMPETITION; MATHEMATICAL MODELS; NONLINEAR EQUATIONS; RECURRENT NEURAL NETWORKS;

COMPETITION MECHANISM; CONVERGENCE BEHAVIORS; DYNAMIC COMPETITIONS; DYNAMIC EQUATIONS; NON-LINEAR PHENOMENA; NONLINEAR; NUMERICAL RESULTS; WINNER TAKE ALLS;

FEEDBACK;

ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; FEEDBACK SYSTEM;

FEEDBACK; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 84886865748     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2230451     Document Type: Article

References (35)

1. E. A. Dun, B. J. Ferguson, and C. A. Beveridge, "Apical dominance and shoot branching. divergent opinions or divergent mechanisms?" Plant Physiol., vol. 142, no. 3, pp. 812-819, 2006 (Pubitemid 44764610)
2. R. H. Frank and P. J. Cook, The Winner-Take-All Society: Why the Few at the Top Get so Much More Than the Rest of Us. London, U.K.: Penguin Books, 2010
3. D. K. Lee, L. Itti, C. Koch, and J. Braun, "Attention activates winnertake- all competition among visual filters," Nature Neurosci., vol. 2, no. 4, pp. 375-81, 1999 (Pubitemid 30491282)
4. L. Clark, R. Cools, and T. W. Robbins, "The neuropsychology of ventral prefrontal cortex: Decision-making and reversal learning," Brain Cognit., vol. 55, no. 1, pp. 41-53, 2004 (Pubitemid 38591429)
5. S. Kurt, A. Deutscher, J.M. Crook, F.W. Ohl, E. Budinger, C.K. Moeller, H. Scheich, and H. Schulze, "Auditory cortical contrast enhancing by global winner-take-all inhibitory interactions," PLoS ONE, vol. 3, no. 3, p. 12, 2008
6. M. Enquist and S. Ghirlanda, Neural Networks and Animal Behavior. Princeton, NJ: Princeton Univ. Press, Aug. 2005
7. D. Dubnau, "Winner takes all in a race for cell fate," Molecular Syst. Biol., vol. 7, p. 558, Dec. 2011
8. L. Almeida, M. Idiart, and J. E. Lisman, "A second function of gamma frequency oscillations: An E%-max winner-take-all mechanism selects which cells fire," J. Neurosci., vol. 29, no. 23, pp. 7497-7503, 2009
9. C. Benkert and D. Z. Anderson, "Controlled competitive dynamics in a photorefractive ring oscillator: 'Winner-takes-all' and the 'votingparadox' dynamics," Phys. Rev. A, vol. 44, no. 7, pp. 4633-4638, 1991
10. H. G. Emilio, C. Lopez, S. Pigolotti, and K. H. Andersen, "Species competition: Coexistence, exclusion and clustering," Philosoph. Trans. Royal Soc. A, Math. Phys. Eng. Sci., vol. 367, no. 3, pp. 3183-3195, 2008
11. W. Wang and J. E. Slotine, "Fast computation with neural oscillators," Neurocomputing, vol. 69, nos. 16-18, pp. 2320-2326, 2006 (Pubitemid 44340581)
12. M. Oster, R. Douglas, and S. Liu, "Computation with spikes in a winnertake- all network," Neural Comput., vol. 21, pp. 2437-2465, Sep. 2009
13. U. Rutishauser, R. J. Douglas, and J. E. Slotine, "Collective stability of networks of winner-take-all circuits," Neural Comput., vol. 23, no. 3, pp. 735-773, 2011
14. Z. Xu, H. Jin, K. S. Leung, Y. Leung, and C. K. Wong, "An automata network for performing combinatorial optimization," Neurocomputing, vol. 47, nos. 1-4, pp. 59-83, 2002 (Pubitemid 36221474)
15. S. Liu and J. Wang, "A simplified dual neural network for quadratic programming with its KWTA application," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1500-1510, Nov. 2006 (Pubitemid 44824263)
16. X. Hu and J. Wang, "An improved dual neural network for solving a class of quadratic programming problems and its κ-winners-take-all application," IEEE Trans. Neural Netw., vol. 19, no. 12, pp. 2022-2031, Dec. 2008
17. Q. Liu and J. Wang, "A one-layer recurrent neural network with a discontinuous activation function for linear programming," Neural Comput., vol. 20, no. 5, pp. 1366-1383, 2008
18. A. L. Yuille and N. M. Grzywacz, "A winner-take-all mechanism based on presynaptic inhibition feedback," Neural Comput., vol. 1, pp. 334-347, Sep. 1989
19. A. Simon and J. Winder, "A model for biological winner-take-all neural competition employing inhibitory modulation of NMDA-mediated excitatory gain," Neurocomputing, vols. 26-27, pp. 587-592, Jun. 1999
20. Y. Fang, M. A. Cohen, and T. G. Kincaid, "Dynamic analysis of a general class of winner-take-all competitive neural networks," IEEE Trans. Neural Netw., vol. 21, no. 5, pp. 771-783, May 2010
21. J. P. F. Sum, C. S. Leung, P. K. S. Tam, G. H. Young, W. K. Kan, and L. W. Chan, "Analysis for a class of winner-take-all model," IEEE Trans. Neural Netw., vol. 10, no. 1, pp. 64-71, Jan. 1999
22. H. Khalil, Nonlinear Systems. Englewood Cliffs, NJ: Prentice-Hall, Jan. 2002
23. A. Isidori, Nonlinear Control Systems, 3rd ed. New York: Springer- Verlag, 1995
24. D. P. Mandic and J. A. Chambers, "A normalised real time recurrent learning algorithm," Signal Process., vol. 80, no. 9, pp. 1909-1916, 2000
25. D. R. Mandic, E. V. Papoulis, and C. G. Boukis, "A normalized mixed-norm adaptive filtering algorithm robust under impulsive noise interference," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 6. Apr. 2003, pp. 333-336
26. Q. Liu and J. Wang, "Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 601-613, Apr. 2011
27. Q. Liu and J.Wang, "A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming," IEEE Trans. Neural Netw., vol. 19, no. 4, pp. 558-570, Apr. 2008 (Pubitemid 351583574)
28. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Stat. Soc., Ser. B, vol. 58, no. 1, pp. 267-288, 1994
29. S. Boyd and L. Vandenberghe, Convex Optimization. New York: Cambridge Univ. Press, 2004
30. T. R. Diethe, "Sparse machine learning methods with applications in multivariate signal processing," Ph.D. dissertation, Dept. Compt. Sci., University College London, London, U.K., 2010
31. J. P. LaSalle and S. Lefschetz, Stability by Liapunov's Direct Method with Applications. New York: Academic, 1973
32. Y. Shen and J.Wang, "Robustness analysis of global exponential stability of recurrent neural networks in the presence of time delays and random disturbances," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 1, pp. 87-96, Jan. 2012
33. Y. Xiao, Y. Liu, C. Leung, J. P. Sum, and K. Ho, "Analysis on the convergence time of dual neural network-based," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 676-682, Apr. 2012
34. B. Ammar, F. Cherif, and A. M. Alimi, "Existence and uniqueness of pseudo almost-periodic solutions of recurrent neural networks with timevarying coefficients and mixed delays," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 1, pp. 109-118, Jan. 2012
35. Z. Wang, H. Zhang, and B. Jiang, "LMI-based approach for global asymptotic stability analysis of recurrent neural networks with various delays and structures," IEEE Trans. Neural Netw., vol. 22, no. 7, pp. 1032-1045, Jul. 2011.