Common nature of learning between back-propagation and hopfield-type neural networks for generalized matrix inversion with simplified models

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 4, 2013, Pages 579-592

  Zhang, Yunong ^a   Guo, Dongsheng ^a   Li, Zhan ^b,c  

^a SUN YAT SEN UNIVERSITY   (China)

^b LIRMM   (France)

^c INRIA   (France)

Author keywords

Back propagation type neural networks (BPNN); common nature of learning; discrete time; generalized matrix inversion; hopfield type neural networks

Indexed keywords

BACK-PROPAGATION-TYPE NEURAL NETWORKS (BPNN); COMMON NATURE OF LEARNING; DISCRETE-TIME; GENERALIZED MATRIX INVERSIONS; HOPFIELD TYPE NEURAL NETWORKS;

EQUATIONS OF STATE; NETWORK ARCHITECTURE; NEURAL NETWORKS;

BACKPROPAGATION;

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; TRENDS;

ARTIFICIAL INTELLIGENCE; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 84875885639     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2238555     Document Type: Article

References (45)

1. P. Dean and J. Porrill, "Pseudo-inverse control in biological systems: A learning mechanism for fixation stability," Neural Netw., vol. 11, nos. 7-8, pp. 1205-1218, 1998. (Pubitemid 28527120)
2. F.-W. Sun, Y. Jiang, and J. S. Baras, "On the convergence of the inverses of Toeplitz matrices and its applications," IEEE Trans. Inform. Theory, vol. 49, no. 1, pp. 180-190, Jan. 2003.
3. G. A. González, "On-line parameter estimator derived from modified matrix pseudoinverse," Appl. Math. Comput., vol. 189, no. 2, pp. 1139-1145, 2007. (Pubitemid 46802731)
4. A. Wiesel, Y. C. Eldar, and S. Shamai, "Zero-forcing precoding and generalized inverses," IEEE Trans. Signal Process., vol. 56, no. 9, pp. 4409-4418, Sep. 2008.
5. A. Ben-Israel and T. N. E. Greville, Generalized Inverses: Theory and Applications, 2nd ed. New York: Springer-Verlag, 2003.
6. G. Wu, J. Wang, and J. Hootman, "A recurrent neural network for computing pseudoinverse matrices," Math, Comput. Model., vol. 20, no. 1, pp. 13-21, 1994.
7. J. Wang, "Recurrent neural networks for computing pseudoinverses of rank-deficient matrices," SIAM J. Sci. Comput., vol. 18, no. 5, pp. 1479-1493, 1997. (Pubitemid 127642163)
8. M. A. Rakha, "On the Moore-Penrose generalized inverse matrix," Appl. Math. Comput., vol. 158, no. 1, pp. 185-200, 2004.
9. Y. Zhang, N. Tan, B. Cai, and Z. Chen, "MATLAB simulink modeling of Zhang neural network solving for time-varying pseudoinverse in comparison with gradient neural network," in Proc. 2nd Int. Symp. Intell. Inf. Tech. Appl., Dec. 2008, pp. 39-43.
10. Y. Zhang, Y. Yang, N. Tan, and B. Cai "Zhang neural network solving for time-varying full-rank matrix Moore-Penrose inverse," Computing, vol. 92, no. 2, pp. 97-121, 2011.
11. Y. Wei, J. Cai, and M. K. Ng, "Computing Moore-Penrose inverses of Toeplitz matrices by Newton's iteration," Math. Comput. Model., vol. 40, nos. 1-2, pp. 181-191, 2004. (Pubitemid 39588034)
12. L. Chen, E. V. Krishnamurthy, and I. Macleod, "Generalised matrix inversion and rank computation by successive matrix powering," Parallel Comput., vol. 20, no. 3, pp. 297-311, 1994.
13. J. Zhou, Y. M. Zhu, X. R. Li, and Z. S. You, "Variants of the Greville formula with applications to exact recursive least squares," SIAM J. Matrix Anal. Appl., vol. 24, no. 1, pp. 150-164, 2002. (Pubitemid 36260160)
14. X.-B. Gao, L.-Z. Liao, and L. Qi, "A novel neural network for variational inequalities with linear and nonlinear constraints," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1305-1317, Nov. 2005. (Pubitemid 41709632)
15. S. Razavi and B. A. Tolson, "A new formulation for feedforward neural networks," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1588-1598, Nov. 2011.
16. L. Xiao and Y. Zhang, "Zhang neural network versus gradient neural network for solving time-varying linear inequalities," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1676-1684, Oct. 2011.
17. Q. Liu and J. Wang, "Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 601-613, Apr. 2011.
18. Q. F. Wang, "Theoretical and computational issues of optimal control for distributed Hopfield neural network equations with diffusion term," SIAM J. Sci. Comput., vol. 29, no. 2, pp. 890-911, 2007.
19. Y. Zhang, Z. Li, K. Chen, and B. Cai, "Common nature of learning exemplified by BP and Hopfield neural networks for solving online a system of linear equations," in Proc. IEEE Int. Conf. Netw., Sens. Control, Apr. 2008, pp. 832-836.
20. Y. Zhang, Y. Shi, B. Cai, Z. Li, C. Yi, and J. Mai, "Common nature of learning between BP and Hopfield-type neural networks for convex quadratic minimization with simplified network models," in Proc. IEEE Int. Conf. Mech. Autom., Aug. 2009, pp. 2500-2505.
21. R. Zhang, Z.-B. Xu, G.-B. Huang, and D. Wang, "Global convergence of online BP training with dynamic learning rate," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 2, pp. 330-341, Feb. 2012.
22. D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing. Cambridge, MA: MIT Press, 1986.
23. W. Wu, G. Feng, Z. Li, and Y. Xu, "Deterministic convergence of an online gradient method for BP neural networks," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 533-540, May 2005. (Pubitemid 40718001)
24. Z. Man, H. R. Wu, S. Liu, and X. Yu, "A new adaptive backpropagation algorithm based on lyapunov stability theory for neural networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1580-1591, Nov. 2006. (Pubitemid 44824269)
25. A. Khashman, "A modified backpropagation learning algorithm with added emotional coefficients," IEEE Trans. Neural Netw., vol. 19, no. 11, pp. 1896-1909, Nov. 2008.
26. R. Ilin, R. Kozma, and P. J. Werbos, "Beyond feedforward models trained by backpropagation: A practical training tool for a more efficient universal approximator," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 929-937, Jun. 2008.
27. J. J. Rubio, P. Angelov, and J. Pacheco, "Uniformly stable backpropagation algorithm to train a feedforward neural network," IEEE Trans. Neural Netw., vol. 22, no. 3, pp. 356-366, Mar. 2011.
28. G. A. F. Seber, A Matrix Handbook for Statisticians. New York: Wiley, 2007.
29. X.-B. Liang, "Equivalence between local exponential stability of the unique equilibrium point and global stability for Hopfield-type neural networks with two neurons," IEEE Trans. Neural Netw., vol. 11, no. 5, pp. 1194-1196, Sep. 2000.
30. J. Peng, Z.-B. Xu, H. Qiao, and B. Zhang, "A critical analysis on global convergence of Hopfield-type neural networks," IEEE Trans. Circuits Syst. I, Regular Papers, vol. 52, no. 4, pp. 804-814, Apr. 2005. (Pubitemid 40608442)
31. G. Pajares, "A Hopfield neural network for image change detection," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1250-1264, Sep. 2006. (Pubitemid 46445182)
32. S. Mou, H. Gao, J. Lam, and W. Qiang, "A new criterion of delaydependent asymptotic stability for Hopfield neural networks with time delay," IEEE Trans. Neural Netw., vol. 19, no. 3, pp. 532-535, Mar. 2008. (Pubitemid 351411577)
33. Y. Zhang, Y. Shi, K. Chen, and C. Wang, "Global exponential convergence and stability of gradient-based neural network for online matrix inversion," Appl. Math. Comput., vol. 215, no. 3, pp. 1301-1306, Oct. 2009.
34. Y. Zhang, K. Chen, and H. Tan, "Performance analysis of gradient neural network exploited for online time-varying matrix inversion," IEEE Trans. Autom. Control, vol. 54, no. 8, pp. 1940-1945, Aug. 2009.
35. J. Zhang, W. Tang, and P. Zheng, "Estimating the ultimate bound and positively invariant set for a class of Hopfield networks," IEEE Trans. Neural Netw., vol. 22, no. 11, pp. 1735-1743, Nov. 2011.
36. A. Balavoine, J. Romberg, and C. J. Rozell, "Convergence and rate analysis of neural networks for sparse approximation," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 9, pp. 1377-1389, Sep. 2012.
37. C. Mead, Analog VLSI and Neural Systems, Reading, MA: Addison-Wesley, 1989.
38. J. H. Mathews and K. D. Fink, Numerical Methods Using MATLAB, 4th ed. Englewood Cliffs, NJ: Prentice-Hall, 2004.
39. S. K. Mitra, Digital Signal Processing: A Computrer-Based Approach, 3rd ed. New York: McGraw-Hill, 2006.
40. N. Kasabov, "Evolving fuzzy neural networks for supervised/ unsupervised online knowledge-based learning," IEEE Trans. Syst. Man, Cybern. B, vol. 31, no. 6, pp. 902-918, Dec. 2001. (Pubitemid 34035554)
41. J. J. Rubio, "SOFMLS: Online self-organizing fuzzy modified least squares network," IEEE Trans. Fuzzy Syst., vol. 17, no. 6, pp. 1296-1309, Dec. 2009.
42. J. J. Rubio, D. M. Vázquez, and J. Pacheco, "Backpropagation to train an evolving radial basis function neural network," Evolv. Syst., vol. 1, no. 3, pp. 173-180, 2010.
43. P. Angelov, "Fuzzily connected multimodel systems evolving autonomously from data streams," IEEE Trans. Syst. Man, Cybern. B, vol. 41, no. 4, pp. 898-910, Aug. 2011.
44. A. Lemos, W. Caminhas, and F. Gomide, "Fuzzy evolving linear regression trees," Evolv. Syst., vol. 2, no. 1, pp. 1-14, 2011.
45. A. Lemos, W. Caminhas, and F. Gomide, "Multivariable Gaussian evolving fuzzy modeling system," IEEE Trans. Fuzzy Syst., vol. 19, no. 1, pp. 91-104, Feb. 2011.