From Zhang neural network to newton iteration for matrix inversion

IEEE Transactions on Circuits and Systems I: Regular Papers

Volumn 56, Issue 7, 2009, Pages 1405-1415

  Zhang, Yunong ^a   Ma, Weimu ^a   Cai, Binghuang ^a  

^a SUN YAT SEN UNIVERSITY   (China)

Author keywords

Activation function; Initial state; Matrix inversion; Newton iteration; Recurrent neural network (RNN); Step size

Indexed keywords

CHEMICAL ACTIVATION; DIFFERENCE EQUATIONS;

ACTIVATION FUNCTIONS; INITIAL STATE; MATRIX INVERSIONS; NEWTON ITERATIONS; RECURRENT NEURAL NETWORK (RNN); STEP SIZE;

RECURRENT NEURAL NETWORKS;

EID: 67651154605     PISSN: 10577122     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSI.2008.2007065     Document Type: Article

References (35)

1. Y. Zhang, "Towards piecewise-linear primal neural networks for optimization and redundant robotics," in Proc. IEEE Int. Conf. Netw. Sens. Control, Ft. Lauderdale, FL, 2006, pp. 374-379.
2. Y. Zhang and J. Wang, "A dual neural network for convex quadratic programming subject to linear equality and inequality constraints," Phys. Lett. A, vol. 298, no. 4, pp. 271-278, Jun. 2002.
3. R. J. Steriti and M. A. Fiddy, "Regularized image reconstruction using SVD and a neural network method for matrix inversion," IEEE Trans. Signal Proces., vol. 41, no. 10, pp. 3074-3077, Oct. 1993.
4. 2 operations and O(N) -level storage," in Proc. 44th IEEE Conf. Decisio Control, Seville, Spain, 2005, pp. 3711-3716.
5. T. Sarkar, K. Siarkiewicz, and R. Stratton, "Survey of numerical methods for solution of large systems of linear equations for electromagnetic field problems," IEEE Trans. Antennas Propag., vol. 29, no. 6, pp. 847-856, Nov. 1981.
6. Y. Zhang, "Revisit the analog computer and gradient-based neural system for matrix inversion," in Proc. IEEE Int. Symp. Intell. Control, Limassol, Cyprus, 2005, pp. 1411-1416.
7. R. H. Sturges, Jr., "Analog matrix inversion (robot kinematics ," IEEE J. Robot. Autom., vol. 4, no. 2, pp. 157-162, Apr. 1988.
8. W. E. Leithead and Y. Zhang, "O(N)-operation approximation of covariance matrix inverse in Gaussian process regression based on quasi-Newton BFGS methods," Commun. Stat. Simul. Comput., vol. 36, no. 2, pp. 367-380, Mar. 2007.
9. S. Van Huffel and J. Vandewalle, "Analysis and properties of the generalized total least squares problem AX ≈ when some or all columns in A are subject to errors," SIAM J. Matrix Anal. Appl., vol. 10, pp. 294-315, Jul. 1989.
10. K. S. Yeung and F. Kumbi, "Symbolic matrix inversion with application to electronic circuits," IEEE Trans. Circuits Syst., vol. 35, no. 2, pp. 235-238, Feb. 1988.
11. A. El-Amawy, "A systolic architecture for fast dense matrix inversion," IEEE Trans. Comput., vol. 38, no. 3, pp. 449-455, Mar. 1989.
12. Y. Q. Wang and H. B. Gooi, "New ordering methods for space matrix inversion via diagonalization," IEEE Trans. Power Syst., vol. 12, no. 3, pp. 1298-1305, Aug. 1997.
13. J. Xu, D. Pi, Y.-Y. Cao, and S. Zhong, "On stability of neural networks by a Lyapunov functional-based approach," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 54, no. 4, pp. 912-924, Apr. 2007.
14. R. K. Manherz, B. W. Jordan, and S. L. Hakimi, "Analog methods for computation of the generalized inverse," IEEE Trans. Autom. Control vol. AC-13, no. 5, pp. 582-585, Oct. 1968.
15. X. Xue and W. Bian, "Subgradient-based neural networks for nonsmooth convex optimization problems," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 55, no. 8, pp. 2378-2391, Sep. 2008.
16. D. P. Mandic and J. A. Chambers, Recurrent Neural Networks for Prediction. Chichester, U.K.: Wiley, 2001.
17. J. C. Principe, N. R. Euliano, and W. C. Lefebvre, Neural and Adaptive Systems: Fundamentals Through Simulations. Chichester, U.K.: Wiley, 2000.
18. S. Haykin, Neural Networks: A Comprehensive Foundation. Chichester, U.K.: Wiley, 1998.
19. J.-S. Wang and Y.-P. Chen, "A fully automated recurrent neural network for unknown dynamic system identification and control," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 53, no. 6, pp. 1363-1372, Jun. 2006.
20. J. J. Rubio and W. Yu, "Stability analysis of nonlinear system identification via delayed neural networks," IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 54, no. 2, pp. 161-165, Feb. 2007.
21. S. J. Yoo, Y. H. Choi, and J. B. Park, "Generalized predictive control based on self-recurrent wavelet neural network for stable path tracking of mobile robots: Adaptive learning rates approach," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 53, no. 6, pp. 1381-1394, Jun. 2006.
22. G. Setti, P. Thiran, and C. Serpico, "An approach to information propagation in 1-D cellular neural networks - Part II: Global propagation," IEEE Trans. Circuits Syst. I, Fundam. Theory and Appl. vol. 45, no. 8, pp. 790-811, Aug. 1998.
23. Y. Zhang, D. Jiang, and J. Wang, "A recurrent neural network for solving sylvester equation with time-varying coefficients," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1053-1063, Sep. 2002.
24. Y. Zhang and S. S. Ge, "A general recurrent neural network model for time-varying matrix inversion," in Proc. 42nd IEEE Conf. Decision Control, HI, Maui, 2003, pp. 6169-6174.
25. Y. Zhang and S. S. Ge, "Design and analysis of a general recurrent neural network model for time-varying matrix inversion," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1477-1490, Nov. 2005.
26. H. Zhang, Z.Wang, and D. Liu, "Global asymptotic stability and robust stability of a class of Cohen-Grossberg neural networks with mixed delays," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 56, no. 3, pp. 616-629, Mar. 2009.
27. C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989.
28. N. C. F. Carneiro and L. P. Caloba, "A new algorithm for analog matrix inversion," in Proc. 38th Midwest Symp. Circuits Syst., Rio de Janeiro, Brazil, 1995, vol. 1, pp. 401-404.
29. J. Jang, S. Lee, and S. Shin, "An optimization network for matrix inversion," in Neural Information Processing and Computing. New York: Amer. Inst. Phys., 1988, pp. 397-401.
30. J. Wang, "A recurrent neural network for real-time matrix inversion," Appl. Math. Comput., vol. 55, no. 1, pp. 89-100, Apr. 1993.
31. "MATLAB R2007a," MathWorks Inc., Natick, MA, 2007.
32. V. Pan and J. Reif, "Fast and efficient parallel solution of dense linear systems," Comput. Math. Appl., vol. 17, no. 11, pp. 1481-1491, 1989.
33. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985.
34. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
35. F. M. Ham and I. Kostanic, Principles of Neurocomputing for Science and Engineering. New York: McGraw-Hill, 2001.