Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural networks

Neural Computation

Volumn 22, Issue 10, 2010, Pages 2655-2677

  Xu, Dongpo ^a   Zhang, Huisheng ^b   Liu, Lijun ^c  

^a HARBIN ENGINEERING UNIVERSITY   (China)

^b DALIAN MARITIME UNIVERSITY   (China)

^c DALIAN NATIONALITIES UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; LETTER; NONLINEAR SYSTEM; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 78149331700     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00021     Document Type: Letter

References (27)

1. Amin, Md. F., & Murase, K. (2009). Single-layered complex-valued neural network for real-valued classification problems. Neurocomputing, 72(4-6), 945-955.
2. Coelho, P. H. G. (2001). A complex EKF-RTRL neural network. In Proc. Int. Joint Conf. Neural Networks (pp. 120-125). San Francisco: Morgan Kaufmann.
3. Goh, S. L., & Mandic, D. P. (2004). A complex-valued RTRL algorithm for recurrent neural networks. Neural Computation, 16(12), 2699-2713.
4. Goh, S. L., & Mandic, D. P. (2007). Stochastic gradient-adaptive complex-valued nonlinear neural adaptive filters with a gradient-adaptive step size. IEEE Trans. Neural Networks, 18(5), 1511-1516.
5. Gori, M., & Maggini, M. (1996) Optimal convergence of on-line backpropagation. IEEE Trans. Neural Networks, 17, (1), 251-254.
6. Hanna, A. I., &Mandic, D. P. (2003). A fully adaptive normalized nonlinear gradient descent algorithm for complex-valued nonlinear adaptive filters. IEEE Trans. Signal Processing, 51(10), 2540-2549.
7. Hirose, A. (2006). Complex-valued neural networks. Berlin: Springer-Verlag.
8. Kechriotis, G., & Manolakos, E. S. (1994). Training fully recurrent neural networks with complex weights. IEEE Trans. Circuits and Systems-II: Analog and Digital Signal Processing, 41(3), 235-238.
9. Leung, H., & Haykin, S. (1991). The complex backpropagation algorithm. IEEE Trans. Signal Processing, 39(9), 2101-2104.
10. Mandic, D. P. (2004). A generalized normalized gradient descent algorithm. IEEE Signal Processing Letters, 11(2), 115-118.
11. Mandic, D. P., & Chambers, J. A. (2001). Recurrent neural networks for prediction: Learning algorithms, architectures and stability. NewYork:Wiley.
12. Mandic, D. P., & Goh, S. L. (2009). Complex valued nonlinear adaptive filters: Noncircularity, widely linear and neural models. NewYork:Wiley.
13. Mandic, D. P., Hanna, A. I., & Razaz, M. (2001). A normalized gradient descent algorithm for nonlinear adaptive filters using a gradient adaptive step size. IEEE Signal Processing Letters, 8(11), 295-297.
14. Nitta, T. (2009). Complex-valued neural networks: Utilizing high-dimensional parameters. Hershey, PA: Information Science Reference.
15. Widrow, B., McCool, J., & Ball, M. (1975). The complex LMS algorithm. Proc. IEEE, 63, 712-720.
16. Williams, R. J., & Zipser, D. A. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2), 270-280.
17. Wu, W., Feng, G. R., Li, Z. X., & Xu, Y. S. (2005). Deterministic convergence of an online gradient method for BP neural networks. IEEE Trans. Neural Networks, 16(3), 533-540.
18. Wu, W., Shao, H. M., & Qu, D. (2005). Strong convergence for gradient methods for BP networks training. Proc. Intl. Conf. Neural Networks and Brains (pp. 332-334). San Francisco: Morgan Kaufmann.
19. Wu, W., Xu, D. P., & Li, Z. X. (2008). Convergence of gradient method for Elman networks. Appl. Math. Mech., 29(9), 1231-1238.
20. Wu, W., Zhang, N. M., Li, Z. X., Li, L., & Liu, Y. (2008). Convergence of gradient method with momentum for back-propagation neural networks. J. Comput.Math., 26(4), 613-623.
21. Xiong, Y., Wu, W., Kang, X., & Zhang, C. (2007). Training pi-sigma network by online gradient algorithm with penalty for small weight update. Neural Computation, 19(12), 3356-3368.
22. Xu, D. P., Li, Z. X., & Wu, W. (2010). Convergence of gradient method for a fully recurrent neural network. Soft Computing, 14(3), 245-250.
23. Yang, S. S., Siu, S., & Ho, C. L. (2008). Analysis of the initial values in split-complex backpropagation algorithm. IEEE Trans. Neural Networks, 19(9), 1564-1573.
24. Zhang, C., Wu, W., Chen, X. H., & Xiong, Y. (2008). Convergence of BP algorithm for product unit neural networks with exponential weights. Neurocomputing, 72(1-3), 513-520.
25. Zhang, H. S., Wu, W., Liu, F., & Yao, M. C. (2009). Boundedness and convergence of online gradient method with penalty for feedforward neural networks. IEEE Trans. Neural Networks, 20(6), 1050-1054.
26. Zhang, H. S., Xu, D. P., &Wang, Z. P. (2010). Convergence of an online split-complex gradient algorithm for complex-valued neural networks. Discrete Dyn. Nat. Soc., DOI: 10.1155/2010/829692.
27. Zhang, H. S., Zhang, C., & Wu, W. (2009). Convergence of batch split-complex backpropagation algorithm for complex-valued neural networks. Discrete Dyn. Nat. Soc., DOI: 10.1155/2009/329173.