Convergence analysis of batch gradient algorithm for three classes of Sigma-Pi neural networks

Neural Processing Letters

Volumn 26, Issue 3, 2007, Pages 177-189

  Zhang, Chao ^a   Wu, Wei ^a   Xiong, Yan ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Batch gradient algorithm; Convergence; Monotonicity; Sigma Pi Sigma neural networks; Sigma Pi Sigma Pi neural networks; Sigma Sigma Pi neural networks

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; ERROR ANALYSIS; ITERATIVE METHODS; LEARNING SYSTEMS;

BATCH GRADIENT ALGORITHMS; ERROR FUNCTIONS; MONOTONICITY;

FEEDFORWARD NEURAL NETWORKS;

EID: 35548986202     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-007-9050-0     Document Type: Article

References (14)

1. Rumelhart DE and McClelland JL (1986). Parallel distributed processing, explorations in the microstructure of cognition. MIT Press, Cambridge
2. Li JY and Yu YL (1995). The realization of arbitrary Boolean function by two layer higher-order neural network. J South China Univ Techn 23: 111-116
3. Lenze B (2004). Note on interpolation on the hypercube by means of sigma-pi neural networks. Neurocomputing 61: 471-478
4. Bertsekas DP and Tsiklis J (1996). Neuro-dynamic programming. Athena Scientific, Boston, MA
5. Kushner HJ and Yang J (1995). Analysis of adaptive step size SA algorithms for parameter rates. IEEE Transac Automat Control 40: 1403-1410
6. Shao HM, Wu W, Li F and Zheng GF (2004). Convergence of gradient algorithm for feedforward neural network training. Proceed Int Symposium Comput Inform 2: 627-631
7. Wu W, Feng G and Li X (2002). Training multilayer perceptrons via minimization of sum of ridge functions. Adv Computat Math 17: 331-347
8. Liang YC (2002). Successive approximation training algorithm for feedforward neural networks. Neurocomputing 42: 311-322
9. Wu W, Feng G, Li Z and Xu Y (2005). Deterministic convergence of an online gradient method for BP neural networks. IEEE Transac Neural Networks 16: 533-540
10. Durbin R and Rumelhart D (1989). Product units: a computationally powerful and biologically pausible extension to backpropagation networks. Neural Comput 1: 133-142
11. Lenze B (1994). How to make sigma-pi neural networks perform perfectly on regular training sets. Neural Networks 7: 1285-1293
12. Heywood M and Noakes P (1995). A Framework for improved training of sigma-pi networks. IEEE Transac Neural Networks 6: 893-903
13. Yuan Y and Sun W (2001). Optimization theory and methods. Science Press, Beijing
14. Wu W, Shao HM, Qu D (2005) Strong convergence for gradient methods for BP networks training. Proceedings of the International Conference on on Neural Networks and Brains (ICNNB'05), pp 332-334