Boundedness and convergence of online gradient method with penalty for linear output feedforward neural networks

Neural Processing Letters

Volumn 29, Issue 3, 2009, Pages 205-212

  Zhang, Huisheng ^a,b   Wu, Wei ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^b DALIAN MARITIME UNIVERSITY   (China)

Author keywords

Boundedness; Convergence; Feedforward neural networks; Linear output; Online gradient method; Penalty

Indexed keywords

BOUNDEDNESS; CONVERGENCE; LINEAR OUTPUT; ONLINE GRADIENT METHOD; PENALTY;

CONVERGENCE OF NUMERICAL METHODS; GRADIENT METHODS;

FEEDFORWARD NEURAL NETWORKS;

EID: 67649085858     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-009-9104-6     Document Type: Article

References (12)

1. T Chen 1995 Universal approximation to nonlinear operations by Neural Networks with arbitrary activation functions and its application to dynamical system IEEE Trans Neural Netw 6 4 911 917
2. TL Fine S Mukherjee 1999 Parameter convergence and learning curves for neural networks Neural Comput 11 747 769
3. AA Gaivoronski 1994 Convergence properties of backpropagation for neural nets via theory of stochastic gradient methods (Part I) Optim Methods Softw 4 117 134
4. Hagan MT, Demuth HB, Beale M (2003) Neural network design. China Machine Press, Beijing
5. SJ Hanson LY Pratt 1989 Comparing biases for minimal network construction with back-propagation Neural Inf Process 1 177 185
6. Hu ST (1965) Elements of general topology. Holden-Day, San Francisco
7. R Reed 1993 Pruning algorithms: a survey IEEE Trans Neural Netw 4 5 740 747
8. K Saito R Nakano 2000 Second-order learning algorithm with squared penalty term Neural Comput 12 709 729
9. H Shao W Wu L Liu 2007 Convergence and monotonicity of an online gradient method with penalty for neural networks WSEAS Trans Math 6 3 469 476 (Pubitemid 46358891)
10. Tadic V, Stankovic S (2000) Learning in neural networks by normalized stochastic gradient algorithm: local convergence. Proceedings of the 5th Seminar on Neural Network Applications in Electrical Engineering, Yugoslavia, pp 11-17
11. H White 1989 Some asymptotic results for learning in single hidden-layer feedforward network models J Am Stat Ass 84 1003 1013
12. W Wu G Feng Z Li 2005 Convergence of an online gradient method for BP neural networks IEEE Trans Neural Netw 16 3 533 540