Strong convergence of gradient methods for BP networks training

Proceedings of 2005 International Conference on Neural Networks and Brain Proceedings, ICNNB'05

Volumn 1, Issue , 2005, Pages 332-334

  Wu, Wei ^a,b   Shao, Hongmei ^a   Qu, Di ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^b DALIAN MARITIME UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; FEEDFORWARD NEURAL NETWORKS; OPTIMIZATION; PROBLEM SOLVING;

GRADIENT PENALTY ALGORITHMS; NETWORKS TRAINING; OPTIMIZATION PROBLEMS;

GRADIENT METHODS;

EID: 33847148535     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. D.E. Rumelhart et al., Parall Distributed Processing-Explorations in the Microstructure of Cognition, MIT press, 1986, pp.318-362.
2. M.T. Hagan et al., Neural Network Design, Publishing House of Machinery Industry, second edition, 2003.
3. G. Hinton, "Connectionist learning procedures," Artificial Intelligence, 1989, pp. 185-234.
4. R. Reed, "Pruning algoithms-a survey," IEEE Transactions on Neural Networks, vol.8, pp. 185-204, 1997.
5. L. Armijo, "Mininization of function having Lipschitz continious first partial derivaties," Pacific J. Math, vol.16, pp.1-3, 1966.
6. A.A. Goldstein, "Cauchy's method of minimization," Numer. Math. vol.4, pp. 146-150, 1962.
7. J.C. Allwright, "Global Convergence Rates for Descent Algorithms: Improved Results for Strictly Convex Problems," Journal of Optimization Theory and Applications, vol.26, pp.367-372, 1978.
8. E. Polar, Computational Methods in Optimization: A Unified Approach, New York: Academic Press, 1971.
9. M. Gori and M. Maggini, "Optimal convergence of online backprop-agation," IEEE Trans. Neural Networks, vol.7, pp.251-254, 1996.
10. H.M. Shao, W. Wu and F. Li, "Convergence of online gradient method with a penalty term for feedforward neural networks with stochastic inputs," Numerical Mathematics (English series), Vol.14, No.1, pp.87-96, 2005.
11. H.M. Shao, W. Wu and Z.X. Li, "Convergence of batch gradient algorithm with penalty for freedforward neural network training," submitted.
12. H.M. Shao, W. Wu, F. Li and G.F. Zheng, "Convergence of gradient algorithm for freedforward neural network Training," the Proceedings of International Symposium on Computing and Information, Vol.2, pp.627-631, 2004.
13. W. Wu and Y.S. Xu, "Deterministic convergence of an online gradient method for neural networks," Journal of Computational and Applied Mathematics, vol.144, pp.335-347, 2002.
14. W. Wu, G.R. Feng and X. Li, "Training multilayer perceptrons via minimization of sum of ridge functions," Advances in Computational Mathematics, vol.17, pp.331-347, 2002.
15. Z.X. Li, W. Wu and Y.L. Tian, "Convergence of an online gradient method for feedforword neural networks with stochastic inputs," Journal of Computational and Applied Mathematics, vol.163, pp. 165-176, 2004.
16. W. Wu, G.R. Feng and Z.X. Li, "Convergence of an online gradient method for BP neural networks," IEEE Tran. Neural Networks, vol.16, pp.533-540, 2005.
17. Y.X. Yuan and W.Y. Sun, Optimization Theory and Methods, Science Press, Beijing, 2001.
18. R. Ortega, Iterative Solution of Nonlinear Equations in Several Variables (cf. Th.14.1.4 and Th.14.1.5), Academic Press, 1970.