Deterministic convergence of an online gradient method for neural networks

Journal of Computational and Applied Mathematics

Volumn 144, Issue 1-2, 2002, Pages 335-347

  Wu, Wei ^a   Xu, Yuesheng ^b,c  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

^b NORTH DAKOTA STATE UNIVERSITY   (United States)

^c INSTITUTE OF APPLIED MATHEMATICS   (China)

Author keywords

Constant learning rate; Deterministic convergence; Monotonicity; Nonlinear feedforward neural networks; Online stochastic gradient method

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; ERRORS; GRADIENT METHODS;

MONOTONICITY;

NEURAL NETWORKS;

MATHEMATICAL METHOD; NEURAL NETWORK;

EID: 0036644480     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0377-0427(01)00571-4     Document Type: Article

References (18)

1. First- and second-order methods for learning: Between steepest sescent and Newton's method
2. The numerical approach of neural networks
3. Parameter convergence and learning curves for neural networks
4. Diffusion approximations for the constant learning rate backpropagation algorithm and resistance to locol minima
5. Convergence properties of backpropagation for neural nets via theory of stochastic gradient methods
6. Convergent on-line algorithms for supervised learning in neural networks
7. Convergence of learning algorithms with constant learning rates
8. On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks
9. Analysis of an approximate gradient projection method with applications to the backpropagation algorithm
10. Serial and parallel backpropagation convergence via nonmonotone perturbed minimization
11. Online steepest descent yields weights with nonnormal limiting distribution
12. Improving the error backpropagation algorithm with a modified error function
13. Online learning from finite training sets and robustness to input bias