Sufficient and necessary condition for the convergence of stochastic approximation algorithms

Statistics and Probability Letters

Volumn 76, Issue 2, 2006, Pages 203-210

  Chen, Neiping ^a   Liu, Wenbin ^b   Feng, Jianfeng ^c,d  

^a HUNAN UNIVERSITY OF COMMERCE   (China)

^b UNIVERSITY OF KENT   (United Kingdom)

^c HUNAN NORMAL UNIVERSITY   (China)

^d UNIVERSITY OF WARWICK   (United Kingdom)

Author keywords

Global minima; Local minima; Simulated annealing; Stochastic approximation algorithms

Indexed keywords

EID: 30344481937     PISSN: 01677152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.spl.2005.07.020     Document Type: Article

References (17)

1. S. Albeverio J. Feng M. Qian Role of noises in neural networks Phys. Rev. E 52 1995 6593-6606
2. A. Benveniste M. Métivier P. Priouret Adaptive Algorithms and Stochastic Approximations 1990 Springer Berlin
3. O. Catoni Rough large deviation estimates for simulated annealing: application to exponential schedules Ann. Probab. 20 1992 1109-1146
4. T.S. Chiang Y. Chow On the convergence rate of annealing processes SIAM J. Control Optim. 26 1988 1455-1470
5. J. Feng H.-O. Georgii D. Brown Convergence to global minima for a class of diffusion processes Physica A 276 2000 465-476
6. M.I. Freidlin A.D. Wentzell Random Perturbations of Dynamic System 1984 Springer Berlin
7. G.C. Goodwin K. Sin Adaptive Filtering, Prediction, and Control 1984 Prentice-Hall Englewood Cliffs, NJ
8. B. Hajek Cooling schedules for optimal annealing Math. Oper. Res. 13 1988 311-329
9. C.-R. Hwang S.-J. Sheu Large-time behavior of perturbed diffusion Markov processes with applications to the second eigenvalue problem for Fokker-Planck operators and simulated annealing Acta Appl. Math. 19 1990 253-295
10. H. Kushner D. Clark Stochastic Approximation Methods for Constrained and Unconstrained Systems 1978 Springer New York
11. L. Ljung T. Soderström Theory and Practice of Recursive Identification 1983 MIT Press Cambridge
12. Nevel'son, M.B., Has'minskii, R.Z., 1976. Stochastic Approximation and Recursive Estimation. Translation of Mathematical Monograph, vol. 47, American Mathematical Society, Providence, RI.
13. H. Robbins S. Monro A stochastic approximation method Ann. Math. Statist. 22 1951 400-407
14. Sirlantize, K., Feng, J., Liu, W., 2002. Novel gradient algorithms for stochastic optimization, submitted for publication.
15. V.N. Smelyanskiy M.I. Dykman Optimal control of large fluctuations Phys. Rev. E 55 1997 2516-2521
16. G. Winkler Image Analysis, Random Fields and Dynamic Monte Carlo Methods. An Introduction to Mathematical Aspects 1994 Springer Berlin
17. G. Yin K. Yin Asymptotically optimal rate of convergence of smoothed stochastic recursive algorithms Stochastics Stochastics Rep. 47 1994 21-46