Convergence of a global stochastic optimization algorithm with partial step size restarting

Advances in Applied Probability

Volumn 32, Issue 2, 2000, Pages 480-498

  Yin, G ^a  

^a WAYNE STATE UNIVERSITY   (United States)

Author keywords

Diffusion; Global optimization; Monte Carlo methods; Restarting; Weak convergence

Indexed keywords

EID: 0034207054     PISSN: 00018678     EISSN: None     Source Type: Journal    
DOI: 10.1239/aap/1013540175     Document Type: Article

References (25)

1. CAI, X., KELLY, P. and Gong, W. B. (1995). Digital diffusion networkforimage segmentation. Proc. IEEE Internal. Conf. Image Processing, Vol III, pp. 73-76.
2. Chen, H.-F. (1998). Stochastic approximation with non-additive measurement noise. J. Appl. Prob. 35, 407-417.
3. n. SIAM J. Control Optim. 25, 737-752.
4. Dippon, J. (1998). Globally convergentstochastic optimization with optimal asymptotic distribution. J. Appl. Prob. 35, 395-406.
5. Dippon, J. and Fabian, V. (1994). Stochastic approximation of global minimum points. J. Statist. Plann. Inference 41, 327-347.
6. Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes, Characterization and Convergence. JohnWiley, New York.
7. Eweda, E. and Macchi, O. (1983). Quadratic mean and almost sure convergence of unbounded stochastic approximation algorithm with correlated observations. Ann. Inst. Henri Poincaré 19, 235-255.
8. d. SIAMJ. ControlOptim. 29, 999-1018.
9. Geman, D. and Geman, S. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Machine Intelligence 6, 721-741.
10. Gidas, B. (1985). Nonstationary Markov chains and convergence of the annealing algorithm. J. Statist. Physics 39, 73-131.
11. Hwang, C.-R. (1980). Laplace’s method revised: weak convergence of probability measures. Ann. Prob. 8, 1177-1182.
12. KIRKPATRICK, S., Gelatt, C. D. and Vecchi, M. P. (1983). Optimization by simulated annealing. Science 220, 671-680.
13. KUSHNER, H. J. (1984). Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory. MIT Press, Cambridge.
14. KUSHNER, H. J. (1987). Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via Monte Carlo. SIAM J. Appl. Math. 47, 169-185.
15. KUSHNER, H. J. and Yin, G. (1997). StochasticApproximationAlgorithms andApplications. Springer, New York.
16. L’ECUYER, P. and Yin, G. (1998). Budget-dependent convergence rate of stochastic approximation. SIAMJ. Optim. 8, 217-247.
17. MANJUNATH, B., SIMCHONY, T. and Chellappa, R. (1990). Stochastic and deterministic networks for texture segmentation. IEEE Trans. ASSP. 38, 1039-1049.
18. Metropolis, M., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. and Teller, E. (1953). Equations of state calculations by fast computing machines. J. Chem. Phys. 21,1087-1091.
19. MÜLLER, H.-G. (1989). Adaptive nonparametric peak estimation. Ann. Statist. 17, 1053-1069.
20. Revesz, P. (1977). How to apply the method of stochastic approximation in the non-parametric estimation of regression function. Matem. Operations Stat. Ser. Statist. 8, 119-126.
21. SPALL, J. C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automat. Control AC-37, 331-341.
22. WANG, I. J., Chong, E. K. P. and Kulkarni, S. R. (1996). Equivalent necessary and sufficient conditions on noise sequences for stochastic approximation algorithms. Adv. Appl. Prob. 28, 784-801.
23. WONG, E. (1991). Stochastic neural networks. Algorithmica 6, 466-478.
24. YAKOWITZ, S. (1993). A globally convergent stochastic approximation. SIAM J. Control Optim. 31, 30-40.
25. Yin, G. (1999). Rates of convergence for a class of global stochastic optimization algorithms. SIAM J. Optim. 10, 99-120.