Stability of stochastic approximation under verifiable conditions

SIAM Journal on Control and Optimization

Volumn 44, Issue 1, 2006, Pages 283-312

  Andrieu, Christophe ^a   Moulines, Éric ^b   Priouret, Pierre ^c  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b TELECOM PARISTECH   (France)

^c SORBONNE UNIVERSITÉ   (France)

Author keywords

Adaptive Markov chain Monte Carlo; Randomly varying truncation; State dependent noise; Stochastic approximation

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; SET THEORY; SYSTEM STABILITY;

ADAPTIVE TRUNCATION SETS; STOCHASTIC APPROXIMATION; SUBSETS;

STOCHASTIC CONTROL SYSTEMS;

EID: 33244461073     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012902417267     Document Type: Article

References (31)

1. J. ABOUNADI, D. P. BERTSEKAS, AND V. BORKAR, Stochastic approximation for nonexpansive maps: Application to Q-learning algorithms, SIAM J. Control Optim., 41 (2002), pp. 1-22.
2. C. ANDRIEU AND E. MOULINES, On the ergodicity properties of some Markov chain Monte Carlo algorithms, Ann. Appl. Probab., to appear.
3. C. ANDRIEU AND C. P. ROBERT, Controlled MCMC for Optimal Sampling, Cahiers du Cérémade 0125, Université de Paris Dauphine, Paris, 2001.
4. J. BARTUSEK, Stochastic Approximation and Optimization of Markov Chains, Ph.D. thesis, The Institute for Systems Research, University of Maryland, College Park, MD, 2000.
5. A. BENVENISTE, M. MÉTIVIER, AND P. PRIOURET, Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, New York, 1990.
6. V. BORKAR, Stability of annealing schemes and related processes, Systems Control Lett., 41 (2000), pp. 325-331.
7. V. S. BORKAR AND S. P. MEYN, The o.d.e. method for convergence of stochastic approximation and reinforcement learning, SIAM J. Control Optim., 38 (2000), pp. 447-469.
8. R. BUCHE AND H. J. KUSHNER, Rate of convergence for constrained stochastic approximation algorithms, SIAM J. Control Optim., 40 (2001), pp. 1011-1041.
9. H. CHEN, Stochastic approximation with state-dependent noise, Sci. China Ser. E, 43 (2000), pp. 531-541.
10. H. CHEN, L. GUO, AND A. GAO, Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds, Stochastic Process. Appl., 27 (1988), pp. 217-231.
11. H. CHEN AND Y.-M. ZHU, Stochastic approximation procedures with randomly varying truncations, Sci. Sinica, 29 (1986), pp. 914-926.
12. H.-F. CHEN, Stochastic Approximation and Its Applications, Nonconvex Optimization and Its Applications 64, Kluwer Academic Publishers, Dordrecht, 2002.
13. B. DELYON, Stochastic Approximation with Decreasing Gain: Convergence and Asymptotic Theory, Tech. report, Université de Rennes, Rennes, France, 2000.
14. B. DELYON, M. LAVIELLE, AND E. MOULINES, Convergence of a stochastic approximation version of the EM algorithm, Ann. Stat., 27 (1999), pp. 94-128.
15. R. DOUC, E. MOULINES, AND J. ROSENTHAL, Quantitative bounds on convergence of time-inhomogeneous Markov chains, Ann. Appl. Probab., 14 (2004), pp. 1643-1665.
16. M. DUFLO, Random Iterative Systems, Appl. Math. 34, Springer-Verlag, Berlin, 1997.
17. L. GERENCSÉR AND S. S. WILSON, Rate of convergence of recursive estimators, SIAM J. Control Optim., 30 (1992), pp. 1200-1227.
18. P. W. GLYNN AND S. P. MEYN, A Liapounov bound for solutions of the Poisson equation, Ann. Probab., 24 (1996), pp. 916-931.
19. H. HAARIO, E. SAKSMAN. AND J. TAMMINEN, An adaptive Metropolis algorithm, Bernoulli, 7 (2001), pp. 223-242.
20. P. HALL AND C. HEYDE, Martingale Limit Theory and Its Application, Academic Press, New York, London, 1980.
21. S. JARNER AND E. HANSEN, Geometric ergodicity of Metropolis algorithms, Stochastic Process. Appl., 85 (2000), pp. 341-361.
22. H. KUSHNER AND D. CLARK, Stochastic Approximation for Constrained and Unconstrained Systems, Springer-Verlag, Berlin, Heidelberg, 1978.
23. H. KUSHNER AND G. YIN, Stochastic Approximation Algorithms and Applications, Appl. Math. 35, Springer-Verlag, New York, 1997.
24. N. METROPOLIS, A. ROSENBLUTH, M. ROSENBLUTH, A. TELLER, AND M. TELLER, Equations of state calculations by fast computing machines, J. Chem. Phys., 21 (1953), pp. 1087-1091.
25. S. MEYN AND R. TWEEDIE, Markov Chains and Stochastic Stability, Communication and Control Engineering Series, Springer-Verlag, London, 1993.
26. E. NUMMELIN, On the Poisson equation in the potential theory of a single kernel, Math. Scand., 68 (1991), pp. 59-82.
27. G. ROBERTS AND R. TWEEDIE, Geometric convergence and central limit theorem for multidimensional Hastings and Metropolis algorithms, Biometrika, 83 (1996), pp. 95-110.
28. G. ROBERTS AND R. TWEEDIE, Bounds on regeneration times and convergence rates for Markov chains, Stochastic Process. Appl., 80 (1999), pp. 211-229.
29. J. ROSENTHAL, Minorization conditions and convergence rates for Markov chain Monte Carlo, J. Amer. Statis. Assoc., 90 (1995), pp. 558-566.
30. V. TADIC, Stochastic gradient with random truncations, European J. Oper. Res., 101 (1997), pp. 261-284.
31. V. TADIC, Stochastic approximations with random truncations, state dependent noise and discontinuous dynamics, Stochastics Stochastics Rep., 64 (1998), pp. 283-326.