Asymptotic Properties of Two Time-Scale Stochastic Approximation Algorithms with Constant Step Sizes

Proceedings of the American Control Conference

Volumn 5, Issue , 2003, Pages 4426-4431

  Tadić, Vladislav B ^a   Meyn, Sean P ^b  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b University of Illinois at Urbana Champaign   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ASYMPTOTIC STABILITY; MARKOV PROCESSES; PROBLEM SOLVING; SPURIOUS SIGNAL NOISE;

MARTINGALE DIFFERENCE SEQUENCE;

STOCHASTIC CONTROL SYSTEMS;

EID: 0142231039     PISSN: 07431619     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (15)

1. J. S. Baras and V. S. Borkar, "A learning algorithm for Markov decision processes with adaptive state aggregation," 39th IEEE Conference on Decision and Control, 2000.
2. A. Benveniste, M. Metivier, and P. Priouret, Adaptive Algorithms and Stochastic Approximation, Springer Verlag, 1990.
3. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996.
4. V. S. Borkar, "Stochastic approximation with two time scales," Systems and Control Letters, vol. 29, pp. 291-294, 1997.
5. S. Bhatnagar, M. C. Fu, and S. I. Marcus, "Optimal multilevel feedback policies for ABR flow control using two timescale SPSA," Technical Report TR 99-18, Institute of systems Research, University of Maryland, 1999.
6. S. Bhatnagar, M. C. Fu, S. I. Marcus, and S. Bhatnagar, "Randomized difference two-timescale simultaneous perturbation stochastic approximation algorithms for simulation optimization of hidden Markov models," Technical Report TR 2000-13, Institute of systems Research, University of Maryland, 2000.
7. P. Doukhan, Mixing: Properties and Examples, Springer Verlag, 1994.
8. V. R. Konda and V. S. Borkar, "Actor-critic like learning algorithms for Markov decision processes," SIAM Journal on control and Optimization, vol. 38, pp. 94-123, 1999.
9. V. R. Konda, Actor-Critic Algorithms, PhD Thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, 2002.
10. V. R. Konda and J. N. Tsitsiklis, "On actor-critic algorithms, " SIAM Journal on control and Optimization, to appear.
11. H. J. Kushner and G. G. Yin, Stochastic Approximation Algorithms and Applications, Springer Verlag, 1997.
12. R. E. O'Malley Jr, Singular Perturbations Methods for Ordinary Differential Equations, Springer Verlag, 1991.
13. A. N. Shiryayev, Probability, Springer Verlag, 1984.
14. V. B. Tadić, "Stochastic Approximation with Random Truncations, State Dependent Noise and Discontinuous Dynamics," Stochastics and Stochastics Reports, vol. 64, 283-325, 1998.
15. V. B. Tadić and S. P. Meyn, "Asymptotic Properties of Two Time-Scale Stochastic Approximation Algorithms with Constant Step Sizes," to be submitted.