Asynchronous stochastic approximations

SIAM Journal on Control and Optimization

Volumn 36, Issue 3, 1998, Pages 840-851

  Borkar, Vivek S ^a  

^a INDIAN INSTITUTE OF SCIENCE   (India)

Author keywords

Asynchronous algorithms; Communication delays; Distributed algorithms; ODE limit; Stochastic approximation

Indexed keywords

ALGORITHMS; DIFFERENTIAL EQUATIONS; RANDOM PROCESSES;

ASYNCHRONOUS ALGORITHMS; ASYNCHRONOUS STOCHASTIC APPROXIMATION; COMMUNICATION DELAYS; DISTRIBUTED ALGORITHMS;

APPROXIMATION THEORY;

EID: 0032075427     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012995282784     Document Type: Article

References (22)

1. J. ABOUNADI, D. BERTSEKAS, AND V.S. BORKAR, O.D.E. Analysis for Q-Learning Algorithms, preprint, Laboratory for Information and Decision Systems, M.I.T., Cambridge, MA, 1996.
2. A. BENVENISTE, M. METIVIER, AND P. PRIOURET, Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, Berlin, Heidelberg, 1990.
3. D.P. BERTSEKAS AND J.N. TSITSIKLIS, Parallel and Distributed Computation, Prentice-Hall, Englewood Cliffs, NJ, 1989.
4. V.S. BORKAR, Topics in Controlled Markov Chains, Pitman Res. Notes in Math. Ser. 240, Longman Scientific and Technical, Harlow, UK, 1991.
5. V.S. BORKAR, Distributed computation of fixed points of ∞-nonexpansive maps, Proc. Indian Acad. Sci. Math. Sci., 106 (1996), pp. 289-300.
6. V.S. BORKAR AND V. V. PHANSALKAR, Managing interprocessor delays in distributed recursive algorithms, Sādhanā, 19 (1994), pp. 995-1003.
7. V.S. BORKAR AND K. SOUMYANATH, A new analog parallel scheme for fixed point computation, Part I: Theory, IEEE Trans. Circuits Systems I Fund. Theory Appl., 44 (1997), pp. 509-522.
8. H.F. CHEN, Stochastic approximation and its new applications, in Proc. 1994 Hong Kong Internat. Workshop on New Directions in Control and Manufacturing, 1994, pp. 2-12.
9. D. FOSTER AND P. YOUNG, Stochastic evolutionary game dynamics, Theoret. Population Biol., 38 (1990), pp. 229-232.
10. M. FREIDLIN AND A. WENTZELL, Random Perturbations of Dynamical Systems, Springer-Verlag, Berlin, New York, 1984.
11. d, SLAM J. Control Optim., 29 (1992), pp. 999-1018.
12. R. GHARAVI AND V. ANANTHARAM, A Structure Theorem for Partially Asynchronous Relaxations with Random Delays, ERL Memo. No. M92/143, Electronics Research Laboratory, University of California, Berkeley, 1993.
13. S. HAYKIN, Neural Networks: A Comprehensive Foundation, McMillan, New York, 1994.
14. M. HIRSCH, Convergent activation dynamics in continuous time networks, Neural Networks, 2 (1987), pp. 331-349.
15. T. JAAKOLA, M. JORDAN, AND S.P. SINGH, On the convergence of stochastic iterative dynamic programming algorithms, Neural Computation, 6 (1994). pp. 1185-1201.
16. H. KUSHNER AND D. CLARK, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer-Verlag, New York, 1978.
17. H. KUSHNER AND G. YIN, Stochastic approximation algorithms for parallel and distributed processing, Stochastics, 22 (1987), pp. 219-250.
18. H. KUSHNER AND G. YIN, Asymptotic properties of distributed and communicating stochastic approximation algorithms, SIAM J. Control Optim., 25 (1987). pp. 1266-1290.
19. J. NEVEU, Discrete Parameter Martingales, North-Holland, Amsterdam, 1975.
20. T. SARGENT, Bounded Rationality in Macroeconomics, Clarendon Press, Oxford, UK, 1993.
21. J. TSITSIKLIS, Asynchronous stochastic approximation and Q-learning, Machine Learning, 16 (1994), pp. 185-202.
22. J. TSITSIKLIS, D. BERTSEKAS, AND M. ATHANS, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automatic. Control, AC-31 (1986), pp. 803-812.