On the lock-in probability of stochastic approximation

Combinatorics Probability and Computing

Volumn 11, Issue 1, 2002, Pages 11-20

  Borkar, V S ^a  

^a TATA INSTITUTE OF FUNDAMENTAL RESEARCH   (India)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ASYMPTOTIC STABILITY; COMPUTATIONAL COMPLEXITY; CONVERGENCE OF NUMERICAL METHODS; GRADIENT METHODS; INTEGRATION; MATHEMATICAL MODELS; PROBABILITY; PROBLEM SOLVING; RANDOM PROCESSES; RECURSIVE FUNCTIONS; THEOREM PROVING;

LOCK-IN PROBABILITY; MARTINGALE DIFFERENCE SEQUENCE; STOCHASTIC APPROXIMATION; STOCHASTIC GRADIENT METHOD;

APPROXIMATION THEORY;

EID: 0036003892     PISSN: 09635483     EISSN: None     Source Type: Journal    
DOI: 10.1017/S0963548301004953     Document Type: Article

References (27)

1. Arthur, W. B. (1994) Increasing Returns and Path Dependence in the Economy, University of Michigan Press, Ann Arbor, MI.
2. Benaim, M. (1996) A dynamical system approach to stochastic optimization. SIAM J. Control Optim. 34 437-472.
3. Benaim, M. (1999) Dynamics of stochastic approximations. In Le Séminaire de Probabilités (J. Azéma, M. Émery, M. Ledoux and M. Yor, eds), Vol. 1709 of Springer Lecture Notes in Mathematics, pp. 1-68.
4. Benveniste, A., Metivier, M. and Priouret, P. (1990) Adaptive Algorithms and Stochastic Approximation, Springer, Berlin/New York.
5. Borkar, V. S. (1995) Probability Theory: An Advanced Course, Springer, New York.
6. Borkar, V. S. (1996) Stochastic approximation with two time scales. Systems Control Lett. 29 291-294.
7. Borkar, V. S. (1998) Asynchronous stochastic approximation. SIAM J. Control Optim. 36 841-851. (Correction note in ibid. (2000) 38 662-663.)
8. Borkar, V. S. (2000) On the number of samples required for Q-learning. In 38th Annual Allerton Conf. on Communication, Control and Computing (Monticello, Illinois, Oct. 2000).
9. Borkar, V. S. and Meyn, S. P. (2000) The O.D.E. method for convergence of stochastic approximation and reinforcement learning. SIAM J. Control Optim. 38 447-469.
10. Brandiere, O. (1998) Some pathological traps for stochastic approximation. SIAM J. Control Optim. 36 1293-1314.
11. Brandiere, O. and Duflo, M. (1996) Les algorithms stochastiques contournent-ils les pièges? Ann. Inst. Henri Poincaré 32 395-427.
12. Dupuis, P. and Kushner, H. J. (1989) Stochastic approximation and large deviations: Upper bounds and w. p. 1 convergence. SIAM J. Control Optim. 27 1108-1135.
13. Gerencser, L. (1992) Rate of convergence of recursive estimators. SIAM J. Control Optim. 30 1200-1227.
14. Haykin, S. (1991) Adaptive Filter Theory, 2nd edn, Prentice-Hall, Englewood Cliffs, NJ.
15. Haykin, S. (1999) Neural Networks: A Comprehensive Introduction, 2nd edn, Prentice-Hall, Englewood Cliffs, NJ.
16. Kelly, K. (1998) New Rules for the New Economy, Viking, New York.
17. Krasovskii, N. N. (1963) Stability of Motion, Stanford University Press, Stanford, CA.
18. Kushner, H. J. and Yin, G. (1997) Stochastic Approximation Algorithms and Applications, Springer, New York.
19. Ljung, L. (1977) Analysis of recursive stochastic algorithms. IEEE Trans. Automatic Control. 22 551-575.
20. McDiarmid, C. (1998) Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics (M. Habib, C. McDiarmid, J. Ramirez-Alfonsin and B. Reed, eds), Springer, Berlin/Heidelberg, pp. 195-248.
21. Pelletier, M. (1998) On almost sure asymptotic behavior of stochastic algorithms. Stoch. Process. Appl. 78 217-244.
22. Pemantle, R. (1995) Nonconvergence to unstable points in urn models and stochastic approximations. Ann. Prob. 18 698-712.
23. Pflug, G. (1990) Non-asymptotic confidence bounds for stochastic approximation algorithms with constant step size. Monatshefte für Mathematik 110 297-314.
24. Sargent, T. (1993) Bounded Rationality in Macroeconomics, Clarendon Press, Oxford.
25. Shapiro, C. and Varian, H. (1999) Information Rules: A Strategic Guide to the Network Economy, Harvard Business School Press, Boston.
26. Vapnik, V. (1999) Statistical Learning Theory, Wiley, New York.
27. Wilson, F. W. (1969) Smoothing derivatives of functions and applications. Trans. Amer. Math. Soc. 139 413-428.