Budget-dependent convergence rate of stochastic approximation

SIAM Journal on Optimization

Volumn 8, Issue 1, 1998, Pages 217-247

  L'Ecuyer, Pierre ^a   Yin, George ^b  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

^b WAYNE STATE UNIVERSITY   (United States)

Author keywords

Discrete event systems; Gradient estimate; Limit theorems; Rate of convergence; Stochastic approximation; Stochastic optimization

Indexed keywords

EID: 0032326765     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/S1052623495270723     Document Type: Article

References (47)

1. S. ANDRADÓTTIR, A stochastic approximation algorithm with varying bounds, Oper. Res., 43 (1995), pp. 1037-1048.
2. F. AZADIVAR AND J. TALAVAGE. Optimization of stochastic simulation models, Math. and Comput. in Simulation, XXII (1980), pp. 231-241.
3. J. A. BATHER, Stochastic approximation: A generalization of the Robbins-Monro procedure, in Proc. 4th Prague Symposium Asymptotic Statist., P. Mandl and M. Hušková. eds., 1989, pp. 13-27.
4. H. F. CHEN AND Y. M. ZHU, Stochastic approximation procedure with randomly varying truncations, Scientic Sinica, 29 (1986), pp. 914-926.
5. K. L. CHUNG, On a stochastic approximation method, Annals of Mathematical Statistics, 25 (1954), pp. 463-483.
6. S. N. ETHIER AND T. G. KURTZ, Markov Processes: Characterization and Convergence, John Wiley, New York, 1986.
7. V. FABIAN, On asymptotic normality in stochastic approximation, Annals of Mathematical Statistics, 39 (1968), pp. 1327-1332.
8. B. L. FOX AND P. W. GLYNN, Simulating discounted costs, Management Science, 35 (1989), pp. 1297-1315.
9. P. GLASSERMAN, Gradient Estimation Via Perturbation Analysis, Kluwer Academic Publishers, Norwell, MA, 1991.
10. P. GLASSERMAN AND D. D. YAO, Some guidelines and guarantees for common random numbers, Management Science, 38 (1992), pp. 884-908.
11. P. W. GLYNN, A GSMP formalism for discrete event systems, Proc. of the IEEE, 77 (1989), pp. 14-23.
12. P. W. GLYNN, Likelihood ratio gradient estimation for stochastic systems, Communications of the ACM, 33 (1990), pp. 75-84.
13. P. W. GLYNN AND P. HEIDELBERGER, Bias properties of budget constrained simulations, Oper. Res., 38 (1990), pp. 801-814.
14. P. W. GLYNN AND P. L'ECUYER, On the existence and estimation of performance measure derivatives for stochastic recursions, in Proc. of the 11th Internat. Conf. on Anal. and Optim. of Systems, Lecture Notes in Control and Inform. Sci., 199, Springer-Verlag, New York, 1994, pp. 429-435.
15. P. W. GLYNN, P. L'ECUYER, AND M. ADÈS, Gradient estimation for ratios, in Proc. of the 1991 Winter Simulation Conference, IEEE Press, Piscataway, NJ, December 1991, pp. 986-993.
16. P. W. GLYNN AND W. WHITT, The asymptotic efficiency of simulation estimators, Oper. Res., 40 (1992), pp. 505-520.
17. A. HAURIE, P. L'ECUYER, AND CH. VAN DELFT, Monte-Carlo optimization of parametrized policies in a class of piecewise deterministic control systems arising in manufacturing flow control, Discrete Event Dynamic Systems: Theory and Appl., 4 (1994), pp. 87-111.
18. J. KOMLÓS AND P. RÉVÉZ, A modification of the Robbins-Monro process, Studia Sci. Math. Hungar., 8 (1973), pp. 329-340.
19. H. J. KUSHNER, Approximation and Weak Convergence Methods for Random Processes, MIT Press, Cambridge, MA, 1984.
20. H. J. KUSHNER AND D. S. CLARK, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Appl. Math. Sci., 26, Springer-Verlag, Berlin, New York, 1978.
21. H. J. KUSHNER AND H. HUANG, Rates of convergence for stochastic approximation type algorithms, SIAM J. Control Optim., 17 (1979), pp. 607-617.
22. H. J. KUSHNER AND F. J. VÁZQUEZ-ABAD, Stochastic approximation methods for systems over an infinite horizon, SIAM J. Control Optim., 34 (1996), pp. 712-756.
23. H. J. KUSHNER AND J. YANG, Stochastic approximation with averaging of the iterates: Optimal asymptotic rate of convergence for general processes, SIAM J. Control Optim., 31 (1993), pp. 1045-1062.
24. H. J. KUSHNER AND G. YIN, Stochastic Approximation Algorithms and Applications, Springer-Verlag, Berlin, New York, 1997.
25. P. L'ECUYER, An overview of derivative estimation, in Proc. of the 1991 Winter Simulation Conference, IEEE Press, Piscataway, NJ, December 1991, pp. 207-217.
26. P. L'ECUYER, Convergence rates for steady-state derivative estimators, Ann. Oper. Res., 39 (1992), pp. 121-136.
27. P. L'ECUYER AND G. PERRON, On the convergence rates of IPA and FDC derivative estimators. Oper. Res., 42 (1994), pp. 643-656.
28. P. L'ECUYER, N. GIROUX, AND P. W. GLYNN, Stochastic optimization by simulation: Numerical experiments with the M/M/1 queue in steady-state, Management Science, 40 (1994), pp. 1245-1261.
29. P. L'ECUYER AND P. W. GLYNN, Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue in steady-state, Management Science, 40 (1994), pp. 1562-1578.
30. P. L'ECUYER AND P. W. GLYNN, Uniform convergence of truncated-horizon estimators for parametrized regenerative systems, manuscript, 1996.
31. S. P. MEYN AND R. L. TWEEDIE, Markov Chains and Stochastic Stability, Springer-Verlag, Berlin, New York, 1993.
32. M. B. NEVEL'SON AND R. Z. KHASMINSKII, Stochastic Approximation and Recursive Estimation, Transl. Math. Monogr., 47. American Mathematical Society, Providence, RI, 1976.
33. B. T. POLYAK, New method of stochastic approximation type, Avtomat. i Telemekh., 7 (1990), pp. 937-946. Translation in Automat. Remote Control, 51 (1991), pp. 937-946.
34. B. T. POLYAK, New method of stochastic approximation type, Avtomat. i Telemekh., 7 (1990), pp. 937-946. Translation in Automat. Remote Control, 51 (1991), pp. 937-946.
35. B. T. POLYAK AND A. B. JUDITSKY, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30 (1992), pp. 838-855.
36. R. Y. RUBINSTEIN AND A. SHAPIRO, Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method, John Wiley, New York, 1993.
37. R. RUPPERT, Almost-sure approximation to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise, Annals Probab., 10 (1982), pp. 178-187.
38. R. RUPPERT, Handbook in sequential analysis, in Stochastic Approximation, B. K. Ghosh and P. K. Sen. eds., Marcel Dekker, New York, 1991, pp. 503-529.
39. R. SCHWABE, Stability results for smoothed stochastic approximation procedures, Tech. rep. Preprint Nr. A-92-14, Fachbereich Mathematik, Freie Universitat Berlin, Berlin, Germany, 1992.
40. K. SIGMAN AND R. W. WOLFF, A review of regenerative processes, SIAM Rev., 35 (1993), pp. 269-288.
41. M. T. WASAN, Stochastic Approximation, Cambridge University Press, London, 1969.
42. G. YIN, A stopping rule for the Robbins-Monro method, J. Optim. Theory Appl., 67 (1990), pp. 151-173.
43. G. YIN, On extensions of Polyak's averaging approach to stochastic approximation, Stochastics, 36 (1992), pp. 245-264.
44. G. YIN, H. YAN, AND X. LOU, On a class of stochastic optimization algorithms with applications to manufacturing models, in Proceedings of the Third Model-Oriented Data Analysis, H. P. W. W. G. Müller and A. A. Zhigljavsky, eds., Physica-Verlag, Heidelberg, 1993, pp. 213-226.
45. G. YIN AND K. YIN, Asymptotically optimal rate of convergence of smoothed stochastic recursive algorithms, Stochastics, 47 (1994), pp. 21-46.
46. G. YIN AND Y. M. ZHU, Almost sure convergence of stochastic approximation algorithms with non-additive noise, Internat. J. Control, 47 (1989), pp. 1361-1376.
47. M. A. ZAZANIS AND R. SURI, Convergence rates of finite-difference sensitivity estimates stochastic systems, Oper. Res., 41 (1993), pp. 694-703.