Stochastic approximation methods for systems over an infinite horizon

SIAM Journal on Control and Optimization

Volumn 34, Issue 2, 1996, Pages 712-756

  Kushner, Harold J ^a   Vázquez Abad, Felisa J ^b  

^a BROWN UNIVERSITY   (United States)

^b UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; DIFFERENTIAL EQUATIONS; MARKOV PROCESSES; MATHEMATICAL MODELS; MONTE CARLO METHODS; OPTIMIZATION; PROBLEM SOLVING; RECURSIVE FUNCTIONS;

DISCRETE EVENT DYNAMICAL SYSTEMS; ORDINARY DIFFERENTIAL EQUATIONS; PIECEWISE DETERMINISTIC DYNAMICAL SYSTEMS; STATIONARY COST PROBLEMS; WEAK CONVERGENCE;

APPROXIMATION THEORY;

EID: 0030105021     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012994263743     Document Type: Article

References (47)

1. M. BENAIM, A dynamical systems approach to stochastic approximations, Univ. of California, Berkeley, CA, 1993, preprint.
2. A. BENVENISTE, M. METIVIER, AND P. PRIOURET, Adaptive Algorithms and Stochastic Approximation, Springer-Verlag, New York, Berlin, 1990.
3. P. BILLINGSLEY, Convergence of Probability Measures, John Wiley, New York, 1968.
4. P. BRÉMAUD AND F. J. VÁZQUEZ-ABAD, On the pathwise computation of derivatives with respect to the rate of a point process: The phantom RPA method, Queueing Systems Theory Appl., 10 (1992), pp. 249-270.
5. F. CAMPILLO AND E. PARDOUX, Numerical methods in ergodic optimal stochastic control, in Applied Stochastic Analysis, I. Karatzas and D. Ocone, eds., Springer-Verlag, Berlin, 1991, pp. 59-73.
6. M. CARAMANIS AND G. LIBEROPOULOS, Perturbation analysis for the design of flexible manufacturing system flow controllers, Oper. Res., 40 (1992), pp. 1107-1125.
7. E. K. P. CHONG AND P. J. RAMADGE, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Control Optim., 31 (1993), pp. 698-732.
8. M. H. A. DAVIS, Markov Models and Optimization, Chapman-Hall, New York, London, 1993.
9. P. DUPUIS AND H. J. KUSHNER, Stochastic approximation and large deviations: Upper bounds and w.p.1 convergence, SIAM J. Control Optim., 27 (1989), pp. 1108-1135.
10. S. N. ETHIER AND T. G. KURTZ, Markov Processes: Characterization and Convergence, John Wiley, New York, 1986.
11. M. C. FU, Convergence of the stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theory Appl., 65 (1990), pp. 149-160.
12. I. I. GIHMAN AND A. V. SKOROHOD, Stochastic Differential Equations, Springer-Verlag, Berlin, 1972.
13. P. GLASSERMAN, Gradient Estimation via Perturbation Analysis, Kluwer, Amsterdam, 1990.
14. _, Filtered Monte Carlo, Math. Oper. Res., 18 (1993), pp. 610-634.
15. J. GUCKENHEIMER AND P. HOLMES, Nonlinear Oscillations, Dynamical Systems, and Bifurctions of Vector Fields, Springer-Verlag, Berlin, 1983.
16. A. HAURIE, P. L'ECUYER, AND CH. VAN DELFT, Convergence of stochastic approximation coupled with perturbation analysis in a class of manufacturing flow control models, J. Discrete Event Dynamical Systems, 4 (1994), pp. 87-111.
17. Y.-C. Ho AND X.-R CAO, Performance sensitivity to routing changes in queueing networks and flexible manufacturing systems using perturbation analysis, IEEE J. on Robotics and Automation, RA-1 (1985), pp. 165-172.
18. _, Perturbation Analysis of Discrete Event Dynamical Systems, Kluwer, Boston, 1991.
19. Y. C. Ho, M. A. EYLER, AND T. T. CHIEN, A gradient technique for general buffer storage design in a production line, Internat. J. of Production Research, 17 (1979), pp. 577-580.
20. _, A new approach to determine parameter sensitivities of transfer lines, Management Science, 29 (1983), pp. 700-714.
21. T. G. KURTZ, Semigroups of conditional shifts and approximation of Markov processes, Ann. Probab., 4 (1975), pp. 618-642.
22. H. J. KUSHNER, Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic System Theory, MIT Press, Cambridge, MA, 1984.
23. _, Robustness and approximation of escape times and large deviations estimates for systems with small noise effects, SIAM J. Appl. Math., 44 (1984), pp. 160-182.
24. _, Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, Systems and Control, Vol. 3, Birkhäuser, Boston, 1990.
25. H. J. KUSHNER AND D. S. CLARK, Stochastic Approximation for Constrained and Unconstrained Systems, Springer-Verlag, Berlin, New York, 1978.
26. H. J. KUSHNER AND J. YANG, A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems: The ergodic case, SIAM J. Control Optim., 30 (1992), pp. 440-464.
27. H. J. KUSHNER AND A. SHWARTZ, An invariant measure approach to the convergence of stochastic approximations with state dependent noise, SIAM J. Control Optim., 22 (1984), pp. 13-27.
28. _, Weak convergence and asymptotic properties of adaptive filters with constant gains, IEEE Trans. Inform. Theory, IT-30 (1984), pp. 177-182.
29. H. J. KUSHNER AND J. YANG, Stochastic approximation with averaging: Optimal asymptotic rates of convergence for general processes, SIAM J. Control Optim., 31 (1993), pp. 1045-1062.
30. H. J. KUSHNER AND G. YIN, Stochastic approximation algorithms for parallel and distributed processing, Stochastics, 22 (1987), pp. 219-250.
31. P. L'ECUYER, N. GIROUX, AND P. W. GLYNN, Stochastic optimization by simulation: Numerical experiments for a simple queue in steady state, Management Sci., 40 (1994), pp. 1245-1261.
32. P. L'ECUYER AND P. W. GLYNN, Stochastic optimization by simulation: Convergence proofs for the GI/G/1 queue, Management Sci., 40 (1994), pp. 1562-1578.
33. L. LJUNG, Analysis of recursive stochastic algorithms, IEEE Trans. Automat. Control, AC-22 (1977), pp. 551-575.
34. J. PAN AND C. G. CASSANDRAS, Flow control of bursty traffic using infinitesimal perturbation analysis, J. Discrete Event Systems, 4 (1994), pp. 325-358.
35. G. CH. PFLUG, On line optimization of simulated Markov processes, Math. Oper. Res., 15 (1990), pp. 381-395.
36. B. T. POLYAK AND A. B. JUDITSKY, Acceleration of stochastic approximation by averaging, SIAM J. Control Optim., 30 (1992), pp. 838-855.
37. M. I. REIMAN AND A. WEISS, Sensitivity analysis for simulation via likelihood ratios, Oper. Res., 37 (1989), pp. 930-844.
38. R. Y. RUBINSTEIN, Sensitivity analysis and performance extrapolation for computer simulation models, Oper. Res., 37 (1989), pp. 72-81.
39. R. SURI AND B. R. FU, On using continuous flow lines for performance estimation of discrete production lines, in Proc. 1991 Winter Simulation Conference, 1991, B.L. Nelson, W. D. Kelton, and G. M. Clark, eds., pp. 968-977.
40. F. J. VÁZQUEZ-ABAD, Stochastic Recursive Algorithms for Optimal Routing in Queueing Networks, Ph.D. thesis, Brown University, 1989.
41. F. J. VÁZQUEZ-ABAD AND H. J. KUSHNER, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Probab., 29 (1992), pp. 343-352.
42. _, The surrogate estimation approach for sensitivity analysis in queueing networks, in G. W. Evans, M. Mollaghasemi, E. C. Russel, and W. E. Biles, eds., Proc. Winter Simulation Conference, 1993, pp. 347-355.
43. F. J. VÁZQUEZ-ABAD AND L. MASON, Adaptive decentralized control under non uniqueness of the optimal control, J. Discrete Event Dyn. Syst., in press.
44. H. YAN, G. YIN, AND S. X. C. LOU, Using stochastic approximation to determine threshold values for control of unreliable manufacturing systems, J. Optim. Theory Appl., 83 (1994), pp. 511-539.
45. H. YAN, X.Y. ZHOU, AND G. YIN, Optimal numbers of circulating kanbans in a two machine flowshop with uncertainty, Oper. Res., to appear.
46. G. YIN, On extensions of Polyak's averaging approach to stochastic approximation, Stochastics, 36 (1992), pp. 245-264.
47. _, Stochastic approximation via averaging: Polyak's approach revisited, in Simulation and Optimization: Proceedings of the International Workshops on Computationally Intensive Methods in Simulation and Optimization held at the International Institute for Applied Systems Analysis, Laxenburg, Austria, August 23-25, 1990, Lecture Notes in Economics and Mathematical Systems 374, G. Pflug and U. Dieter, eds., Springer-Verlag, Berlin, 1992, pp. 119-134.