Optimization of the transient and steady-state behavior of discrete event systems

Management Science

Volumn 42, Issue 5, 1996, Pages 717-737

  Andradóttir, Sigrún ^a,b  

^a University of Wisconsin Madison   (United States)

^b Georgia Institute of Technology   (United States)

Author keywords

Gradient Estimation; Harris Recurrence; Likelihood Ratio Method; Markov Chains; Regenerative Method; Robbins Monro Algorithm; Simulation; Stochastic Approximation; Stochastic Optimization

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; DECISION THEORY; MARKOV PROCESSES; OPTIMIZATION; PARAMETER ESTIMATION; PERFORMANCE; STATE SPACE METHODS;

DISCRETE EVENT SYSTEMS; GRADIENT ESTIMATION; LIKELIHOOD RATIO METHOD; REGENERATIVE METHOD; STEADY STATE BEHAVIOR; STOCHASTIC APPROXIMATION; STOCHASTIC OPTIMIZATION; TRANSIENT BEHAVIOR;

COMPUTER SYSTEMS;

EID: 0030143339     PISSN: 00251909     EISSN: None     Source Type: Journal    
DOI: 10.1287/mnsc.42.5.717     Document Type: Article

References (46)

1. Andradóttir, S., "A New Algorithm for Stochastic Optimization," in Proc. 1990 Winter Simulation Conf., R. P. Sadowski, R. E. Nance, and O. Balci (Eds.), Institute of Electrical and Electronics Engineers, Piscataway, NJ, 1990, 364-366.
2. _, "A Projected Stochastic Approximation Algorithm," in Proc. 1991 Winter Simulation Conf., W. D. Kelton, G. M. Clark, and B. L. Nelson (Eds.), Institute of Electrical and Electronics Engineers, Piscataway, NJ, 1991, 954-957.
3. _, "Optimization of the Transient and Steady-State Behavior of Discrete Event Systems," Technical Report 93-9, Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, 1993a.
4. _, "Optimization of the Steady-State Behavior of Discrete Event Systems Modeled as Harris Recurrent Markov Chains," Technical Report 93-10, Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, 1993b.
5. _, "A Stochastic Approximation Algorithm with Varying Bounds," Oper. Res., 43 (1995), 1037-1048.
6. _, "A Scaled Stochastic Approximation Algorithm," to appear in Management Sci., 42 (1996).
7. _, D. P. Heyman, and T. J. Ott, "On the Choice of Alternative Measures in Importance Sampling with Markov Chains," Oper. Res., 43 (1995), 509-519.
8. _, _, and _, "Potentially Unlimited Variance Reduction in Importance Sampling of Markov Chains," to appear in Advances Applied Prob., 28 (1996).
9. _, J. M. Calvin, and P. W. Glynn, "Accelerated Regeneration for Markov Chain Simulations," Probability Engineering Informational Sci., 9 (1995), 497-523.
10. Asmussen, S., Applied Probability and Queues, Wiley, Chichester, 1987.
11. _ and R. Y. Rubinstein, "The Performance of Likelihood Ratio Estimators using the Score Function," Working Paper, Chalmers University of Technology, Göteborg, Sweden, 1991.
12. Athreya, K. B. and P. Ney, "A New Approach to the Limit Theory of Recurrent Markov Chains," Trans. Amer. Math. Soc., 245 (1978), 493-501.
13. Billingsley, P., Probability and Measure, 2nd ed., Wiley, New York, 1986.
14. Chen, H. F. and Y. M. Zhu, "Stochastic Approximation Procedures with Randomly Varying Truncations," Scientia Sinica (Series A), 29 (1986), 914-926.
15. Doob, J. L., Stochastic Processes, Wiley, New York, 1953.
16. Glasserman, P., Gradient Estimation via Perturbation Analysis, Kluwer, Norwell, MA, 1991.
17. Glynn, P. W., Simulation Output Analysis for General State Space Markov Chains, Ph.D. dissertation, Department of Operations Research, Stanford University, Stanford, CA, 1982.
18. _, "Stochastic Approximation for Monte Carlo Optimization," in Proc. 1986 Winter Simulation Conf., J. R. Wilson, J. O. Henriksen, \ and S. D. Roberts (Eds.), Institute of Electrical and Electronics Engineers, Piscataway, NJ, 1986, 356-365.
19. _, "Likelihood Ratio Gradient Estimation: an Overview," in Proc. 1987 Winter Simulation Conf., A. Thesen, F. H. Grant, and W. D. Kelton (Eds.), Institute of Electrical and Electronics Engineers, Piscataway, NJ, 1987, 366-375.
20. _, "A GSMP Formalism for Discrete Event Systems," Proc. IEEE, 77 (1989), 14-23.
21. _ and D. L. Iglehart, "Importance Sampling for Stochastic Simularions," Management Sci., 35 (1989), 1367-1392.
22. _ and P. L'Ecuyer, "Likelihood Ratio Gradient Estimation for Stochastic Recursions," Advances Applied Prob., 27 (1995), 1019-1053.
23. Ho, Y.-C. and X.-R. Cao, Perturbation Analysis of Discrete Event Dynamical Systems, Kluwer, Norwell, MA, 1991.
24. Kushner, H. J. and D. S. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems, Springer-Verlag, New York, NY, 1978.
25. _ and J. Yang, "Stochastic Approximation with Averaging of the Iterates: Optimal Asymptotic Rate of Convergence for General Processes," SIAM J. Control and Optimization, 31 (1993), 1045-1062.
26. Lai, T. L. and H. Robbins, "Adaptive Design and Stochastic Approximation," Ann. Statist., 7 (1979), 1196-1221.
27. L'Ecuyer, P. and P. W. Glynn, "Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-State," Management Sci., 40 (1994), 1562-1578.
28. _, N. Giroux, and P. W. Glynn, "Stochastic Optimization by Simulation: Numerical Experiments with a Simple Queue in Steady-State," Management Sci., 40 (1994), 1245-1261.
29. Métivier, M. and P. Priouret, "Applications of a Kushner and Clark Lemma to General Classes of Stochastic Algorithms," IEEE Trans. Information Theory, IT-30 (1984), 140-151.
30. Nevel'son, M. B. and R. Z. Has'minskii, "An Adaptive Robbins-Monro Procedure," Automat. Remote Control, 34 (1973), 1594-1607.
31. Nummelin, E., "A Splitting Technique for Harris Recurrent Markov Chains," Z. Wahrsch. Verw. Gebiete, 43 (1978), 309-318.
32. Orey, S., Limit Theorems for Markov Chain Transition Probabilities, Van Nostrand Reinhold, London, 1971.
33. Pflug, G. Ch., "On-line Optimization of Simulated Markovian Processes," Math. Oper. Res., 15 (1990), 381-395.
34. Polyak, B. T., "New Method of Stochastic Approximation Type," Automat. Remote Control, 51 (1990), 937-946.
35. _ and A. B. Juditsky, "Acceleration of Stochastic Approximation by Averaging," SIAM J. Control and Optimization, 30 (1992), 838-855.
36. Robbins, H. and S. Monro, "A Stochastic Approximation Method," Ann. Math. Stat., 22 (1951), 400-407.
37. Ruppert, D., "A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure," Ann. Statist., 13 (1985), 236-245.
38. Shahabuddin, P., "Importance Sampling for the Simulation of Highly Reliable Markovian Systems," Management Sci., 40 (1994), 333-352.
39. Shedler, G. S., Regenerative Stochastic Simulation, Academic Press, Boston, MA, 1993.
40. Sigman, K., "Queues as Harris Recurrent Markov Chains," Queueing Systems, 3 (1988), 179-198.
41. Suri, R., "Perturbation Analysis: the State of the Art and Research Issues Explained via the GI/G/1 Queue," Proc. IEEE, 77 (1989), 114-137.
42. Venter, J. H., "An Extension of the Robbins-Monro Procedure," Ann. Math. Stat., 38 (1967), 181-190.
43. Wei, C. Z., "Multivariate Adaptive Stochastic Approximation," Ann. Statist., 15 (1987), 1115-1130.
44. Whitt, W., "Continuity of Generalized Semi-Markov Processes," Math. Oper. Res., 5 (1980), 494-501.
45. Yin, G., "On Extensions of Polyak's Averaging Approach to Stochastic Approximation," Stoch. Stoch. Rep., 36 (1991), 245-264.
46. _ and Y. M. Zhu, "Almost Sure Convergence of Stochastic Approximation Algorithms with Non-Additive Noise," International J. Control, 49 (1989), 1361-1376.