Chapter 19 Gradient Estimation

Handbooks in Operations Research and Management Science

Volumn 13, Issue C, 2006, Pages 575-616

  Fu, Michael C ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77950463930     PISSN: 09270507     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S0927-0507(06)13019-4     Document Type: Review

References (103)

1. Andradóttir S. Optimization of the transient and steady-state behavior of discrete event systems. Management Science 42 (1996) 717-737
2. Andradóttir S. Simulation optimization. In: Banks J. (Ed). Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice (1998), Wiley, New York 307-333 Chapter 9
3. Banks J., Carson J.S., Nelson B.L., and Nicol D.M. Discrete Event Systems Simulation. 3rd edition (2000), Prentice Hall, Englewood Cliffs, NJ
4. Bashyam S., and Fu M.C. Application of perturbation analysis to a class of periodic review (s, S) inventory systems. Naval Research Logistics 41 (1994) 47-80
5. Bashyam S., and Fu M.C. Optimization of (s, S) inventory systems with random lead times and a service level constraint. Management Science 44 (1998) S243-S256
6. Benhamou E. Smart Monte Carlo: Various tricks using Malliavin calculus. Quantitative Finance 2 (2002) 329-336
7. Bhatnagar S., Fu M.C., Marcus S.I., and Wang I.J. Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences. ACM Transactions on Modeling and Computer Simulation 13 (2003) 180-209
8. Bowman R.A. Stochastic gradient-based time-cost tradeoffs in PERT network using simulation. Annals of Operations Research 53 (1994) 533-551
9. Brémaud P., and Vázquez-Abad F.J. On the pathwise computation of derivatives with respect to the rate of a point process: The phantom RPA method. Queueing Systems: Theory and Applications 10 (1992) 249-270
10. Broadie M., and Glasserman P. Estimating security price derivatives using simulation. Management Science 42 (1996) 269-285
11. Cao X.R. Convergence of parameter sensitivity estimates in a stochastic experiment. IEEE Transactions on Automatic Control 30 (1985) 834-843
12. Cao X.R. Realization Probabilities: The Dynamics of Queuing Systems (1994), Springer-Verlag, New York
13. Cao X.R. A unified approach to Markov decision problems and performance sensitivity analysis. Automatica 36 (2000) 771-774
14. Cao X.R. From perturbation analysis to Markov decision processes and reinforcement learning. Discrete Event Dynamic Systems: Theory and Applications 13 (2003) 9-39
15. Cao X.R. Semi-Markov decision problems and performance sensitivity analysis. IEEE Transactions on Automatic Control 48 (2003) 758-769
16. Cassandras C.G., and Strickland S.G. On-line sensitivity analysis of Markov chains. IEEE Transactions on Automatic Control 34 (1989) 76-86
17. Cassandras C.G., Wardi Y., Melamed B., Sun G., and Panayiotou C.G. Perturbation analysis for on-line control and optimization of stochastic fluid models. IEEE Transactions on Automatic Control 47 (2002) 1234-1248
18. Chen J., and Fu M.C. Hedging beyond duration and convexity. Proceedings of the 2002 Winter Simulation Conference (2002), IEEE Press, Piscataway, NJ 1593-1599
19. Chong E.K.P., and Ramadge P.J. Convergence of recursive optimization algorithms using infinitesimal perturbation analysis. Discrete Event Dynamic Systems: Theory and Applications 1 (1992) 339-372
20. Clewlow L.J., and Strickland C.R. Implementing Derivative Models (1999), Wiley, New York
21. Dai L., and Ho Y.C. Structural infinitesimal perturbation analysis for derivative estimation in discrete event dynamic systems. IEEE Transactions on Automatic Control 40 (1995) 1154-1166
22. Dussault J.P., Labrecque D., L'Ecuyer P., and Rubinstein R.Y. Combining the stochastic counterpart and stochastic approximation methods. Discrete Event Dynamic Systems: Theory and Applications 7 (1997) 5-28
23. Fournié E., Lasry J.M., Lebuchoux J., Lions P.L., and Touzi N. Applications of Malliavin calculus to Monte Carlo methods in finance. Finance and Stochastics 3 (1999) 391-412
24. Fu M.C. Convergence of a stochastic approximation algorithm for the GI / G / 1 queue using infinitesimal perturbation analysis. Journal of Optimization Theory and Applications 65 (1990) 149-160
25. Fu M.C. Optimization via simulation: A review. Annals of Operations Research 53 (1994) 199-248
26. Fu M.C. Sample path derivatives for (s, S) inventory systems. Operations Research 42 (1994) 351-364
27. Fu M.C. Perturbation analysis. In: Gass S.I., and Harris C.M. (Eds). Encyclopedia of Operations Research and Management Science. 2nd edition (2001), Kluwer Academic Publishers, Boston, MA 608-611
28. Fu M.C. Simulation optimization. In: Gass S.I., and Harris C.M. (Eds). Encyclopedia of Operations Research and Management Science. 2nd edition (2001), Kluwer Academic Publishers, Boston, MA 756-759
29. Fu M.C. Optimization for simulation: Theory vs. practice (Feature Article). INFORMS Journal on Computing 14 (2002) 192-215
30. Fu, M.C. (2006). Sensitivity analysis for stochastic activity networks. In: Alt, F.B., Fu, M.C., Golden, B.L. (Eds.), Topics in Modeling, Optimization, and Decision Technologies: Honoring Saul Gass' Contributions to Operations Research. Kluwer Academic Publishers, Boston, MA, in press
31. Fu M.C., and Healy K.J. Simulation optimization of (s, S) inventory systems. Proceedings of the Winter Simulation Conference (1992), IEEE Press, Piscataway, NJ 506-514
32. Fu M.C., and Healy K.J. Techniques for simulation optimization: An experimental study on an (s, S) inventory system. IIE Transactions 29 (1997) 191-199
33. Fu M.C., and Hill S.D. Optimization of discrete event systems via simultaneous perturbation stochastic approximation. IIE Transactions 29 (1997) 233-243
34. Fu M.C., and Hu J.Q. On choosing the characterization for smoothed perturbation analysis. IEEE Transactions on Automatic Control 36 (1991) 1331-1336
35. Fu M.C., and Hu J.Q. Extensions and generalizations of smoothed perturbation analysis in a generalized semi-Markov process framework. IEEE Transactions on Automatic Control 37 (1992) 1483-1500
36. Fu M.C., and Hu J.Q. Second derivative sample path estimators for the GI / G / m queue. Management Science 39 (1993) 359-383
37. Fu M.C., and Hu J.Q. (s, S) inventory systems with random lead times: Harris recurrence and its implications in sensitivity analysis. Probability in the Engineering and Informational Sciences 8 (1994) 355-376
38. Fu M.C., and Hu J.Q. Sensitivity analysis for Monte Carlo simulation of option pricing. Probability in the Engineering and Information Sciences 9 (1995) 417-446
39. Fu M.C., and Hu J.Q. Conditional Monte Carlo: Gradient Estimation and Optimization Applications (1997), Kluwer Academic Publishers, Boston, MA
40. Fu M.C., and Hu J.Q. Efficient design and sensitivity analysis of control charts using Monte Carlo simulation. Management Science 45 (1999) 395-413
41. Fu M.C., Hu J.Q., and Shi L. An application of perturbation analysis to replacement problems in maintenance. Proceedings of the 1993 Winter Simulation Conference (1993), IEEE Press, Piscataway, NJ 329-337
42. Gaivoronski A., Shi L., and Sreenivas R.S. Augmented infinitesimal perturbation analysis: An alternate explanation. Discrete Event Dynamic Systems: Theory and Applications 2 (1992) 121-138
43. Glasserman P. Gradient Estimation via Perturbation Analysis (1991), Kluwer Academic Publishers, Boston, MA
44. Glasserman P. Monte Carlo Methods in Financial Engineering (2004), Springer-Verlag, New York
45. Glynn P.W. Likelihood ratio gradient estimation for stochastic systems. Communicatios of the ACM 33 (1990) 75-84
46. Gong W.B., and Ho Y.C. Smoothed perturbation analysis of discrete-event dynamic systems. IEEE Transactions on Automatic Control 32 (1987) 858-867
47. Gürkan G., Özge A.Y., and Robinson S.M. Sample-path solution of stochastic variational inequalities. Mathematical Programming 84 (1999) 313-333
48. Heidelberger P., Cao X.R., Zazanis M., and Suri R. Convergence properties of infinitesimal perturbation analysis estimates. Management Science 34 (1988) 1281-1302
49. Heidergott B. Optimization of a single-component maintenance system: A smoothed perturbation analysis approach. European Journal of Operational Research 119 (1999) 181-190
50. Heidergott B. A weak derivative approach to optimization of threshold parameters in a multi-component maintenance system. Journal of Applied Probability 38 (2001) 386-406
51. Heidergott B. Option pricing via Monte Carlo simulation: A weak derivative approach. Probability in Engineering and Informational Sciences 15 (2001) 335-349
52. Heidergott, B., Vázquez-Abad, F. (2006a). Measure-valued differentiation for random horizon problems. Markov Processes and Related Fields, in press
53. Heidergott, B., Vázquez-Abad, F. (2006b). Measure-valued differentiation for Markov chains. Journal of Optimization Theory and Applications, in press
54. Heidergott B., Pflug G., and Vázquez-Abad F. Measure-valued differentiation for stochastic systems: From simple distributions to Markov chains. Manuscript. Available at (2003). http://staff.feweb.vu.nl/bheidergott/
55. Ho Y.C., and Cao X.R. Discrete Event Dynamics Systems and Perturbation Analysis (1991), Kluwer Academic Publishers, Boston, MA
56. Ho Y.C., and Li S. Extensions of infinitesimal perturbation analysis. IEEE Transactions on Automatic Control 33 (1988) 827-838
57. Homem-de-Mello T., Shapiro A., and Spearman M.L. Finding optimal material release times using simulation based optimization. Management Science 45 (1999) 86-102
58. Howell W.C., and Fu M.C. Application of perturbation analysis to traffic light signal timing. Proceedings of the 42nd IEEE Conference on Decision and Control (2003), IEEE Press, Piscataway, NJ 4837-4840
59. Hull J.C. Options, Futures, and Other Derivative Securities. 6th edition (2005), Prentice Hall, Englewood Cliffs, NJ
60. Jacobson S.H. Convergence results for harmonic gradient estimators. ORSA Journal on Computing 6 (1994) 381-397
61. Jacobson S.H., and Schruben L.W. A review of techniques for simulation optimization. Operations Research Letters 8 (1989) 1-9
62. Jacobson S.H., Buss A., and Schruben L.W. Driving frequency selection for frequency domain simulation experiments. Operations Research 39 (1991) 917-924
63. Kapuscinski R., and Tayur S.R. Optimal policies and simulation based optimization for capacitated production inventory systems. In: Tayur S.R., Ganeshan R., and Magazine M.J. (Eds). Quantitative Models for Supply Chain Management (1999), Kluwer Academic Publishers, Boston, MA Chapter 2
64. Kiefer J., and Wolfowitz J. Stochastic estimation of the maximum of a regression function. Annals of Mathematical Statistics 23 (1952) 462-466
65. Kushner H.J., and Clark D.S. Stochastic Approximation Methods for Constrained and Unconstrained Systems (1978), Springer-Verlag, New York
66. Kushner H.J., and Yin G.G. Stochastic Approximation Algorithms and Applications (1997), Springer-Verlag, New York
67. Law A.M., and Kelton W.D. Simulation Modeling and Analysis. 3rd edition (2000), McGraw-Hill, New York
68. L'Ecuyer P. A unified view of the IPA, SF, and LR gradient estimation techniques. Management Science 36 (1990) 1364-1383
69. L'Ecuyer P. On the interchange of derivative and expectation for likelihood ratio derivative estimators. Management Science 41 (1995) 738-748
70. L'Ecuyer P., and Glynn P.W. Stochastic optimization by simulation: Convergence proofs for the GI / G / 1 queue in steady-state. Management Science 40 (1994) 1562-1578
71. L'Ecuyer P., and Perron G. On the convergence rates of IPA and FDC derivative estimators. Operations Research 42 (1994) 643-656
72. L'Ecuyer P., and Yin G. Budget-dependent convergence rate of stochastic approximation. SIAM Journal on Optimization 8 (1998) 217-247
73. L'Ecuyer P., Giroux N., and Glynn P.W. Stochastic optimization by simulation: Numerical experiments with a simple queue in steady-state. Management Science 40 (1994) 1245-1261
74. L'Ecuyer P., Martin B., and Vázquez-Abad F. Functional estimation for a multicomponent age-replacement model. American Journal of Mathematical and Management Sciences 19 (1999) 135-156
75. McLeish D.L., and Rollins S. Conditioning for variance reduction in estimating the sensitivity of simulations. Annals of Operations Research 39 (1992) 157-173
76. Panayiotou, C.G., Howell, W.C., Fu, M.C. (2005). Online traffic light control through gradient estimation using stochastic fluid models. Proceedings of the IFAC 16th Triennial World Congress. CD-ROM
77. Pflug G.C. Sampling derivatives of probabilities. Computing 42 (1989) 315-328
78. Pflug G.C. On-line optimization of simulated Markovian processes. Mathematics of Operations Research 15 (1990) 381-395
79. Pflug G.C. Optimization of Stochastic Models (1996), Kluwer Academic Publishers, Boston, MA
80. Pflug G.C., and Rubinstein R.Y. Inventory processes: Quasi-regenerative property, performance evaluation and sensitivity estimation via simulation. Stochastic Models 18 (2002) 469-496
81. Plambeck E.L., Fu B.-R., Robinson S.M., and Suri R. Sample-path optimization of convex stochastic performance functions. Mathematical Programming 75 (1996) 137-176
82. Robinson S.M. Analysis of sample-path optimization. Mathematics of Operations Research 21 (1996) 513-528
83. Reiman M.I., and Weiss A. Sensitivity analysis for simulations via likelihood ratios. Operations Research 37 (1989) 830-844
84. Robbins H., and Monro S. A stochastic approximation method. Annals of Mathematical Statistics 22 (1951) 400-407
85. Rubinstein R.Y. Sensitivity analysis of computer simulation models via the score efficient. Operations Research 37 (1989) 72-81
86. Rubinstein R.Y. Sensitivity analysis of discrete event systems by the 'push out' method. Annals of Operations Research 39 (1992) 229-251
87. Rubinstein R.Y., and Shapiro A. Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method (1993), Wiley, New York
88. Schruben L.W. Simulation optimization using frequency domain methods. Proceedings of the Winter Simulation Conference (1986), IEEE Press, Piscataway, NJ 366-369
89. Schruben L.W., and Cogliano V.J. Simulation sensitivity analysis: A frequency domain approach. Proceedings of the Winter Simulation Conference (1981), IEEE Press, Piscataway, NJ 455-459
90. Shapiro A. Monte Carlo sampling methods. In: Ruszczynski A., and Shapiro A. (Eds). Stochastic Programming. Handbook in Operations Research and Management Science (2003), Elsevier, New York Chapter 6
91. Shapiro A., and Homem-de-Mello T. A simulation-based approach to two-stage stochastic programming with recourse. Mathematical Programming 81 (1998) 301-325
92. Shi L.Y. Discontinuous perturbation analysis of discrete event dynamic systems. IEEE Transactions on Automatic Control 41 (1996) 1676-1681
93. Spall J.C. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37 (1992) 332-341
94. Spall J.C. Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Transactions on Automatic Control 45 (2000) 1839-1853
95. Spall J.C. Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control (2003), Wiley, Hoboken, NJ
96. Sun G., Cassandras C.G., Wardi Y., Panayiotou C.G., and Riley G.F. Perturbation analysis and optimization of stochastic flow networks. IEEE Transactions on Automatic Control 49 (2004) 2143-2159
97. Suri R., and Zazanis M.A. Perturbation analysis gives strongly consistent sensitivity estimates for the M / G / 1 queue. Management Science 34 (1988) 39-64
98. Tang Q.-Y., L'Ecuyer P., and Chen H.-F. Asymptotic efficiency of perturbation analysis-based stochastic approximation with averaging. SIAM Journal on Control and Optimization 37 (1999) 1822-1847
99. Vakili P. Using a standard clock technique for efficient simulation. Operations Research Letters 10 (1991) 445-452
100. Wardi Y., Melamed B., Cassandras C.G., and Panayiotou C.G. IPA gradient estimators in single-node stochastic fluid models. Journal of Optimization Theory and Applications 115 (2002) 369-406
101. Xiong X., Wang I.J., and Fu M.C. Randomized-direction stochastic approximation algorithms using deterministic perturbation sequences. Proceedings of the 2002 Winter Simulation Conference (2002), IEEE Press, Piscataway, NJ 285-291
102. Zazanis M.A., and Suri R. Convergence rates of finite-difference sensitivity estimates for stochastic systems. Operations Research 41 (1993) 694-703
103. Zhang H., and Fu M.C. Sample path derivatives for (s, S) inventory systems with price determination. In: Golden B.L., Raghavan S., and Wasil E.A. (Eds). The Next Wave in Computing, Optimization, and Decision Technologies (2005), Kluwer Academic Publishers, Boston, MA 229-246