Zero-variance importance sampling estimators for Markov process expectations

Mathematics of Operations Research

Volumn 38, Issue 2, 2013, Pages 358-388

  Awad, Hernan P ^a   Glynn, Peter W ^b   Rubinstein, Reuven Y ^c  

^a UNIVERSITY OF MIAMI   (United States)

^b Stanford University   (United States)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Importance sampling; Markov process; Simulation

Indexed keywords

DISCOUNTED REWARD; FUNCTIONALS; IMPORTANCE MEASURE; INFINITE HORIZONS; LYAPUNOV INEQUALITY; MARKOVIAN; POSITIVE SOLUTION; SIMULATION;

IMPORTANCE SAMPLING; LINEAR SYSTEMS; MARKOV PROCESSES;

ESTIMATION;

EID: 84878288428     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1120.0569     Document Type: Article

References (36)

1. Ahamed TPI, Borkar VS, Juneja S (2006) Adaptive importance sampling technique for Markov chains using stochastic approximation. Oper. Res. 54(3): 489-504.
2. Baggerly K, Cox D, Picard R (2000) Exponential convergence of adaptive importance sampling for Markov chains. J. Appl. Probab. 37(2): 342-358.
3. Blanchet J, Glynn P (2008) Efficient rare event simulation for the maximum of heavy-tailed random walks. Ann. Appl. Probab. 18(4): 1351-1378.
4. Bolia N, Juneja S, Glasserman P (2004) Function-approximation-based importance sampling for pricing American options. Ingalls RG, Rosetti MD, Smith JS, Peters BA, eds. Proc. 2004 Winter Simulation Conf. (IEEE Computer Society, Washington, DC), 604-611.
5. Booth TE (1989) Zero-variance solutions for linear Monte-Carlo. Nuclear Sci. Engrg. 102(4): 332-340.
6. Borkar VS, Juneja S, Kherani AA (2004) Performance analysis conditioned on rare events: An adaptive simulation scheme. Comm. Inform. Systems 3(4): 259-278.
7. Borodin AN, Salminen P (2002) Handbook of Brownian Motion : Facts and Formulae, 2nd ed. (Birkhäuser, Basel, Boston).
8. Chang C-S, Heidelberger P, Juneja S, Shahabuddin P (1994) Effective bandwidth and fast simulation of ATM intree networks. Performance Eval. 20(1): 45-66.
9. Chung KL (1967) Markov Chains with Stationary Transition Probabilities, 2nd ed. (Springer, Berlin).
10. Dupuis P, Wang H (2004) Importance sampling, large deviations, and differential games. Stochastic and Stochastics Rep. 76(6): 481-508.
11. Dupuis P, Wang H (2005) Dynamic importance sampling for uniformly recurrent Markov chains. Ann. Appl. Probab. 15(1A): 1-38.
12. Fox BL, Glynn PW (1990) Discrete-time conversion for simulating finite-horizon Markov processes. SIAM J. Appl. Math. 50(5): 1457-1473.
13. Glasserman P (1993) Filtered Monte Carlo. Math. Oper. Res. 18(3): 610-634.
14. Glasserman P (2004) Monte Carlo Methods in Financial Engineering (Springer, New York).
15. Glasserman P, Heidelberger P, Shahabuddin P (1999) Asymptotically optimal importance sampling and stratification for pricing pathdependent options. Math. Finance 9(2): 117-152.
16. Glynn PW, Iglehart DL (1989) Importance sampling for stochastic simulations. Management Sci. 35(11): 1367-1392.
17. Glynn PW, Heidelberger P, Nicola VF, Shahabuddin P (1993) Efficient estimation of the mean time between failures in non-regenerative dependability models. Evans GW, Mollaghasemi M, Russell EC, Biles WE, eds. Proc. 1993 Winter Simulation Conf. (IEEE Computer Society, Washington, DC), 361-366.
18. Goyal A, Shahabuddin P, Heidelberger P, Nicola VF, Glynn PW (1992) A unified framework for simulating Markovian models of highly dependable systems. IEEE Trans. Comput. 41(1): 36-51.
19. Halton JH (1962) Sequential Monte Carlo. Proc. Cambridge Philos. Soc. 58: 57-78.
20. Hammersley JM, Handscomb DC (1964) Monte Carlo Methods, Methuen's Monographs on Applied Probability and Statistics, (Methuen, London).
21. Harrison JM (1985) Brownian Motion and Stochastic Flow Systems (Wiley, New York).
22. Heidelberger P (1995) Fast simulation of rare events in queueing and reliability models. ACM Trans. Modeling Comput. Simulation 5(1): 43-85.
23. Heidelberger P, Shahabuddin P, Nicola VF (1994) Bounded relative error in estimating transient measures of highly dependable non-Markovian systems. ACM Trans. Modeling Comput. Simulation 4(2): 137-164.
24. Henderson SG, Glynn PW (2002) Approximating martingales for variance reduction in Markov process simulation. Math. Oper. Res. 27(2): 253-271.
25. Karatzas I, Shreve S (1991) Brownian Motion and Stochastic Calculus, 2nd ed. (Springer-Verlag, New York).
26. Kollman C, Baggerly K, Cox D, Picard R (1999) Adaptive importance sampling on discrete Markov chains. Ann. Appl. Probab. 9(2): 391-412.
27. L'Ecuyer P, Tuffin B (2008) Approximate zero-variance simulation. Mason SJ, Hill RR, Mönch L, Rose O, Jefferson T, Fowler JW, eds. Proc. 2008 Winter Simulation Conf. (IEEE Computer Society, Washington, DC), 170-181.
28. Meyn S, Tweedie RL (2009) Markov Chains and Stochastic Stability, 2nd ed. (Cambridge University Press, New York).
29. Nakayama MK (1996) General conditions for bounded relative error in simulations of highly reliable Markovian systems. Adv. Appl. Probab. 28(3): 687-727.
30. Øksendal B (2000) Stochastic Differential Equations: An Introduction with Applications, 5th ed. (Springer, Berlin).
31. Sadowsky JS (1991) Large deviations theory and efficient simulation of excessive backlogs in a GI=GI=m queue. IEEE Trans. Automatic Control 36(12): 1383-1394.
32. Salminen P, Norros I (2001) On busy periods of the unbounded Brownian storage. Queueing Systems 39(4): 317-333.
33. Shahabuddin P (1994) Importance sampling for the simulation of highly reliable Markovian systems. Management Sci. 40(3): 333-352.
34. Smith PJ, Shafi M, Gao H (1997) Quick simulation: A review of importance sampling techniques in communications systems. IEEE J. Selected Areas Comm. 15(4): 597-613.
35. Su Y, Fu MC (2000) Importance sampling in derivative securities pricing. Joines JA, Barton RR, Kang K, Fishwick PA, eds. Proc. 2000 Winter Simulation Conf. (IEEE Computer Society, Washington, DC), 587-596.
36. Whitt W (2002) Stochastic-Process Limits: An Introduction to Stochastic-Process Limits and Their Application to Queues, Springer Series in Operations Research, (Springer-Verlag, New York).