Dynamic importance sampling for uniformly recurrent markov chains

Annals of Applied Probability

Volumn 15, Issue 1 A, 2005, Pages 1-38

  Dupuis, Paul ^a   Wang, Hui ^a  

^a BROWN UNIVERSITY   (United States)

Author keywords

Asymptotic optimality; Importance sampling; Markov chain; Monte carlo simulation; Rare events; Stochastic game; Weak convergence

Indexed keywords

EID: 14544270331     PISSN: 10505164     EISSN: None     Source Type: Journal    
DOI: 10.1214/105051604000001016     Document Type: Article

References (34)

1. ASMUSSEN, S. (1985). Conjugate processes and the simulation of ruin probability. Stochastic Process. Appl. 20 213-229.
2. ASMUSSEN, S. (1989). Risk theory in a Markovian environment. Scand. Actuar. J. 89 69-100.
3. ASMUSSEN, S., RUBINSTEIN, R. and WANG, C. L. (1994). Regenerative rare event simulation via likelihood ratios. J. Appl. Probab. 31 797-815.
4. BERTSEKAS, D. and SHREVE, S. (1978). Stochastic Optimal Control: The Discrete Time Case. Academic Press, San Diego, CA.
5. BILLINGSLEY, P. (1995). Probability and Measure, 3rd ed. Wiley, New York.
6. BREIMAN, L. (1968). Probability Theory. Addison-Wesley, Reading, MA.
7. BUCKLEW, J. A. (1990). Large Deviations Techniques in Decision, Simulation and Estimation. Wiley, New York.
8. BUCKLEW, J. A. (1998). The blind simulation problem and regenerative processes. IEEE Trans. Inform. Theory 44 2877-2891.
9. BUCKLEW, J. A., NEY, P. and SADOWSKY, J. S. (1990). Monte Carlo simulation and large deviations theory for uniformly recurrent Markov chains. J. Appl. Probab. 27 44-59.
10. CHEN, J., LU, D., SADOWSKY, J. S. and YAO, K. (1993). On importance sampling in digital communications. Part I: Fundamentals. IEEE J. Selected Areas in Comm. 11 289-307.
11. COLLAMORE, J. F. (2002). Importance sampling techniques for the multidimensional ruin problem for general Markov additive sequences of random vectors. Ann. Appl. Probab. 12382-421.
12. COTTREL, M., FORT, J. C. and MALGOUYNES, G. (1983). Large deviations and rare events in the study of stochastic algorithms. IEEE Trans. Automat. Control AC-28 907-920.
13. DE ACOSTA, A. (1990). Large deviations for empirical measures of Markov chains. J. Theoret. Probab. 3 395-431.
14. DUPUIS, P. and ELLIS, R. S. (1997). A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York.
15. DUPUIS, P. and KUSHNER, H. J. (1987). Stochastic systems with small noise, analysis and simulation; a phase locked loop example. SIAM J. Appl. Math. 47 643-661.
16. DUPUIS, P. and WANG, H. (2005). Importance sampling, large deviations, and differential games. Ann. Probab. To appear.
17. GLASSERMAN, P. (2004). Monte Carlo Methods in Financial Engineering. Springer, New York.
18. GLASSERMAN, P. and KOU, S. (1995). Analysis of an importance sampling estimator for tandem queues. ACM Trans. Modeling Comp. Simulation 4 22-42.
19. GLASSERMAN, P. and WANG, Y. (1997). Counter examples in importance sampling for large deviations probabilities. Ann. Appl. Probab. 7 731-746.
20. GLYNN, P. W. (1995). Large deviations for the infinite server queue in heavy traffic. IMA Vol. Math. Appl. 71 387-394.
21. HEIDELBERGER, P. (1995). Fast simulation of rare events in queueing and reliability models. ACM Trans. Modeling Comp. Simulation 4 43-85.
22. ISCOE, I., NEY, P. and NUMMELIN, E. (1985). Large deviations of uniformly recurrent Markov additive processes. Adv. in Appl. Math. 6 373-412.
23. KUSHNER, H. J. (1990). Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems. Birkhäuser, Boston.
24. LEHTONEN, T. and NYHRINEN, H. (1992). Simulating level crossing probabilities by importance sampling. Adv. in Appl. Probab. 24 858-874.
25. LEHTONEN, T. and NYHRINEN, H. (1992). On asymptotically efficient simulating of ruin probabilities in a Markovian environment. Scand. Actuar. J. 1 60-75.
26. NEY, P. and NUMMELIN, E. (1987). Markov additive processes I: Eigenvalue properties and limit theorems. Ann. Probab. 15 561-592.
27. NEY, P. and NUMMELIN, E. (1987). Markov additive processes II: Large deviations. Ann. Probab. 15 593-609.
28. ROCKAFELLAR, R. T. (1970). Convex Analysis. Princeton Univ. Press.
29. SADOWSKY, J. S. (1991). Large deviations and efficient simulation of excessive backlogs in a GI/G/m queue. IEEE Trans. Automat. Control 36 1383-1394.
30. SADOWSKY, J. S. (1993). On the optimality and stability of exponential twisting in Monte Carlo estimation. IEEE Trans. Inform. Theory 39 119-128.
31. SADOWSKY, J. S. (1996). On Monte Carlo estimation of large deviations probabilities. Ann. Appl. Probab. 6 399-422.
32. SADOWSKY, J. S. and BUCKLEW, J. A. (1990). On large deviations theory and asymptotically efficient Monte Carlo estimation. IEEE Trans. Inform. Theory 36 579-588.
33. SHAHSBUDDIN, P. (1994). Importance sampling for the simulation of highly reliable Markovian systems. Management Sci. 40 333-352.
34. SIEGMUND, D. (1976). Importance sampling in the Monte Carlo study of sequential tests. Ann. Statist. 4 673-684.