Sensitivity analysis using Itô-malliavin calculus and martingales, and application to stochastic optimal control

SIAM Journal on Control and Optimization

Volumn 43, Issue 5, 2005, Pages 1676-1713

  Gobet, Emmanuel ^a   Munos, Rémi ^a  

^a CENTRE DE MATHÉMATIQUES APPLIQUÉES   (France)

Author keywords

Malliavin calculus; Parameterized control; Sensitivity analysis; Weak approximation

Indexed keywords

COSTS; FINANCE; FUNCTIONS; PARAMETER ESTIMATION; PROBLEM SOLVING; SENSITIVITY ANALYSIS; STOCHASTIC CONTROL SYSTEMS;

MALLIAVIN CALCULUS; PARAMETERIZED CONTROL; RANDOM VARIABLES; WEAK APPROXIMATION;

OPTIMAL CONTROL SYSTEMS;

EID: 27644539475     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012902419059     Document Type: Article

References (52)

1. [AT98] M. ARNAUDON AND A. THALMAIER, Stability of stochastic differential equations in manifolds, in Séminaire de Probabilités, XXXII, Springer-Verlag, Berlin, 1998, pp. 188-214.
2. [BB01] J. BAXTER AND P.L. BARTLETT, Infinite-horizon gradient-based policy search, J. Artificial Intelligence Res., 15 (2001), pp. 319-350.
3. [Ben88] A. BENSOUSSAN, Perturbation Methods in Optimal Control, Wiley/Gauthier-Villars Series in Modern Applied Mathematics, John Wiley & Sons, Chichester, UK, 1988.
4. [BG96] M. BROADIE AND P. GLASSERMAN, Estimating security price derivatives using simulation, Management Sci., 42 (1996), pp. 269-285.
5. [Bis84] J. M. BISMUT, Large Deviations and the Malliavin Calculus, Birkhäuser Boston, Boston, 1984.
6. [BMP90] A. BENVENISTE, M. METIVIER, AND P. PRIOURET, Adaptive Algorithms and Stochastic Approximations, Springer-Verlag, New York, 1990.
7. [BT96a] V. BALLY AND D. TALAY, The law of the Euler scheme for stochastic differential equations: I. Convergence rate of the distribution function, Probab. Theory Related Fields, 104 (1996), pp. 43-60.
8. [BT96b] D. P. BERTSEKAS AND J. TSITSIKLIS, Neuro-Dynamic Programming, Athena Scientific, Nashua, NH, 1996.
9. [CK99] J. CVITANIĆ AND I. KARATZAS, On dynamic measures of risk, Finance Stoch., 3 (1999), pp. 451-482.
10. [CM02] P. CATTIAUX AND L. MESNAGER, Hypoelliptic non-homogeneous diffusions, Probab. Theory Related Fields, 123 (2002), pp. 453-483.
11. [CY01] S. CHEN AND J. YONG, Stochastic linear quadratic optimal control problems, Appl. Math. Optim., 43 (2001), pp. 21-45.
12. [EJL99] K. D. ELWORTHY, Y. LE JAN, AND X.-M. LI, On the Geometry of Diffusion Operators and Stochastic Flows, Springer-Verlag, Berlin, 1999.
13. [FLL+99] E. FOURNIÉ, J. M. LASRY, J. LEBUCHOUX, P. L. LIONS, AND N. TOUZI, Applications of Malliavin calculus to Monte Carlo methods in finance, Finance and Stochastics, 3 (1999), pp. 391-412.
14. [Fri64] A. FRIEDMAN, Partial Differential Equations of Parabolic Type, Prentice-Hall, Englewood Cliffs, NJ, 1964.
15. [Gly86] P. W. GLYNN, Stochastic approximation for Monte Carlo optimization, in Proceedings of the 1986 Winter Simulation Conference, J. Wilson, J. Henriksen, and S. Roberts, eds., 1986, pp. 356-365.
16. [Gly87] P. W. GLYNN, Likelihood ratio gradient estimation: An overview, in Proceedings of the 1987 Winter Simulation Conference, A. Thesen, H. Grant, and W. D. Kelton, eds., 1987, pp. 366-375.
17. [GM03] E. GOBET AND R. MUNOS, Sensitivity Analysis using Itô-Malliavin Calculus and Martingales. Numerical Implementation, Technical report 520, CMAP, Ecole Polytechnique, Palaiseau, France, 2003.
18. [Gob00] E. GOBET, Euler schemes for the weak approximation of killed diffusion, Stochastic Process. Appl., 87 (2000), pp. 167-197.
19. [Gob01a] E. GOBET, Euler schemes and half-space approximation for the simulation of diffusions in a domain, ESAIM Probab. Statist., 5 (2001), pp. 261-297.
20. [Gob01b] E. GOBET, Local asymptotic mixed normality property for elliptic diffusion: A Malliavin calculus approach, Bernoulli, 7 (2001), pp. 899-912.
21. [Gob02] E. GOBET, LAN property for ergodic diffusion with discrete observations, Ann. Inst. H. Poincaré Probab. Statist., 38 (2002), pp. 711-737.
22. [GT77] D. GILBARG AND N. S. TRUDINGER, Elliptic Partial Differential Equations of Second Order, 1st ed., Springer-Verlag, New York, 1977.
23. [GY92] P. GLASSERMAN AND D. D. YAO, Some guidelines and guarantees for common random numbers, Management Sci., 38 (1992), pp. 884-908.
24. [Hay94] S. HAYKIN, Neural Networks: A Comprehensive Foundation, MacMillan, New York, 1994.
25. [JP98] J. JACOD AND P. PROTTER, Asymptotic error distributions for the Euler method for stochastic differential equations, Ann. Probab., 26 (1998), pp. 267-307.
26. [KB99] V. R. KONDA AND V. S. BORKAR, Actor-critic-type learning algorithms for Markov decision processes, SIAM J. Control Optim., 38 (1999), pp. 94-123.
27. [KD01] H. J. KUSHNER AND P. DUPUIS, Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd ed., Appl. Math. 24, Springer-Verlag, New York, 2001.
28. [KH01] A. KOHATSU-HIGA, Weak approximations. A Malliavin calculus approach, Math. Comp., 70 (2001), pp. 135-172.
29. [KHPOO] A. KOHATSU-HIGA AND R. PETTERSSON, On the simulation of some functionals of diffusion processes, Sūrikaisekikenkyūsho Kōkȳroku, 1127 (2000), pp. 153-170.
30. [KHP02] A. KOHATSU-HIGA AND R. PETTERSSON, Variance reduction methods for simulation of densities on Wiener space, SIAM J. Numer. Anal., 40 (2002), pp. 431-450.
31. [KP95] P. E. KLOEDEN AND E. PLATEN, Numerical Solution of Stochastic Differential Equations, Springer-Verlag, New York, 1995.
32. [KS84] S. KUSUOKA AND D. STROOCK, Applications of the Malliavin calculus I, in Stochastic Analysis, K. Itô, ed., Kinokuniya, Tokyo, 1984, pp. 271-306.
33. [KS86] P. KRÉE AND C. SOIZE, Mathematics of Random Phenomena: Random Vibrations of Mechanical Structures, Math. Appl. 32, D. Reidel, Dordrecht, 1986.
34. [KS98] I. KARATZAS AND S. E. SHREVE, Methods of Mathematical Finance, Springer-Verlag, New York, 1998.
35. [Kun84] H. KUNITA, Stochastic differential equations and stochastic flows of diffeomorphisms, in Ecole d'Eté de Probabilités de St-Flour XII, 1982, Lecture Notes in Math. 1097, Springer-Verlag, New York, 1984, pp. 144-305.
36. [KY97] H. J. KUSHNER AND G. YIN, Stochastic Approximation Algorithms and Applications, Springer-Verlag, Berlin, New York, 1997.
37. [LP94] P. L'ECUYER AND G. PERRON, On the convergence rates of IPA and FDC derivative estimators, Oper. Res., 42 (1994), pp. 643-656.
38. [Nua95] D. NUALART, Malliavin Calculus and Related Topics, Springer-Verlag, New York, 1995.
39. [Nua98] D. NUALART, Analysis on Wiener space and anticipating stochastic calculus, in Lectures on Probability Theory and Statistics (Saint-Flour, 1995), Springer-Verlag, Berlin, 1998, pp. 123-227.
40. [Pen90] S. G. PENG, A general stochastic maximum principle for optimal control problems, SIAM J. Control Optim., 28 (1990), pp. 966-979.
41. [Pic02] J. PICARD, Gradient estimates for some diffusion semigroups, Probab. Theory Related Fields, 122 (2002), pp. 593-612.
42. [Pol87] B. T. POLYAK, Introduction to Optimization, Optimization Software Inc., New York, 1987.
43. [Pro90] P. PROTTER, Stochastic Integration and Differential Equations, Springer-Verlag, New York, 1990.
44. [RM86] D. E. RUMELHART AND J. L. MCCLELLAND, Parallel Distributed Processing, Vols. I and II, MIT Press, Cambridge, MA, 1986.
45. [RT99] C. R. RAO AND H. TOUTENBURG, Linear Models, Springer Series in Statistics, 2nd ed., Springer-Verlag, New York, 1999.
46. [Run02] W. J. RUNGGALDIER, On stochastic control in finance, in Mathematical Systems Theory in Biology, Communication, Computation and Finance (MTNS-2002), Springer-Verlag, New York, 2002.
47. [RW86] M. I. REIMAN AND A. WEISS, Sensitivity analysis via likelihood ratios, in Proceedings of the 1986 Winter Simulation Conference, J. Wilson, J. Henriksen, and S. Roberts, eds., 1986, pp. 285-289.
48. [Tha97] A. THALMAIER, On the differentiation of heat semigroups and Poisson integrals, Stochastics Stochastics Rep., 61 (1997), pp. 297-321.
49. [TL90] D. TALAY AND L. TUBARO, Expansion of the global error for numerical schemes solving stochastic differential equations, Stochastic Anal. Appl., 8 (1990), pp. 94-120.
50. [TZ04] D. TALAY AND Z. ZHENG, Approximation of quantiles of components of diffusion processes, Stochastic Process. Appl., 109 (2004), pp. 23-46.
51. [YK91] J. YANG AND H. J. KUSHNER, A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems, SIAM J. Control Optim., 29 (1991), pp.1216-1249.
52. [YZ99] J. YONG AND X. Y. ZHOU, Stochastic Controls: Hamiltonian Systems and HJB Equations, Appl. Math. 43, Springer-Verlag, New York, 1999.