Controlled diffusion processes

Probability Surveys

Volumn 2, Issue 1, 2005, Pages 213-244

  Borkar, Vivek S ^a  

^a TATA INSTITUTE OF FUNDAMENTAL RESEARCH   (India)

Author keywords

Controlled diffusions; Dynamic programming; Hamilton jacobi bellman equations; Optimal control; Partial observations

Indexed keywords

EID: 51549090039     PISSN: None     EISSN: 15495787     Source Type: Journal    
DOI: 10.1214/154957805100000131     Document Type: Article

References (119)

1. Allinger, D. F., & Mitter, S. K., New results on the innovations problem of nonlinear filtering, Stochastics 4 (1981), pp. 339-348.
2. Alvarez, L. H. R., Singular stochastic control, linear diffusions, and optimal stopping: a class of solvable problems, SIAM J. Control and Optim. 39 (2001), pp. 1697-1710. MR1825860
3. Alvarez, L. H. R., Stochastic forest stand value and optimal timber harvesting, SIAM J. Control and Optim. 42 (2004), pp. 1972-1993. MR2080924
4. Barles, G., & Souganidis, P., Convergence of approximation schemes for fully nonlinear second order equations, J. Asymptotic Analysis 4 (1991), pp. 271-283. MR1115933
5. Beneš, V., Existence of optimal strategies based on specific information, for a class of stochastic decision problems, SIAM J. Control and Optim. 8 (1970), pp. 179-188. MR265043
6. Beneš, V., Existence of optimal stochastic controls, SIAM J. Control and Optim. 9 (1971), pp. 446-472.
7. Bensoussan, A., Stochastic Control by Functional Analysis Methods, North Holland, Amsterdam, 1982.
8. Bensoussan, A., Maximum principle and dynamic programming approaches to the optimal control of partially observed diffusions, Stochastics 9 (1983), pp. 169-222. MR705471
9. Bensoussan, A., & Frehse, J., On Bellman equations of ergodic control in Rn, J. Reine Angew. Math. 429 (1992), pp. 125-160. MR1173120
10. Bensoussan, A., & Frehse, J., Stochastic games for N players, J. Optim. Theory and Appl. 105 (2000), pp. 543-565. MR1783877
11. Bensoussan, A., & Lions, J. L., Impulse Control and Quasi-variational Inequalities, Gauthier-villars, Paris, 1980.
12. Bensoussan, A., & Lions, J. L., Applications of Variational Inequalities in Stochastic Control, Noth Holland, Amsterdam, 1982.
13. Bensoussan, A.; Sethi, S. P.; Vickson, R, & Derzko, N., Stochastic production planning with production constraints, SIAM J. Control and Optim. 22 (1984), pp. 920-935. MR762630
14. Bertsekas, D. P., & Tsitsiklis, J. N., Neuro-dynamic Programming, Athena Scientific, Belmont, Mass., 1996.
15. Bhatt, A. G., & Borkar, V. S., Occupation measures for controlled Markov processes: characterization and optimality, Annals of Probability 24 (1996), pp. 1531-1562. MR1411505
16. Bhatt, A. G., & Borkar, V. S., Existence of optimal Markov solutions for ergodic control of Markov processes, Sankhya 67 (2005), pp. 1-18.
17. Bielecki, T., & Pliska, S., Risk sensitive asset management with transaction costs, Finance and Stochastics 4 (2000), pp. 1-33. MR1790132
18. Bismut, J.-M., Théorie probabiliste du contrôle des diffusions, Mem. Amer. Math. Soc. 4 (1976), no. 167.
19. Borkar, V. S., A remark on the attainable distributions of controlled diffusions, Stochastics 18 (1986), pp. 17-23. MR854653
20. Borkar, V. S., Control of a partially observed diffusion up to an exit time, Systems & Control Letters 8 (1987), pp. 429-434. MR890079
21. Borkar, V. S., Optimal Control of Diffusion Processes, Pitman Research Notes in Math. 203, Longman Scientific and Technical, Harlow, UK, 1989.
22. Borkar, V. S., Self-tuning control of diffusions without the identifiability condition, J. Optim. Theory and Appl. 68 (1991), pp. 117-138. MR1089344
23. Borkar, V. S., On extremal solutions to stochastic control problems, Applied Maths. and Optim. 24 (1991), pp. 317-330.
24. Borkar, V. S., On extremal solutions to stochastic control problems II, Applied Maths. and Optim. 28 (1993), pp. 49-56.
25. Borkar, V. S., & Ghosh, M. K., Ergodic control of multidimensional diffusions I: the existence results, SIAM J. Control and Optim. 26 (1988), pp. 112-126. MR923306
26. Borkar, V. S., & Ghosh, M. K., Ergodic control of multidimensional diffusions II: adaptive control, Applied Math. and Optim. 21 (1990), pp. 191- 220. MR1019400
27. Borkar, V. S., & Ghosh, M. K., Controlled diffusions with constraints, J. Math. Anal. and Appl. 152 (1990), pp. 88-108. MR1072929
28. Borkar, V. S., & Ghosh, M. K., Stochastic differential games: an occupation measure based approach, J. Optim. Theory and Appl. 73 (1992), pp. 359- 385, Erratum in ibid. 88 (1996), pp. 251-252.
29. Borkar, V. S., Ghosh, M. K., & Sahay, P., Optimal control of a stochastic hybrid system with discounted cost, J. Optim. Theory and Appl. 101 (1999), pp. 557-580. MR1692204
30. Budhiraja, A., An ergodic control problem for constrained diffusion processes: existence of optimal Markov control, SIAM J. Control and Optim. 42 (2003), pp. 532-558. MR1982282
31. Caines, P. E., Linear Stochastic Systems, John Wiley, New York, 1988.
32. Charalambous, C. D., & Hibey, J. L., Necessary conditions of optimization for partially observed controlled diffusions, SIAM J. Control and Optim. 37 (1999), pp. 1676-1700. MR1720132
33. Cho, M. J., & Stockbridge, R. H., Linear programming formulation for optimal stopping problems, SIAM J. Control and Optim. 40 (2002), pp. 1965-1982. MR1897205
34. Davis, M. H. A. & Varaiya, P. P., Dynamic programming conditions for partially observable stochastic systems, SIAM J. Control and Optim. 11 (1973), pp. 226-261. MR319642
35. Dufour, F., & Miller, B, Singular stochastic control problems, SIAM J. Control and Optim. 43 (2004), pp. 708-730. MR2086181
36. Duncan, T. E., Pasik-Duncan, B., & Stettner, L., Almost self-optimizing strategies for adaptive control of diffusion processes, J. Optim. Theory and Appl. 81 (1994), pp. 479-507. MR1281735
37. Duncan, T. E., & Varaiya, P. P., On the solutions of a stochastic control system, SIAM J. Control and Optim. 9 (1971), pp. 354-371. MR304044
38. Dupuis, P., James, M. R., & Petersen, I., Robust properties of risk-sensitive control, Proc. of IEEE Conf. on Decision and Control, Tampa, Florida (1998), pp. 2365-2370.
39. Echeverria, P., A criterion for invariant measures of Markov processes, Z. Wahrsch. Verw. Gebiete 61 (1982), pp. 1-16. MR671239
40. El Karoui, N., Huu Nguyen, D., & Jeanblanc-Pique, M., Compactification methods in in the control of degenerate diffusions: existence of an optimal control, Stochastics 20 (1987), pp. 169-219. MR878312
41. El Karoui, N., Huu Nguyen, D., & Jeanblanc-Pique, M., Existence of an optimal Markovian filter for control under partial observations, SIAM J. Control and Optim. 26 (1988), pp. 1025-1061. MR957652
42. El Karoui, N., & Mazliak, L., The Theory of Backward Stochastic Differential Equations, Pitman Research Notes in Math. no. 364, Addison Wesley Longman, Harlow, UK, 1997. MR1752671
43. Fabes, E. B., & Kenig, C. E., Examples of singular probability measures and singular transition probability densities, Duke Math. Journal 48 (1981), pp. 845-856. MR782580
44. Fleming, W. H., Generalized solutions in optimal stochastic control, in 'Differential Games and Control Theory III' (E. Roxin, P. T. Liu, R. Sternberg, eds.), Marcel Dekker, New York, 1977, pp. 147-165. MR524401
45. Fleming, W. H., Nonlinear semigroup for controlled partially observed diffusions, SIAM J. Control and Optim. 20 (1982), pp. 286-301. MR646954
46. Fleming, W. H., & McEneaney, W. M., Risk-sensitive control on an infinite horizon, SIAM J. Control and Optim. 33 (1995), pp. 1881-1915. MR1358100
47. Fleming, W. H., & Nisio, M., On the existence of optimal stochastic controls, J. Math. Mech. 15 (1966), pp. 777-794. MR199020
48. Fleming, W. H., & Pardoux, E., Existence of optimal controls for partially observed diffusions, SIAM J. Control and Optim. 20 (1982), pp. 261-283.
49. Fleming, W. H., & Sheu, S. J., Risk-sensitive Control and an optimal investment model, Math. Finance 10 (2000), pp. 197-214. MR1802598
50. Fleming, W. H., & Soner, H. M., Controlled Markov Processes and Viscosity Solutions, Springer Verlag, New York, 1992.
51. Fukushima, M., & Taksar, M., Dynkin games via Dirichlet forms and singular control of one-dimensional diffusions, SIAM J. Control and Optim. 41 (2002), pp. 682-699. MR1939866
52. Gihman, I. I., & Skorohod, A. V., Controlled Stochastic Processes, Springer Verlag, New York, 1979.
53. Gilbarg, D., & Trudinger, N. S., Elliptic Partial Differential Equations of Second Order (2nd ed.), Springer Verlag, Berlin-Heidelberg, 2001.
54. Ghosh, M. K., Arapostathis, A., & Marcus, S. I., Optimal control of switching diffusions with applications to exible manufacturing systems, SIAM J. Control and Optim. 31 (1993), pp. 1183-1204. MR1233999
55. Ghosh, M. K., & Rao, K. S. M., A probabilistic approach to second order variational inequalities with bilateral constraints, Proc. Indian Academy of Sciences (Math. Sciences) 113 (2003), pp. 431-442. MR2020075
56. Glasserman, P., Monte Carlo Methods in Financial Engineering, Springer Verlag, New York, 2004.
57. Gozzi, F., & Swiech, A., Hamilton-Jacobi-Bellman equations for the optimal control of the Duncan-Mortensen-Zakai equation, J. Funct. Anal. 172 (2000), pp. 466-510. MR1753181
58. Guo, X., & Pham, H., Optimal partially reversible investment with entry decision and general production function, Stochastic Processes and Their Applications 115 (2005), pp. 705-736. MR2132595
59. Gÿongi, I., Mimicking the one-dimensional marginals of processes having an Ito differential, Prob. Theory and Related Fields 71 (1986), pp. 501-516. MR833267
60. Harrison, J. M., Brownian Motion and Stochastic Flow Systems, John Wiley, New York, 1985.
61. Harrison, J. M., & Van Miegham, J. A., Dynamic control of Brownian networks: state space collapse and equivalent workload formulation, Ann. Appl. Prob. 7 (1997), pp. 747-771. MR1459269
62. Harrison, J. M., & Zeevi, A., Dynamic scheduling of a multiclass queue in the Halfin-Whitt heavy traffic regime, Op. Research 52 (2004), pp. 243-257. MR2066399
63. Haussmann, U. G., Existence of optimal Markov controls for degenerate diffusions, in 'Stochastic Differential Systems', Lecture Notes in Control and Info. Sciences 78 (N. Christopeit, K. Helmes, M. Kohlmann, eds.), Springer Verlag, Berlin-Heidelberg, 1986, pp. 171-186.
64. Haussmann, U. G., A Stochastic Maximum Principle for Optimal Control of Diffusions, Pitman Research Notes in Math. 151, Longman Scientific andTechnical, Harlow, UK, 1986.
65. Haussmann, U. G., L'Equation de Zakai et le Problème Séparé du Contrôle Optimal Stochastique, in 'Seminaire de Probabilites XIX', Springer Lecture Notes in Maths. No. 1123, Springer Verlag, Berlin-Heidelberg, 1985, pp. 37- 62.
66. Helmes, K., & Stockbridge, R. H., Numerical comparison of controls and verification of optimality for stochastic control problems, J. Optim. Theory and Appl. 106 (2000), pp. 107-127. MR1780110
67. Heunis, A. J., On the prevalence of stochastic differential equations with unique strong solutions, Annals of Probability 14 (1986), pp. 653-662. MR832028
68. Kabanov, Yu., & Kluppelberg, C., A geometric approach to portfolio optimization in models with transaction costs, Finance and Stochastics 8 (2004), pp. 207-227. MR2048828
69. Kabanov, Yu., & Pergamenshchikov, S., Two-Scale Stochastic Systems, Springer Verlag, Berlin-Heidelberg, 2002.
70. Karatzas, I., Lectures on the Mathematics of Finance, CRM Monograph Series Vol. 8, American Math. Society, Providence, R.I., 1997.
71. Karatzas, I., & Shreve, S. E., Methods of Mathematical Finance, Springer Verlag, New York, 1998.
72. Kloeden, P. E., & Platen, E., Numerical Solution of Stochastic Differential Equations, Springer Verlag, Berlin, 1992. MR1217084
73. Korn, R., & Kraft, H., A stochastic control approach to portfolio problems with stochastic interest rates, SIAM J. Control and Optim. 40 (2001/02), pp. 1250-1269. MR1882732
74. Kruk, L., Optimal policies for n-dimensional singular stochastic control problems I. The Skorokhod problem, SIAM J. Control and Optim. 38 (2000), pp. 1603-1622. MR1766432
75. Kruk, L., Optimal policies for n-dimensional singular stochastic control problems. II. The radially symmetric case. Ergodic control, SIAM J. Control Optim. 39 (2000), pp. 635-659. MR1788074
76. Krylov, N. V., Controlled Diffusion Processes, Springer Verlag, Berlin- Heidelberg, 1980.
77. Krylov, N. V., On weak uniqueness for some diffusions with discontinuous coefficients, Stochastic Process and Their Applications 113 (2004), pp. 37- 64. MR2078536
78. Kuroda, K., & Nagai, H., Risk-sensitive portfolio optimization on infinite time horizon, Stochastics and Stochastics Report 73 (2002), pp. 309-331. MR1932164
79. Kurtz, T. G., & Stockbridge, R., Existence of Markov controls and characterization of Markov controls, SIAM J. Control and Optim. 36 (1998), pp. 609-653. Erratum in ibid., 37 (1999), pp. 1310-1311.
80. Kushner, H. J., On the existence of optimal stochastic controls, SIAM J. Control 3 (1965), pp. 463-474.
81. Kushner, H. J., Existence results for optimal stochastic controls, J. Optim. Theory and Appl. 15 (1975), pp. 347-359.
82. Kushner, H. J., Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, Birkhäuser, Boston, 1990.
83. Kushner, H. J., Numerical approximations for stochastic differential games: the ergodic case, SIAM J. Control and Optim. 42 (2002), pp. 1911-1933.
84. Kushner, H. J., & Dupuis, P., Numerical Methods for Stochastic Control Problems in Continuous Time (2nd edition), Springer Verlag, New York, 2001.
85. Kushner, H. J., & Schweppe, F. C., A maximum principle for stochastic control systems, J. Math. Anal. Appl. 8 (1964), pp. 287-302. MR158795
86. Ladyženskaja, O. A., Solonnikov, V. A., & Ural'seva, N. N., Linear and Quasilinear Equations of Parabolic Type, Translations of Math. Monographs Vol. 23, A.M.S., Providence, R.I., 1968.
87. Lapeyre, B., Pardoux, E., & Sentis, R., Introduction to Monte-Carlo Methods for Transport and Diffusion Equations, Oxford Uni. Press, Oxford, UK, 2003.
88. Lenhart, S., Integro-differential operators associated with diffusion processes with jumps, Appl. Math. Optim. 9 (1982), pp. 177-191. Erratum in ibid. 13 (1985), pp. 283. MR680451
89. Lions, P. L., Optimal control of diffusion processes and the Hamilton- Jacobi-Bellman equations I: The dynamic programming principle and applications, Comm. PDEs 8 (1983), pp. 1101-1134.
90. Lions, P. L., Optimal control of diffusion processes and the Hamilton- Jacobi-Bellman equations II: Viscosity solutions and uniquenes, Comm. PDEs 8 (1983), pp. 1229-1276. MR709162
91. Lions, P. L., Optimal control of diffusion processes and the Hamilton- Jacobi-Bellman equations III: Regularity of the optimal cost function, in 'Nonlinear PDEs and Appl., College de France Seminar V', H. Brezis and J. L. Lions (eds.), Pitman Research Notes in Math. 93, Longman Scientific and Technical, Harlow, UK, 1983, pp. 1-18.
92. Lions, P.-L., & Perthame, B., An impulsive control problem with state constraints, Ann. Inst. H. Poincare: Analyse Nonlineaire 6 Suppl. (1989), pp. 385-414.
93. Liptser, R. H., & Shiryayev, A. N., Statistics of Random Processes I: General Theory (2nd ed.), Springer Verlag, New York, 2000.
94. Ma, J., & Yong, J., Dynamic programming for multidimensional stochastic control problems, Acta Academia Sinica 15 (1999), pp. 485-506. MR1760983
95. Ma, J., & Yong, J., Forward-Backward Stochastic Differential Equations and Their Applications, Springer Lecture Notes in Maths. no. 1702, Springer Verlag, Berlin-Heidelberg, 1999.
96. Menaldi, J.-L., Perthame, B., & Robin, M., Ergodic problem for optimal stochastic switching, J. Math. Anal. Appl. 147 (1990), no. 2, pp. 512-530. MR1050224
97. Mendiondo, M. S., & Stockbridge, R. H., Approximation of infinitedimensional linear programming problems which arise in stochastic control, SIAM J. Control and Optim. 36 (1998), pp. 1448-1472. MR1627589
98. Mikami, T., Markov marginal problems and their applications to Markov optimal control, in 'Stochastic Analysis, Control, Optimization and Applications', W. M. McEneaney, G. George Yin, Q. Zhang (eds.), Birkhäuser, Boston, 1999, pp. 457-476. MR1702949
99. Nadirashvili, N., Nonuniqueness in the martingale problem and the Dirichlet problem for uniformly elliptic operators, Ann. Scoula Norm. Sup. Pisa Cl. Sc. (4) 24 (1997), pp. 537-549. MR1612401
100. Nagai, H., Bellman equations of risk-sensitive control, SIAM J. Control and Optim. 34 (1996), pp. 74-101. MR1372906
101. Nisio, M., On a nonlinear semigroup attached to stochastic control theory, Publ. RIMS, Kyoto Uni., 1976.
102. Parthasarathy, T., Selection Theorems and Their Applications, Lecture Notes in Math. 263, Springer Verlag, Berlin, 1972.
103. Pham, H., On the smooth-fit property of one-dimensional optimal switching problem, eprint arXiv:math/0410285 (2004).
104. Rishel, R., Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control, SIAM J. Control and Optim. 8 (1970), pp. 559-571. MR274161
105. Runggaldier, W. J. (ed.), Financial Mathematics : Lectures given at the 3rd Session of the Centro Internazionale Matematico Estivo (C.I.M.E.) held in Bressanone, Italy, July 8-13, 1996, Springer Lecture Notes in Mathematics No. 1656, Springer Verlag, Berlin-Heidelberg, 1997.
106. Shiryayev, A. N., Optimal Stopping Rules, Springer Verlag, New York, 1978.
107. Shreve, S., Introduction to singular control, in 'Stochastic Differential Systems, Stochastic Control Theory and Applications', W. H. Fleming and P. L. Lions (eds.), IMA Volumes in Math. and its Appl. 10, Springer Verlag, New York, 1988, pp. 513-528. MR934712
108. Stettner, L., On impulse control with long run average cost criterion, Studia Math. 76 (1986), pp. 279-298.
109. Stockbridge, R.H., Time-average control of a martingale problem: existence of a stationary solution, Ann. Prob. 18 (1990), pp. 190-205. MR1043943
110. Striebel, C., Martingale conditions for the optimal control of continuous time stochastic systems, Stoch. Proc. and Appl. 18 (1984), pp. 329-347. MR770198
111. Stroock, D. W., & Varadhan, S. R. S., Multidimensional Diffusion Processes, Springer Verlag, New York, 1979. MR532498
112. Tang, S., The maximum principle for partially observed optimal control of stochastic differential equations, SIAM J. Control and Optim. 36 (1998), pp. 1596-1617. MR1626880
113. Tang, S., & Yong, J., Finite horizon stochastic optimal switching and impulse controls with a viscosity solution approach, Stochastics and Stochastics Reports 45 (1993), pp. 145-176. MR1306930
114. Varaiya, P. P., N-player stochastic differential games, SIAM J. Control and Optim. 14 (1976), pp. 538-545.
115. Veretennikov, A. Ju., On strong solutions and explicit formulas for solutions of stochastic differential equations, Math. USSR Sbornik 39 (1981), pp. 387-401.
116. Yong, J., & Zhou, X. Y., Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer Verlag, New York, 1999.
117. Young, L. C., Lectures on the Calculus of Variations and Optimal Control Theory, W. B. Saunders, Philadelphia, 1969.
118. Zhou, X. Y., The connection between maximum principle and dynamic programming in stochastic control, Stochastics and Stoch. Reports 31 (1990), pp. 1-13. MR1080531
119. Zhou, X. Y., & Li, D., Continuous-time mean-variance portfolio selection: a stochastic LQ framework, Appl. Math. Optim. 42 (2000), pp. 19-33. MR1751306