Occupation measures for controlled Markov processes: Characterization and optimality

Annals of Probability

Volumn 24, Issue 3, 1996, Pages 1531-1562

  Bhatt, Abhay G ^a   Borkar, Vivek S ^a  

^a NONE

Author keywords

Controlled Markov processes; Infinite dimensional linear programming; Occupation measures; Optimal control

Indexed keywords

EID: 0030366646     PISSN: 00911798     EISSN: None     Source Type: Journal    
DOI: 10.1214/aop/1065725192     Document Type: Article

References (23)

1. ANDERSON, E. J. and NASH, P. (1987). Linear Programming in Infinite Dimensional Spaces. Wiley, Chichester.
2. BHATT, A. G. and KARANDIKAR, R. L. (1993a). Invariant measures and evolution equations for Markov processes characterized via martingale problems. Ann. Probab. 21 2246-2268.
3. BHATT, A. G. and KARANDIKAR, R. L. (1993b). Weak convergence to a Markov process: the martingale approach. Probab. Theory Related Fields 96 335-351.
4. BORKAR, V. S. (1989). Optimal Control of Diffusion Processes. Longman, Harlow, England.
5. BORKAR, V. S. (1991). On extremal solutions to stochastic control problems. J. Appl. Math. Optim. 24 317-330.
6. BORKAR, V. S. (1993). On extremal solutions to stochastic control problems. II. J. Appl. Math. Optim. 28 49-56.
7. BORKAR, V. S. (1995). A note on ergodic control of degenerate diffusions. J. Optim Theory Appl. 86 251-261.
8. BORKAR, V. S. and GOVINDAN, T. E. (1994). Optimal control of semilinear stochastic evolution equations. Nonlinear Anal. 23 15-35.
9. BORKAR, V. S. and SUNILKUMAR, P. C. (1995). On extremal solutions of controlled nonlinear filtering equations. SIAM J. Control Optim. 33 718-724.
10. CHOQUET, G. (1969). Lectures on Analysis II: Representation Theory. W. A. Benjamin, Reading, MA.
11. ECHEVERRIA, P. E. (1982). A criterion for invariant measures of Markov processes. Z. Wahrsch. Verw. Gebiete 61 1-16.
12. EL KAROUI, N., NGUYEN, D. H. and JEANBLANC-PIQUE, M. (1987). Compactification methods in the control of degenerate diffusions: existence of an optimal control. Stochastics 20 169-219.
13. EL KAROUI, N., NGUYEN, D. H. and JEANBLANC-PIQUE, M. (1988). Existence of an optimal Markovian filter for the control under partial observations. SIAM J. Control Optim. 26 1025-1061.
14. ETHIER, S. N. and KURTZ, T. G. (1986). Markov Processes: Characterization and Convergence. Wiley, New York.
15. FLEMING, W. R. and VERMES. D. (1989) Convex duality approach to the optimal control of diffusions. SIAM J. Control Optim. 27 1136-1155.
16. HERNANDEZ-LERMA, O., HENNET, J. C. and LASSERRE, J. B. (1991). Average cost Markov decision processes: optimality conditions. J. Math. Anal. Appl. 158 396-406.
17. KURTZ, T. G. and STOCKBRIDGE, R. H. (1994). Existence of Markov controls and characterization of optimal Markov controls. Preprint.
18. MANNE, A. (1960). Linear programming and sequential decisions. Management Sci. 6 257-267.
19. STOCKBRIDGE, R. H. (1990a). Time-average control of a martingale problem: existence of a stationary solution. Ann. Probab. 18 190-205.
20. STOCKBRIDGE, R. H. (1990b). Time-average control of a martingale problem: a linear programming formulation. Ann. Probab. 18 206-217.
21. STROOCK, D. W. and VARADHAN, S. R. S. (1979). Multidimensional Diffusion Processes. Springer, Berlin.
22. VINTER, R. B. and LEWIS, R. M. (1978). A necessary and sufficient condition for optimality of dynamic programming type, making no a priori assumptions on the controls. SIAM J. Control Optim. 16 571-583.
23. YOR, M. (1974). Existence et unicité de diffusions à valeurs dans un espace de Hilbert. Ann. Inst. H. Poincaré Probab. Statist. 10 55-88.