Existence of markov controls and characterization of optimal markov controls

SIAM Journal on Control and Optimization

Volumn 36, Issue 2, 1998, Pages 609-653

  Kurtz, Thomas G ^a   Stockbridge, Richard H ^b  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

^b UNIVERSITY OF KENTUCKY   (United States)

Author keywords

Linear programming; Markov controls; Martingale problems; Occupation measures; Optimal controls; Stationary processes

Indexed keywords

COMPUTATIONAL METHODS; CONTROL SYSTEM SYNTHESIS; LINEAR PROGRAMMING; MARKOV PROCESSES; MATHEMATICAL MODELS; OPTIMAL CONTROL SYSTEMS; PROBLEM SOLVING; STOCHASTIC CONTROL SYSTEMS; THEOREM PROVING;

ECHEVERRIA'S THEOREMS; MARTINGALE PROBLEMS; OCCUPATIONAL MEASURES; STATIONARY PROCESSES;

CONTROL SYSTEM ANALYSIS;

EID: 0032024929     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012995295516     Document Type: Article

References (20)

1. A. G. BHATT AND R. L. KARANDIKAR, Invariant measures and evolution equations for Markov processes, Ann. Probab., 21 (1993), pp. 2246-2268.
2. A. G. BHATT AND V. S. BORKAR, Occupation measures for controlled Markov processes: Characterization and optimality, Ann. Probab., 24 (1996), pp. 1531-1562.
3. V. S. BORKAR, A remark on the attainable distributions of controlled diffusions, Stochastics, 18 (1986), pp. 17-23.
4. V. S. BORKAR AND M. K. GHOSH, Ergodic control of multidimensional diffusions I: The existence results, SIAM J. Control Optim., 26 (1988), pp. 112-126.
5. C. DERMAN, On sequential decisions and Markov chains, Management Sci., 9 (1962), pp. 16-24.
6. E. V. DENARDO, On linear programming in a Markov decision problem, Management Sci., 16 (1970), pp. 281-288.
7. A. DVORETSKY, Asymptotic normality for sums of dependent random variables, in Proc. 6th Berkeley Symposium on Mathematical Statistics and Probability, L. M. Le Cam, J. Neyman and E. L. Scott, eds., University of California Press, Berkeley, CA, 1972, pp. 513-535.
8. N. EL KAROUI, D. NGUYEN, AND M. JEANBLANC-PICQUE, Compactification methods in the control of degenerate diffusions: Existence of an optimal control, Stochastics, 20 (1987), pp. 169-219.
9. S. N. ETHIER AND T. G. KURTZ, Markov Processes: Characterization and Convergence, Wiley, New York, 1986.
10. W. H. FLEMING AND D. VERMES, Convex duality approach to the optimal control of diffusions, SIAM J. Control Optim., 27 (1989), pp. 1136-1155.
11. I. GYÖNGY, Mimicking the one-dimensional marginal distributions of processes having an Ito differential, Probab. Theory Related Fields, 71 (1986), pp. 501-516.
12. U. G. HAUSSMANN AND J. P. LEPELTIER, On the existence of optimal controls, SIAM J. Control Optim., 28 (1990), pp. 851-902.
13. N. V. KRYLOV, Once more about the connection between elliptic operators and Itô's stochastic equations, in Statistics and Control of Stochastic Processes, Steklov Seminar, 1984, Optimization Software Inc., New York, 1985, pp. 214-229.
14. T. G. KURTZ, Averaging for martingale problems and stochastic approximation, in Proc. Joint U.S.-French Workshop on Applied Stochastic Analysis, Lecture Notes in Control and Inform. Sci. 177, Springer-Verlag, New York, 1992, pp. 186-209.
15. A. S. MANNE, Linear programming and sequential decisions, Management Sci., 6 (1960), pp. 259-267.
16. B. G. PITTEL, A linear programming problem connected with optimal stationary control in a dynamic decision problem, Theory Probab. Appl., 16 (1971), pp. 724-728.
17. H. L. ROYDEN, Real Analysis, 2nd ed., Macmillan, New York, 1968.
18. R. H. STOCKBRIDGE, Time-average control of martingale problems: Existence of a stationary solution, Ann. Probab., 18 (1990), pp. 190-205.
19. R. H. STOCKBRIDGE, Time-average control of martingale problems: A linear programming formulation, Ann. Probab., 18 (1990), pp. 206-217.
20. P. WOLFE AND G. B. DANTZIG, Linear programming in a Markov chain, Oper. Res., 10 (1962), pp. 702-710.