Numerical comparison of controls and verification of optimality for stochastic control problems

Journal of Optimization Theory and Applications

Volumn 100, Issue 2, 2000, Pages 107-127

  Helmes, K ^a   Stockbridge, R H ^b  

^a HUMBOLDT UNIVERSITY   (Germany)

^b UNIVERSITY OF KENTUCKY   (United States)

Author keywords

Bounded follower; Linear programming; Moments; Numerical comparisons; Numerical verification; Stochastic control

Indexed keywords

STOCHASTIC CONTROL SYSTEMS; STOCHASTIC SYSTEMS;

BOUNDED FOLLOWER; CONTROL PROBLEMS; LINEAR PROGRAMMING FORMULATION; LINEAR-PROGRAMMING; MOMENT; NUMERICAL COMPARISON; NUMERICAL VERIFICATION; OCCUPATION MEASURE; OPTIMALITY; STOCHASTIC CONTROL;

LINEAR PROGRAMMING;

EID: 0033163922     PISSN: 00223239     EISSN: None     Source Type: Journal    
DOI: 10.1023/a:1004659107996     Document Type: Article

References (18)

1. 1. MANNE, A. S., Linear Programming and Sequential Decisions, Management Science, Vol. 6, pp. 259-267, 1960.
2. 2. DERMAN, C., On Sequential Decisions and Markov Chains, Management Science, Vol. 9, pp. 16-24, 1962.
3. 3. DENARDO, E. V., On Linear Programming in a Markov Decision Problem, Management Science, Vol. 16, pp. 281-288, 1970.
4. 4. WOLFE, P., and DANTZIG, G. B., Linear Programming in a Markov Chain, Operations Research, Vol. 10, pp. 702-710, 1962.
5. 5. PITTEL, B. G., A Linear Programming Problem Connected with Optimal Stationary Control in a Dynamic Decision Problem, Theory of Probability and Its Applications, Vol. 16, pp. 724-728, 1971.
6. 6. HERNANDEZ-LERMA, O., HENNET, J. C., and LASSERRE, J. B., Average Cost Markov Decision Processes: Optimally Conditions, Journal of Mathematical Analysis and Applications, Vol. 158, pp. 396-406, 1991.
7. 7. STOCKBRIDGE, R. H., Time-Average Control of Martingale Problems: Existence of a Stationary Solution, Annals of Probability, Vol. 18, pp. 190-205, 1990.
8. 8. STOCKBRIDGE, R. H., Time-Average Control of Martingale Problems: A Linear Programming Formulation, Annals of Probability, Vol. 18, pp. 206-217, 1990.
9. 9. KURTZ, T. G., and STOCKBRIDGE, R. H., Existence of Markov Controls and Characterization of Optimal Markov Controls, SIAM Journal of Control and Optimization, Vol. 36, pp. 609-653, 1998.
10. 10. BHATT, A. G., and BORKAR, V. S., Occupation Measures for Controlled Markov Processes: Characterization and Optimality, Annals of Probability, Vol. 25, pp. 1531-1562, 1996.
11. 11. HELMES, K., ROEHL, S., and STOCKBRIDGE, R. H., Computing Moments of the Exit Time Distribution for Markov Processes by Linear Programming (to appear in Operations Research).
12. 12. MENDIONDO, M. S., and STOCKBRIDGE, R. H., Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control, SIAM Journal of Control and Optimization, Vol. 36, pp. 1448-1472, 1998.
13. 13. ETHIER, S. N., and KURTZ, T. G., Markov Processes: Characterization and Convergence, John Wiley and Sons, New York, NY, 1986.
14. 14. BENEŠ, V. E., SHEPP, L. A., and WITSENHAUSEN, H. S., Some Solvable Stochastic Control Problems, Stochastics, Vol. 4, pp. 39-83, 1980.
15. 15. BILLINGSLEY, P., Probability and Measure, 2nd Edition, Wiley, New York, NY, 1985.
16. 16. FELLER, W., An Introduction to Probability Theory and Its Applications, Vol. 2, Wiley, New York, NY, 1965.
17. 17. FLEMING, W. H., and RISHEL, R. W., Deterministic and Stochastic Optimal Control, Springer Verlag, New York, NY, 1975.
18. 18. FLEMING, W. H., and SONER, H. M., Controlled Markov Processes and Viscosity Solutions, Springer Verlag, New York, NY, 1993.