Sample-path optimality and variance-minimization of average cost Markov control processes

SIAM Journal on Control and Optimization

Volumn 38, Issue 1, 1999, Pages 79-93

  Hernández Lerma, Onésimo ^a   Vega Amaya, Oscar ^b   Carrasco, Guadalupe ^c  

^a DEPARTAMENTO DE FÍSICA   (Mexico)

^b UNIVERSIDAD DE SONORA   (Mexico)

^c UNIVERSIDAD NACIONAL AUTÓNOMA DE MÉXICO   (Mexico)

Author keywords

[No Author keywords available]

Indexed keywords

MARKOV PROCESSES; OPTIMAL CONTROL SYSTEMS; PROCESS CONTROL;

AVERAGE COST CRITERIA; CANONICAL POLICY; DISCRETE TIME MARKOV CONTROL PROCESSES; SAMPLE-PATH AVERAGE COSTS;

DISCRETE TIME CONTROL SYSTEMS;

EID: 18544392871     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012998340673     Document Type: Article

References (37)

1. A. ARAPOSTATHIS, V.S. BORKAR, E. FERNÁNDEZ-GAUCHERAND, M.K. GHOSH, AND S.I. MARCUS, Discrete-time controlled Markov processes with average cost criterion: A survey, SIAM J. Control Optim., 31 (1993), pp. 282-344.
2. D.P. BERTSEKAS, Dynamic Programming: Deterministic and Stochastic Control Models, Prentice-Hall, Englewood Cliffs, NJ, 1987.
3. B.S. BORKAR, Control of Markov chains with long-run average cost criterion, in Stochastic Differential Systems, Stochastic Control Theory and Applications, W. Fleming and P.L. Lions, eds., IMA Vol. Math. Appl. 10, Springer-Verlag, Berlin, 1988, pp. 57-77.
4. V.S. BORKAR, Topics in Controlled Markov Processes, Pitman Res. Notes Math. Ser. 240, Longman, Harlow, UK, 1991.
5. R. CAVAZOS-CADENA AND E. FERNÁNDEZ-GAUCHERAND, Denumerable controlled Markov chains with average reward criterion: Sample path optimality, Math. Methods Oper. Res., 41 (1995), pp. 89-108.
6. R. DEKKER AND A. HORDIJK, Recurrence conditions for average and Blackwell optimality in denumerable state Markov decision chains, Math. Oper. Res., 17 (1992), pp. 271-289.
7. M. DUFLO, Méthodes Récursives Aléatoires, Masson, Paris, 1990.
8. E.B. DYNKIN AND A.A. YUSHKEVICH, Controlled Markov Processes, Springer-Verlag, New York, 1979.
9. P.W. GLYNN AND S.P. MEYN, A Lyapunov bound for solutions of the Poisson equation, Ann. Probab., 24 (1996), pp. 916-931.
10. J. GONZÁLEZ-HERNÁNDEZ AND O. HERNÁNDEZ-LERMA, Envelopes of sets of measures, tightness, and Markov control processes, Appl. Math. Optim., 40 (1999), pp. 377-392.
11. J. GONZÁLEZ-HERNÁNDEZ AND O. HERNÁNDEZ-LERMA, Constrained Markov control processes in Borel spaces: The discounted case, Math. Methods Oper. Res., submitted.
12. E. GORDIENKO AND O. HERNÁNDEZ-LERMA, Average cost Markov control processes with weighted norms: Existence of canonical policies, Appl. Math. (Warsaw), 23 (1995), pp. 199-218.
13. E. GORDIENKO AND O. HERNÁNDEZ-LERMA, Average cost Markov control processes with weighted norms: Value iteration, Appl. Math. (Warsaw), 23 (1995), pp. 219-237.
14. P. HALL AND C.C. HEYDE, Martingale Limit Theory and Its Application, Academic Press, New York, 1980.
15. O. HERNÁNDEZ-LERMA, Adaptive Markov Control Processes, Springer-Verlag, New York, 1989.
16. O. HERNÁNDEZ-LERMA AND J.B. LASSERRE, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York, 1996.
17. O. HERNÁNDEZ-LERMA AND J.B. LASSERRE, Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math., 47 (1997), pp. 125-154.
18. O. HERNÁNDEZ-LERMA AND O. VEGA-AMAYA, Infinite-horizon Markov control processes with undiscounted cost criteria: From average to overtaking optimality, Appl. Math. (Warsaw), 25 (1998), pp. 153-178.
19. A. HORDIJK, Dynamic Programming and Markov Potential Theory, Mathematical Centre Tracts, 51, Mathematisch Centrum, Amsterdam, 1974.
20. A. HORDIJK AND A.A. YUSHKEVICH, Blackwell optimality in the class of stationary policies in Markov decision chains with a Borel state space and unbounded rewards, Math. Methods Oper. Res., 49 (1999), pp. 1-39.
21. Y. HUANG AND L.C.M. KALLENBERG, On finding optimal policies for Markov decision chains: A unifying framework for mean-variance tradeoffs, Math. Oper. Res., 19 (1994), pp. 434-448.
22. N.V. KARTASHOV, Strong Stable Markov Chains, VSP, Utrecht, The Netherlands, 1996.
23. P.R. KUMAR AND P. VARAIYA, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice-Hall, Englewood Cliffs, NJ, 1986.
24. M. KURANO, Markov decision processes with a minimum-variance criterion, J. Math. Anal. Appl., 123 (1987), pp. 572-583.
25. H.J. KUSHNER, Stochastic Stability and Control, Academic Press, New York, 1967.
26. J.B. LASSERRE, Sample-path average optimality for Markov control processes, IEEE Trans. Automat. Control, 44 (1999), pp. 1966-1971.
27. M. LOÈVE, Probability Theory II, Springer-Verlag, New York, 1978.
28. P. MANDL, On the variance in controlled Markov chains, Kybernetika (Prague), 7 (1971), pp. 1-12.
29. P. MANDL, A connection between controlled Markov chains and martingales, Kybernetika (Prague), 9 (1973), pp. 237-241.
30. P. MANDL, Estimation and control in Markov chains, Adv. Appl. Probab., 6 (1974), pp. 40-60.
31. S.P. MEYN AND R.L. TWEEDIE, Markov Chains and Stochastic Stability, Springer-Verlag, London, 1993.
32. M.L. PUTERMAN, Markov Decision Processes, John Wiley, New York, 1994.
33. L.I. SENNOTT, Stochastic Dynamic Programming and the Control of Queueing Systems, John Wiley, New York, 1999.
34. H.C. TIJMS, Stochastic Modelling and Analysis: A Computational Approach, John Wiley, Chichester, U.K., 1986.
35. O. VEGA-AMAYA, Sample-path average optimality of Markov control processes with strictly unbounded cost, Appl. Math. (Warsaw), to appear.
36. O. VEGA-AMAYA, Markov Control Processes in Borel Spaces: Undiscounted Cost Criteria, Doctoral Thesis, UAM-Iztapalapa, México, 1998 (in Spanish).
37. A.A. YUSHKEVICH, On a class of strategies in general Markov decision models, Theory Probab. Appl., 18 (1973), pp. 777-779.