Linear Programming Approximations for Markov Control Processes in Metric Spaces

Acta Applicandae Mathematicae

Volumn 51, Issue 2, 1998, Pages 123-139

  Hernández Lerma, Onésimo ^a   Lasserre, Jean B ^b  

^a DEPARTAMENTO DE FÍSICA   (Mexico)

^b LAAS CNRS   (France)

Author keywords

(discrete time) Markov control processes; Aggregation; Infinite dimensional linear programming; Inner approximations; Relaxation

Indexed keywords

EID: 0012265595     PISSN: 01678019     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1005826226226     Document Type: Article

References (32)

1. Altman, E.: Constrained Markov Decision Processes, Rapport RR-2574, INRIA, Centre Sophia-Antipolis, 1995. (Book in preparation.)
2. Anderson, E. J. and Nash, P.: Linear Programming in Infinite-Dimensional Spaces, Wiley, Chichester, 1987.
3. Balder, E.: Lectures on Young Measures, Cahiers de mathématiques de la décision, CEREMADE, Université Paris IX-Dauphine, Paris, 1995.
4. Billingsley, P.: Convergence of Probability Measures, Wiley, New York, 1968.
5. Bourbaki, N.: Intégration, Chap. IX, Hermann, Paris, 1969.
6. Costa, O. L. V.: Impulse control of piecewise-deterministic processes via linear programming, IEEE Trans. Automat. Control 36 (1991), 371-375.
7. Craven, B. D. and Koliha, J. J.: Generalizations of Farkas' theorem, SIAM J. Math. Anal. Appl. 8 (1977), 983-997.
8. González, R. and Rofman, E.: An algorithm to obtain the maximum solution of the Hamilton-Jacobi equation, in: Lecture Notes in Control and Inform. Sci. 6, Springer-Verlag, Berlin, 1978, pp. 109-116.
9. Hernández-Lerma, O.: Adaptive Markov Control Processes, Springer-Verlag, New York, 1989.
10. Hernández-Lerma, O. and González-Hernández, J.: Infinite linear programming and multichain Markov control processes in uncountable spaces, SIAM J. Control Optim. 36 (1998), to appear.
11. Hernández-Lerma, O. and Lasserre, J. B.: Linear programming and average optimality of Markov control processes on Borel spaces - unbounded costs, SIAM J. Control Optim. 32 (1994), 480-500.
12. Hernández-Lerma, O. and Lasserre, J. B.: Average optimality in Markov control processes via discounted cost problems and linear programming, SIAM J. Control Optim. 34 (1996), 295-310.
13. Hernández-Lerma, O. and Lasserre, J. B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer, New York, 1996.
14. Hernández-Lerma, O. and Lasserre, J. B.: Invariant probabilities for Feller-Markov chains, J. Appl. Math. Stochastic Anal. 8 (1995), 341-345.
15. Hernández-Lerma, O. and Lasserre, J. B.: Cone-constrained linear equations in Banach spaces, J. Convex Anal. 4 (1997), 149-164.
16. Hernández-Lerma, O. and Lasserre, J. B.: Policy iteration for average cost Markov control processes on Borel spaces, Acta Appl. Math. 47(2) (1997), 125-154.
17. Hernández-Lerma, O. and Runggaldier, W.: Monotone approximations for convex stochastic control problems, J. Math. Systems, Estim. Control 4 (1994), 99-140.
18. Hordijk, A. and Lasserre, J. B.: Linear programming formulation of Markov decision processes in countable state space: the multichain case, Z. Oper. Res. 40 (1994), 91-108.
19. Kallenberg, L. C. M.: Linear programming and finite Markovian control problems, Mathematical Centre Tracts #148, Amsterdam, 1983.
20. Kallenberg, L. C. M.: Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory, Part II: Applications, Z. Oper. Res. 40 (1994), 1-42, 127-143.
21. Lasserre, J. B.: Average optimal stationary policies and linear programming in countable state MDPs, J. Math. Anal. Appl. 183 (1994), 233-249.
22. Montes-de-Oca, R. and Hernández-Lerma, O.: Value iteration in average cost Markov control processes on Borel spaces, Acta Appl. Math. 42 (1996), 203-222.
23. Parthasarathy, K. R.: Probability Measures on Metric Spaces, Academic Press, New York, 1967.
24. Puterman, M. L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, New York, 1994.
25. Robertson, A. P. and Robertson, W.: Topological Vector Spaces, Cambridge University Press, Cambridge, 1964.
26. Rubio, J.: Control and Optimization: The Linear Treatment of Nonlinear Problems, Manchester University Press, Manchester and Wiley, New York, London, 1986.
27. Rubio, J.: The global control of nonlinear diffusion equations, SIAM J. Control Optim. 33 (1995), 308-322.
28. Rudolph, H.: The SILP relaxation method in optimal control theory: general boundary conditions, I, II, Z. Anal. Anwendungen 11 (1992), 143-151, 431-436.
29. Taksar, M.: Infinite dimensional linear programming approach to singular stochastic control, preprint, Dept. of Applied Mathematics, SUNY-Stony Brook, 1995.
30. Tijms, H. C.: Stochastic Models, An Algorithmic Approach, Wiley, New York, 1994.
31. Vershik, A.M.: Some remarks on the infinite-dimensional problems of linear programming, Russian Math. Surveys 29 (1970), 117-124.
32. Vershik, A. M. and Temel't, V.: Some questions concerning the approximation of the optimal value of infinite-dimensional problems in linear programming, Siberian Math. J. 9 (1968), 591-601.