Nearly optimal control of singularly perturbed Markov decision processes in discrete time

Applied Mathematics and Optimization

Volumn 44, Issue 2, 2001, Pages 105-129

  Liu, R H ^a   Zhang, Q ^a   Yin, G ^b  

^a UNIVERSITY OF GEORGIA   (United States)

^b WAYNE STATE UNIVERSITY   (United States)

Author keywords

Asymptotically optimal control; Dynamic programming; Markov decision process; Singular perturbation

Indexed keywords

ASYMPTOTIC STABILITY; DECISION THEORY; DISCRETE TIME CONTROL SYSTEMS; DYNAMIC PROGRAMMING; FINITE ELEMENT METHOD; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION; PERTURBATION TECHNIQUES; STATE SPACE METHODS;

CONTINUOUS TIME LIMIT PROBLEMS; MARKOV DECISION PROCESSES; NEARLY OPTIMAL CONTROL; SINGULAR PERTURBATION;

MARKOV PROCESSES;

EID: 0007679039     PISSN: 00954616     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00245-001-0016-8     Document Type: Article

References (20)

1. Abbad M, Filar JA (1992) Perturbation and stability theory for Markov control problems. IEEE Trans Automat Control AC-37: 1415-1420
2. Abbad M, Filar JA, Bielecki T R (1992) Algorithms for singularly perturbed limiting average Markov control problems. IEEE Trans Automat Control AC-37: 1421-1425
3. Bielecki TR, Filar JA (1991) Singularly perturbed Markov control problem: limiting average cost. Ann Oper Res 28: 153-168.
4. Bielecki TR, Stettner L (1998) Ergodic control of a singularly perturbed Markov process in discrete time with general state and compact action spaces. Appl Math Optim 38: 261-281
5. Blankenship G (1981) Singularly perturbed difference equations in optimal control problems. IEEE Trans Automat Control AC-26: 911-917
6. Delebecque F (1983) A reduction process for perturbed Markov chains. SIAM J Appl Math 48: 325-350
7. Delebecque F, Quadrat JP (1981) Optimal control for Markov chains admitting strong and weak interactions. Automatica 17: 281-296
8. Derman C (1970) Finite State Markovian Decision Processes. Academic Press, New York
9. Hoppensteadt FC, Miranker WL (1977) Multitime methods for systems of difference equations. Stud Appl Math 56: 273-289
10. Khasminskii RZ, Yin G, Zhang Q (1996) Asymptotic expansions of singularly perturbed systems involving rapidly fluctuating Markov chains. SIAM J Appl Math 56: 277-293
11. Khasminskii RZ, Yin G, Zhang Q (1997) Constructing asymptotic series for probability distribution of Markov chains with weak and strong interactions. Quart Appl Math LV: 177-200
12. Naidu DS (1988) Singular Perturbation Methodology in Control Systems. Peter Peregrinus, Stevenage, Herts
13. Pervozvanskii AA, Gaitsgori VG (1988) Theory of Suboptimal Decisions: Decomposition and Aggregation. Kluwer Dordrect
14. Quadrat JP (1983) Optimal control of perturbed Markov chains: the multitime scale case. In: Singular Perturbations in Systems and Control (MD Aradema, ed); CISM Courses and Lectures No 280. Springer-Verlag, Berlin, pp. 216-239
15. Ross S (1983) Introduction to Stochastic Dynamic Programming. Academic Press, New York
16. White DJ (1992) Markov Decision Processes. Wiley, New York
17. Yin G, Zhang, Q (1998) Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach. Springer-Verlag, New York
18. Yin G, Zhang Q (2000) Singularly perturbed discrete-time Markov chains. SIAM J Appl Math 61:834-854
19. Yin G, Zhang Q, Badowski G (2000) Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states. ann Appl Probab 10: 549-572
20. Zhang Q (1996) Finite state Markovian decision processes with weak and strong interactions. Stochastics Stochastics Rep 59: 283-304