Asymptotic optimization of a nonlinear hybrid system governed by a Markov decision process

SIAM Journal on Control and Optimization

Volumn 35, Issue 6, 1997, Pages 2070-2085

  Altman, Eitan ^a   Gaitsgory, Vladimir ^b  

^a INRIA   (France)

^b UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

Author keywords

Asymptotic optimality; Averaging; Hybrid stochastic systems; Markov decision processes; Nonlinear dynamics

Indexed keywords

ASYMPTOTIC STABILITY; MARKOV PROCESSES; NONLINEAR CONTROL SYSTEMS; OPTIMAL CONTROL SYSTEMS; OPTIMIZATION; PARAMETER ESTIMATION; PROBLEM SOLVING; STATE ESTIMATION; STATISTICAL METHODS;

MARKOV DECISION PROCESS; NONLINEAR HYBRID SYSTEMS;

STOCHASTIC CONTROL SYSTEMS;

EID: 0031268841     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012995279985     Document Type: Article

References (27)

1. M. ABBAD AND J. A. FILAR, Perturbation and stability theory for Markov control problems, IEEE Trans. Automat. Control, 37 (1992), pp. 1415-1420.
2. E. ALTMAN AND V. A. GAITSGORY, Control of a hybrid stochastic system, Systems Control Lett., 20 (1993), pp. 307-314.
3. E. ALTMAN AND A. SHWARTZ, Adaptive control of constrained Markov chains: Criteria and policies, Ann. Oper. Res., 28 (1991), Special issue on "Markov Decision Processes," O. Hernandez-Lerma and J. B. Lasserre, eds., pp. 101-134.
4. E. ALTMAN AND A. SHWARTZ, Adaptive control of constrained Markov chains: Criteria and policies, Ann. Oper. Res., 28 (1991), Special issue on "Markov Decision Processes," O. Hernandez-Lerma and J. B. Lasserre, eds., pp. 101-134.
5. E. ALTMAN AND O. ZEITOUNI, Rate of convergence of empirical measures and costs in controlled Markov chains and transient optimality, Math. Oper. Res., 19 (1994), pp. 955-974.
6. A. BENSOUSSAN, Perturbation Methods in Optimal Control Problems, John Wiley, New York, 1989.
7. A. BENSOUSSAN AND G.L. BLANKENSHIP, Singular perturbations in stochastic control, in Singular Perturbations and Asymptotic Analysis in Control Systems, P. Kokotovic, A. Bensoussan, and G. Blankenship, eds., Lecture Notes in Control and Inform. Sciences 90, Springer-Verlag, New York, 1987, pp. 171-260.
8. D. BIELECKI AND J.A. FILAR, Singularly perturbed Markov control problem: Limiting average cost, Ann. Oper. Res., 28 (1991), pp. 153-168.
9. F. H. CLARKE, Optimization and Nonsmooth Analysis, John Wiley, New York, 1983.
10. F. DELEBECQUE AND J. QUADRAT, Optimal control of Markov chains admitting strong and weak interactions, Automatica J. IFAC, 17 (1980), pp. 281-296.
11. F. DELEBECQUE AND J. QUADRAT, Contribution of stochastic control singular perturbation averaging and team theories to an example of large scale systems: Management and hydropower production, IEEE Trans Automat. Control, AC-23 (1978), pp. 209-222.
12. C. DERMAN, Finite State Markovian Decision Processes, Academic Press, New York, 1970.
13. C. DERMAN AND R. E. STRAUCH, A note on memoryless rules for controlling sequential control processes, Ann. Math. Stat., 37 (1966), pp. 276-278.
14. W. H. FLEMING AND R. W. RISHEL, Deterministic and Stochastic Optimal Control, Springer-Verlag, Berlin, Heidelberg, New York, 1975.
15. W. H. FLEMING AND H. M. SONER, Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, 1993.
16. V. GAITSGORY, Suboptimization of singularly perturbed control systems, SIAM J. Control Optim., 30 (1992), pp. 1228-1249.
17. I. I. GIKHMAN, Po povodu odnoi teoremi N.I. Bogolubova, Ukrain. Mat. Zh., 4 (1952), pp. 215-218.
18. A. HORDIJK, Dynamic Programming and Markov Potential Theory, 2nd ed., Mathematical Centre Tracts 51, Mathematisch Centrum, Amsterdam, 1977.
19. J. G. KEMENY AND J. L. SNELL, Finite Markov Chains, D. Van Nostrand, New York, 1960.
20. R. Z. KHAS'MINSKY, Ustoichivost' Sistem Differencial'nih Uravnenii Pri Sluchainih Vozmuscheniah Ikh Parametrov, Nauka, Moskva, 1969. English translation: Stochastic Stability of Differential Equations, 2nd ed., Sijthoff and Noordhoff, Alphen aan den Rijn, the Netherlands, Rockville, MD, 1980.
21. R. Z. KHAS'MINSKY, Ustoichivost' Sistem Differencial'nih Uravnenii Pri Sluchainih Vozmuscheniah Ikh Parametrov, Nauka, Moskva, 1969. English translation: Stochastic Stability of Differential Equations, 2nd ed., Sijthoff and Noordhoff, Alphen aan den Rijn, the Netherlands, Rockville, MD, 1980.
22. L. KLEINROCK, Queuing Systems, Volume II: Computer Applications, John Wiley, New York, 1976.
23. P. V. KOKOTOVIC, H. KHALIL, AND J. O'REILLY, Singular Perturbations in Control Analysis and Design, Academic Press, New York, 1986.
24. M. A. KRASNOSEL'SKII AND S.G. KREIN, O principe usrednenia v nelineinoi mechanike, Uspekhi Mat. Nauk, 10 (1955), pp. 147-152.
25. H. KUSHNER, Weak Convergence and Singularly Perturbed Stochastic Control and Filtering Problems, Birkhäuser, Boston, 1990.
26. R. G. PHILIPS AND P.V. KOKOTOVIC, A singular perturbation approach to modeling and control of Markov chains, IEEE Trans. Automat. Control, AC-26 (1981), pp. 1087-1094.
27. A. A. PERVOZVANSKY AND V. GAITSGORY, Theory of Suboptimal Decisions, Kluwer, Dordrecht, the Netherlands, 1988.