Minimax control of discrete-time stochastic systems

SIAM Journal on Control and Optimization

Volumn 41, Issue 5, 2003, Pages 1626-1659

  Gonzalez Trejo J I ^a   Hernandez Lerma O ^b   Hoyos Reyes, L F ^a  

^a UNIVERSIDAD AUTÓNOMA METROPOLITANA AZCAPOTZALCO   (Mexico)

^b DEPARTAMENTO DE FÍSICA   (Mexico)

Author keywords

Discrete time stochastic control systems; Games against nature; Markov games with complete information; Minimax control problems

Indexed keywords

DISCRETE TIME CONTROL SYSTEMS; GAME THEORY; MARKOV PROCESSES;

BOREL SPACES;

STOCHASTIC CONTROL SYSTEMS;

EID: 0141645552     PISSN: 03630129     EISSN: None     Source Type: Journal    
DOI: 10.1137/S0363012901383837     Document Type: Article

References (64)

1. E. ALTMAN AND A. HORDIJK, Zero-sum Markov games and worst-case optimal control of queueing systems, Queueing Syst. Theory Appl., 21 (1995), pp. 415-447.
2. E. ALTMAN, A. HORDIJK, AND F. M. SPIEKSMA, Contraction conditions for average and α-discount optimality in countable state Markov games with unbounded rewards, Math. Oper. Res., 22 (1997), pp. 588-618.
3. M. A. BARRÓN, R. AGUILAR, J. GONZÁLEZ, AND E. MELÉNDEZ, Model-based control of mold level in a steel continuous caster under model uncertainties, Control Engrg. Practice, 6 (1998), pp. 191-198.
4. D. P. BERTSEKAS AND S. E. SHREVE, Stochastic Optimal Control: The Discrete Time Case, Academic Press, New York, 1978.
5. G. BOCHER, R. OBERMANN, B. WINKLER, G. KRUGER, AND P. PATTE, Slab quality improvement by means of advanced mould level control, in Proceedings of the 1st European Conference on Continuous Casting, Florence, Italy, Institute of Materials, 1991, pp. 1205-1214.
6. S. P. CORALUPPI AND S. I. MARCUS, Mixed risk-neutral/minimax control of discrete-time finite-state Markov decision process, IEEE Trans. Automat. Control 45 (2000), pp. 528-532.
7. Y. M. I. DIRICKX AND L. P. JENXERGREN, On the optimality of myopic policies in sequential decision problems, Manage. Sci., 21 (1975), pp. 550-556.
8. C. A. J. M. DIRVEN AND O. J. VRIEZE, Advertising models, stochastic games and myopic strategies, Oper. Res., 34 (1986), pp. 645-649.
9. M. DUSSUD, S. GALICHET, AND L. P. FOULLOY, Application of fuzzy logic control for continuous casting mould level control, IEEE Trans. Control Systems Tech., 6 (1998), pp. 246-256.
10. J. FILAR AND K. VRIEZE, Competitive Markov Decision Processes, Springer-Verlag, New York, 1997.
11. M. K. GHOSH AND A. BAGCHI, Stochastic games with average payoff criterion, Appl. Math. Optim., 38 (1998), pp. 283-301.
12. X. P. GUO AND O. HERNÁNDEZ-LERMA, Zero-Sum Games for Continuous-Time Markov Games with a Discounted Payoff Criterion, preprint, 2000.
13. X. P. GUO AND O. HERNÁNDEZ-LERMA, Nonzero-Sum Games for Continuous-Time Markov Chains with Unbounded Discounted Payoffs, preprint, 2001.
14. X. P. GUO AND O. HERNÁNDEZ-LERMA, Zero-Sum Games for Nonhomogenous Markov Chains with Expected Average Payoff Criterion, preprint, 2002.
15. O. HERNÁNDEZ-LERMA, Adaptive Markov Control Processes, Springer-Verlag, New York, 1989.
16. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Discrete-Time Markov Control Processes: Basic Optimality Criteria, Springer-Verlag, New York, 1996.
17. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Further Topics in Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.
18. O. HERNÁNDEZ-LERMA AND J. B. LASSERRE, Zero-sum stochastic games in Borel spaces: Average payoff criteria, SIAM J. Control Optim., 39 (2001), pp. 1520-1539.
19. O. HERNÁNDEZ-LERMA, R. MONTES-DE-OCA, AND R. CAVAZOS-CADENA, Recurrence conditions for Markov decision processes with Borel state space: A survey, Ann, Oper. Res., 28 (1991), pp. 29-46.
20. T. HESKETH, D. J. CLEMENTS, AND R. WILLIAMS, Adaptive mould level control for continuous steel slab casting, Automatica, 29 (1993), pp. 851-864.
21. N. HILGERT AND J. A. MINJÁREZ-SOSA, Adaptive policies for time-varying stochastic systems under discounted criterion, Math. Methods Oper. Res., 54 (2001), pp. 491-505.
22. A. HORDIJK, O. PASSCHIER, AND F. M. SPIEKSMA, Optimal control against worst case admission policies: A multichained stochastic game, Math. Methods Oper. Res., 45 (1997), pp. 281-301.
23. A. HORDIJK AND A. A. YUSHKEVICH, Blackwell optimality in the class of all policies in Markov decision chains with Borel state space and unbounded rewards, Math. Methods Oper. Res., 49 (1999), pp. 421-448.
24. R. JAGANNATHAN, A minimax ordering policy for the infinite stage dynamic inventory problem, Manage. Sci., 24 (1978), pp. 1138-1149.
25. A. JAŚKIEWICZ AND A. S. NOWAK, On the optimality equation for zero-sum ergodic stochastic games, Math. Methods Oper. Res., 54 (2001), pp. 291-301.
26. H. KATO AND M. YAMASITA, New automation and control technology of slab caster, in Proceedings of the IFAC Symposium on Automation in Mining, Mineral and Metal Processing (MMM'86), Buenos Aires, Argentina, International Federation of Automatic Control (IFAC), 1986, pp. 253-258.
27. R. M. C. DE KEYSER, Improved mold-level control in a continuous steel casting line, Control Engrg. Practice, 5 (1997), pp. 231-237.
28. H. KITADA, O. KONDO, H. KUSACHI, AND K. SASAME, H∞ Control of molten steel level in continuous caster, IEEE Trans. Control Systems Tech., 6 (1998), pp. 200-207.
29. N. KIUPEL, P. M. FRANK, AND J. WOCHNIK, Improvement of mold-level control using fuzzy logic, Eng. Appl. Artif. Intell., 7 (1994), pp. 493-499.
30. G. KRUGER, Advanced mould level control for continuous casting plants, Metallurgical Plant and Technol., 3 (1985), pp. 42-49.
31. H.-U. KÜENLE, On multichain Markov games, in Advances in Dynamic Games and Applications, Ann. Internat. Soc. Dynam. Games 6, Birkhäuser, Boston, 2001, pp. 147-163.
32. H.-U. KÜENLE, Stochastiche Spiele und Entscheidungsmodelle, B. G. Teubner, Leipzig, 1986.
33. H.-U. KÜENLE, On the optimality of (s,S)-strategies in a minimax inventory model with average cost criterion, Optimization, 22 (1991), pp. 123-138.
34. H.-U. KÜENLE, Stochastic games with complete information and average cost criteria, in Advances in Dynamic Games and Applications, J. A. Filar, V. Gaitsgory, and K. Mizukami, eds., Birkhäuser, Boston, 2000, pp. 325-338.
35. M. KURANO, Minimax strategies for average cost stochastic games with an application to inventory models, J. Oper. Res. Soc. Japan, 30 (1987), pp. 232-247.
36. W. S. LOVEJOY, Myopic policies for some inventory models with uncertain demand distributions, Manage. Sci., 36 (1990), pp. 724-738.
37. A. MAITRA AND W. SUDDERTH, Finitely additive and measurable stochastic games, Internat. J. Game Theory, 22 (1993), pp. 201-223.
38. A. MAITRA AND W. SUDDERTH, Borel stochastic games with lim sup payoff, Ann. Probab., 21 (1993), pp. 861-885.
39. A. MAITRA AND W. SUDDERTH, Discrete Gambling and Stochastic Games, Springer-Verlag, New York, 1996.
40. A. MAITRA AND W. SUDDERTH, Finitely additive stochastic games with Borel measurable pay-offs, Internat. J. Game Theory, 27 (1998), pp. 257-267.
41. D. A. MARTIN, The determinacy of Blackwell games, J. Symbolic Logic, 63 (1998), pp. 1565-1581.
42. J. A. MINJÁREZ-SOSA, Nonparametric adaptive control for discrete-time Markov processes with unbounded costs under the average criterion, Appl. Math. (Warsaw), 26 (1999), pp. 267-280.
43. A. S. NOWAK, Minimax selection theorems, J. Math. Anal. Appl., 103 (1984), pp. 106-116.
44. A. S. NOWAK, Measurable selection theorems for minimax stochastic optimization problems, SIAM J. Control Optim., 23 (1985), pp. 466-476.
45. A. S. NOWAK, Zero-sum average payoff stochastic games with general state space, Games Econ. Behavior, 7 (1994), pp. 221-232.
46. A. S. NOWAK, Optimal strategies in a class of zero-sum ergodic stochastic games, Math. Methods Oper. Res., 50 (1999), pp. 399-419.
47. J. PAIUK, A. ZANINI, M. REMORINO, AND O. FROLA, The automatic mould level control for a continuous casting process, in Proceedings of the IFAC Symposium on Automation in Mining, Mineral and Metal Processing (MMM'89), Buenos Aires, Argentina, International Federation of Automatic Control (IFAC), 1989, pp. 205-208.
48. O. PASSCHIER, The Theory of Markov Games and Queueing Control, Ph.D. thesis, Dept. of Mathematics and Computer Science, Leiden University, The Netherlands, 1996.
49. U. RIEDER, Measurable selection theorems for optimization problems, Manuscripta Math., 24 (1978), pp. 115-131.
50. U. RIEDER, On Semi-Continuous Dynamic Games, Technical Report, University of Karlsruhe, Germany, 1978.
51. U. RIEDER, Non-cooperative dynamic games with general utility functions, in Stochastic Games and Related Topics, T. E. S. Raghavan et al., ed., Kluwer, Dordrecht, The Netherlands, 1991, pp. 161-174.
52. U. RIEDER, Average optimality in Markov games with general state space, in Proceedings of the 3rd International Conference on Approximation Theory and Optimization, Puebla, Mexico, 1995. Available online from www.emis.de/proceedings/.
53. D. B. ROSENFIELD, Optimality of myopic policies in disposing excess inventory, Oper. Res., 40 (1992), pp. 800-803.
54. Y. SASABE, S. KUBOTA, A. KOYAMA, AND H. MIKI, Real-time expert system applied to mould bath level control of continuous caster, ISIJ International, 30 (1990), pp. 136-141.
55. M. SCHÄL, Conditions for optimality and for the limit of n-stage optimal policies to be optimal, Z. Wahrsch. Verw. Gebiete, 32 (1975), pp. 179-196.
56. L. I. SENNOTT, Zero-sum stochastic games with unbounded cost: Discounted and average cost cases, Z. Oper. Res., 39 (1994), pp. 209-225.
57. R. SERFOZO, Convergence of Lebesgue integrals with varying measures, Sankhyā Ser. A, 44 (1982), pp. 380-402.
58. M. J. SOBEL, Myopic solutions of Markov decision processes and stochastic games, Oper. Res., 29 (1981), pp. 945-1009.
59. M. J. SOBEL, Myopic solutions of affine dynamic models, Oper. Res., 38 (1990), pp. 847-853.
60. L. TESFATSION, Global and approximate global optimality of myopic economic decisions, J. Econom. Dynamics Control, 2 (1980), pp. 135-160.
61. O. VEGA-AMAYA, The Average Cost Optimality Equation: A Fixed Point Approach, preprint; available from http://fractus.mat.uson.mx/̃tedi/reportes
62. J. H. WATTERS, G. S. JIANG, C. J. TREADGOLD, AND A. KUMAR, Model studies of surface wave phenomena in the continuous casting mould, in Proceedings of the 2nd European Continuous Casting Conference, Vol. 1, Düsseldorf, Germany, Institute of Materials, 1994, pp. 95-102.
63. J. WEISHAUPT, Optimal myopic policies and index policies for stochastic scheduling problems, Z. Oper. Res., 40 (1994), pp. 75-89.
64. J. WESSELS, Markov games with unbounded rewards, in Dynamische Optimierung, Bonner Math. Schriften 98, M. Schäl, ed., University of Bonn, Germany, 1977, pp. 133-147.