Two-Person Zero-Sum Markov Games: Receding Horizon Approach

IEEE Transactions on Automatic Control

Volumn 48, Issue 11, 2003, Pages 1951-1961

  Chang, Hyeong Soo ^a   Marcus, Steven I ^b  

^a Sogang University   (South Korea)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Hindsight optimization; Infinite horizon cost; Markov game; Receding horizon control; Rollout

Indexed keywords

APPROXIMATION THEORY; ERROR ANALYSIS; GAME THEORY; LINEAR CONTROL SYSTEMS; MARKOV PROCESSES; OPTIMIZATION;

HINDSIGHT OPTIMIZATION; HORIZON CONTROL;

OPTIMAL CONTROL SYSTEMS;

EID: 0344395590     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2003.819077     Document Type: Article

References (34)

1. J. M. Alden and R. L. Smith, "Rolling horizon procedures in non-homogeneous Markov decision processes," Oper. Res., vol. 40, pp. S183-S194, 1992.
2. E. Altman, "Zero-sum Markov games and worst-cast optimal control of queuejng systems," QUESTA, vol. 21, pp. 415-447, 1995.
3. _, "Non zero-sum stochastic games in admission, service and routing control in queueing systems," QUESTA, vol. 23, pp. 259-279, 1996.
4. _, "Monotonicity of optimal policies in a zero sum game: A flow control model," Adv. Dyna. Games Applicat., pp. 269-286, 1994.
5. M. Baglietto, T. Parisini, and R. Zoppoli, "Neural approximators and team theory for dynamic routing: A receding horizon approach," Proc. IEEE Conf. Decision Control, pp. 3283-3288, 1999.
6. T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. New York: Academic, 1995.
7. D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont: Athena Scientific, 1995, vol. 1/2.
8. D. P. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Belmont, CA: Athena Scientific, 1996.
9. C. Boutilier, T. Dean, and S. Hanks, "Decision-theoretic planning: Structural assumptions and computational leverage," J. Artif. Intell. Res., vol. 11, pp. 1-94, 1999.
10. W. A. van den Broek, "Moving horizon control in dynamic games, " Comput. Econ. Finance, vol. 122, 1999.
11. H. S. Chang, R. Givan, and E. K. P. Chong, "On-line scheduling via sampling," in Proc. 5th Int. Conf. Artificial Intelligence Planning Scheduling, 2000, pp. 62-71.
12. _, "Parallel rollout for online solution of partially observable Markov decision processes,", 2003, submitted for publication.
13. H. S. Chang and S. Marcus, "Approximate receding horizon approach for Markov decision processes: Average reward case," J. Math. Anal. Applicat., vol. 286, no. 2, pp. 636-651, 2001, submitted for publication.
14. E. K. P. Chong, R. Givan, and H. S. Chang, "A framework for simulation-based network control via hindsight optimization," in Proc. 39th IEEE Conf. Decision Control, vol. 2000, pp. 1433-1438.
15. J. Cruz, M. Simaan, A. Gacic, and Y. Liu, "Moving horizon nash strategies for a military air operation," IEEE Trans. Aero. Electron. Syst., vol. 38, pp. 989-999, June 2002.
16. J. Filar and K. Vrieze, Competitive Markov Decision Processes. New York: Springer-Verlag, 1996.
17. B. L. Fox and D. M. Landi, "An algorithm for identifying the ergodic subchains and transient states of a stochastic matrix," Commun. ACM, vol. 2, pp. 619-621, 1968.
18. M. L. Ginsberg, "GIB: steps toward an expert-level bridge-playing program," in Proc. IJCAI, 1999, pp. 584-589.
19. D. Gillette, "Stochastic games with zero stop probabilities," in Contributions to the Theory of Games, M. Dresher, A. Tucker, and P. Wolfe, Eds. Princeton, NJ: Princeton Univ. Press, vol. III, pp. 179-187.
20. O. Hernández-Lerma and J. B. Lasserre, "Error bounds for rolling horizon policies in discrete-time markov control processes," IEEE Trans. Automat. Contr., vol. 35, pp. 1118-1124, Oct. 1990.
21. Y. -C. Ho and K. -C. Chu, "Team decision theory and information structures in optimal control problems-part I," IEEE Trans. Automat. Contr., vol. AC-17, pp. 15-22, Feb. 1972.
22. L. Johansen, Lectures on Macroeconomic Planning. Amsterdam, The Netherlands: North-Holland, 1977.
23. M. Kearns, Y. Mansour, and S. Singh, "Fast planning in stochastic games," in Proc. UAI, 2000.
24. S. S. Keerthi and E. G. Gilbert, "Optimal, infinite horizon feedback laws for a general class of constrained discrete time systems: Stability and moving-horizon approximations," J. Optim. Theory Applicat., vol. 57, pp. 265-293, 1988.
25. D. Q. Mayne and H. Michalska, "Receding horizon control of nonlinear system," IEEE Trans. Automat. Contr., vol. 38, pp. 814-824, July 1990.
26. M. Morari and J. H. Lee, "Model predictive control: Past, present, and future," Comput. Chem. Eng., vol. 23, pp. 667-682, 1999.
27. T. Parthasarathy and M. Stern, "Markov Games - A survey," in Differential Games and Control Theory II, E. Roxin, P. Liu, and R. Sternberg, Eds. New York: Marcel Dekker, 1977, pp. 1-46.
28. M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, New York: Wiley, 1994.
29. K. W. Ross and R. Varadarajan, "Multichain Markov decision processes with a sample-path constraint: A decomposition approach," Math. Oper. Res., vol. 16, no. 1, pp. 195-207, 1991.
30. L. Shapley, "Stochastic games," in Proc. Nat. Acad. Sci., vol. 39, 1953, pp. 1095-1100.
31. M. Tidball and E. Airman, "Approximations in dynamic zero-sum games," SIAM J. Control Optim., vol. 34, no. 1, pp. 311-328, 1996.
32. J. Tsitsiklis, "Asynchronous stochastic approximation and Q-learning," Mach. Learn., vol. 16, pp. 185-202, 1994.
33. J. Van Der Wal, "Discounted markov games: Generalized policy iteration method," J. of Optim. Theory Applicat., vol. 25, no. 1, pp. 125-138, 1978.
34. _, "Stochastic dynamic programming: Successive approximations and nearly optimal strategies for Markov decision processes and Markov games," Ph.D. dissertation, Eindhoven, The Netherlands, 1980.