Decentralized Learning in Finite Markov Chains

IEEE Transactions on Automatic Control

Volumn 31, Issue 6, 1986, Pages 519-526

  Wheeler, Richard M ^a   Narendra, Kumpati S ^b  

^a SANDIA NATIONAL LABORATORIES   (United States)

^b YALE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONTROL SYSTEMS, OPTIMAL; CONTROL SYSTEMS, STOCHASTIC; DECISION THEORY AND ANALYSIS; PROBABILITY - RANDOM PROCESSES;

DECENTRALIZED LEARNING; DISTRIBUTED DECISION-MAKING; FINITE MARKOV CHAINS; SEQUENTIAL STOCHASTIC GAMES;

SYSTEMS SCIENCE AND CYBERNETICS;

EID: 0022738693     PISSN: 00189286     EISSN: 15582523     Source Type: Journal    
DOI: 10.1109/TAC.1986.1104342     Document Type: Article

References (34)

1. R. A. Howard, Dynamic Programming and Markov Processes. Cambridge, MA: M.I.T. Press, 1960.
2. S. M. Ross, Applied Probability Models with Optimization Applications. San Francisco, CA: Holden-Day, 1970.
3. C. Denman, Finite State Markovian Decision Processes. New York: Academic, 1970.
4. P. Mandl, “Estimation and control in Markov chains,” Adv. App!. Prob., vol. 6, 40–60, 1974.
5. V. Borkar and P. Varaiya, “Adaptive control of Markov chains, I: Finite parameter set,” IEEE Trans. Automat. Contr., vol. AC-24, pp. 953–958, 1979.
6. P. R. Kumar and W. Lin, “Optimal adaptive controllers for unknown Markov chains,” IEEE Trans. Automat. Contr., vol. AC-27, pp. 765–774, 1982.
7. K. Astrom, “Optimal control of Markov processes with incomplete state information,- J. Math. Anal. App!., vol. 10, pp. 174–205, 1965.
8. K. Hsu and S. I. Marcus, “Decentralized control of finite state Markov processes,” IEEE Trans. Automat. Contr., vol. AC-27, pp. 426–431, 431, 1982.
9. R. E. Bellman, “A Markovian decision process,” J. Math. Mech., vol. 6, pp. 679–684, 1957.
10. D. J. White, “Dynamic programming, Markov chains, and the method of successive approximations,” J. Math. Anal. Appl., vol. 6, pp. 373–376, 1963.
11. P. Varaiya, “Optimal and suboptimal stationary controls for Markov chains,” IEEE Trans. Automat. Contr., vol. AC-23, pp. 388–394, 1978.
12. J.-P. Forestier and P. Varaiya, “Multi layer control of large Markov chains,” IEEE Trans. Automat. Contr., vol. AC-23, pp. 298–305, 1978.
13. Y. M. El-Fattah, “Recursive algorithms for adaptive control of finite Markov chains,” IEEE Trans. Syst., Man, Cybern., vol. SMC-1l, pp. 135–144, 1981.
14. M. Sato, K. Abe, and H. Takeda, Learning control of finite Markov chains with unknown transition probabilities,” IEEE Trans. Automat. Contr., vol. AC-27, 502–505, 1982.
15. S. Lakshmivarahan, Learning Algorithms: Theory and Applications. New York: Springer-Verlag, 1981.
16. R. R. Bush and F. Mosteller, Stochastic Models for Learning. New York: Wiley, 1958.
17. R. C. Atkinson, G. H. Bower, and E. J. Crothers, An Introduction to Mathematical Learning Theory. New York: Wiley, 1965.
18. M. F. Norman, Markov Processes and Learning Models. New York: Academic, 1972.
19. J. M. Mendel and K. S. Fu, Eds., Adaptive, Learning and Pattern Recognition Systems. New York: Academic, 1970.
20. M. L. Tsetlin, Automaton Theory and Modeling of Biological Systems. New York: Academic, 1973.
21. K. S. Narendra and M. A. L. Thathachar, “Learning automata—A survey,” IEEE Trans. Syst., Man, Cybern., vol. SMC-4, pp. 323–334, 334, 1974.
22. K. S. Narendra and S. Lakshmivarahan, “Learning automata—A critique,” J. Cybern. Inf. Sci., vol. 4, pp. 53–66, 1977.
23. S. Lakshmivarahan and M. A. L. Thathachar, “Absolute expediency of Q- and 5- model learning algorithms,” Trans. Syst., Man, Cybern., vol. SMC-6, pp. 222–226, 1976.
24. R. M. Wheeler, Jr. and K. S. Narendra, “Learning models for decentralized decision making,” Automatica, vol. 21, pp. 479–484, 1985.
25. S. Lakshmivarahan and K. S. Narendra, “Learning algorithms for two-person person zero-sum stochastic games with incomplete information,” Math. Oper. Res., vol. 6, pp. 379–386, 1981.
26. S. Lakshmivarahan and K. S. Narendra, “Learning algorithms for two-person person zero-sum stochastic games with incomplete information: A unified approach,” SIAM J. Contr. and Opt., vol. 20, pp. 541–552, 1982.
27. J. F. Nash, “Equilibrium points in n-person games,” Proc. National Acad. Sci. USA, vol. 36, pp. 48–49, 1950.
28. V. L. Stefanyuk, “Example of a problem in the joint behavior of two automata,” Automat. Telemekh., vol. 24, pp. 781–784, 1963.
29. S. L. Ginsburg, V. Y. Krylov, and M. L. Tsetlin, “One example of a game for many identical automata,” Automat. Telernekh., vol. 25, pp. 668–671, 1964.
30. V. I. Varshavsldi, “Collective behavior and control problems,” in Machine Intelligence, 3, D. Michie, Ed. Edinburgh: Edinburgh Univ., 1968.
31. R. Viswanathan and K. S. Narendra, “Competitive and cooperative games of variable-structure stochastic automata,” J. Cybern., vol. 3, pp. 1–23, 1973.
32. N. Baba and Y. Sawaragi, “On the learning behavior of stochastic automata under a nonstationary random environment,” IEEE Trans. Syst., Man, Cybern., vol. SMC-5, pp. 273–275, 1975.
33. M. A. L. Thathachar and P. S. Sastry, “A new approach to the design of reinforcement schemes for learning automata,” Dept. Elec. Eng., Ind. Inst. Sci., Bangalore, India, Tech. Rep. EE/60, 1983.
34. B. Chandrasekaran and D. W. C. Shen, “On expediency and convergence in variable-structure automata,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-4, pp. 52–60, 1968.