Learning Automata—A Survey

IEEE Transactions on Systems, Man and Cybernetics

Volumn SMC-4, Issue 4, 1974, Pages 323-334

  Narendra, Kumpati S ^a   Thathachar, M A L ^a  

^a YALE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SYSTEMS SCIENCE AND CYBERNETICS - LEARNING SYSTEMS;

AUTOMATA THEORY;

EID: 0016082525     PISSN: 00189472     EISSN: 21682909     Source Type: Journal    
DOI: 10.1109/TSMC.1974.5408453     Document Type: Article

References (83)

1. R. C. Atkinson, G. H. Bower, and E. J. Crothers, An Introduction to Mathematical Learning Theory. New York: Wiley, 1965.
2. R. R. Bush and F. Mosteller, Stochastic Models for Learning. New York: Wiley, 1958.
3. K. S. Fu, “Learning control systems—Review and outlook,” IEEE Trans. Automat. Contr., vol. 15, pp. 210–221, Apr. 1970.
4. K. S. Fu, Ed., Pattern Recognition and Machine Learning. New York: Plenum, 1971.
5. M. Iosifescu and R. Theodorescu, Random Processes and Learning. New York: Springer, 1969.
6. R. D. Luce, Individual Choice Behavior. New York: Wiley, 1959.
7. R. D. Luce and H. Raiffa, Games and Decisions. New York: Wiley, 1957.
8. J. McKinsey, Introduction to the Theory of Games. New York: McGraw-Hill, 1952.
9. J. M. Mendel and K. S. Fu, Eds., Adaptive, Learning and Pattern Recognition Systems. New York: Academic, 1970.
10. J. M. Mendel, “Reinforcement learning models and their applications to control problems,” Learning Systems—A Symposium of the 1973 Joint Automatic Control Conf. (ASME publication), pp. 3–18, 1973.
11. M. F. Norman, Markov Processes and Learning Models. New York: Academic, 1972.
12. A. Paz, Introduction to Probabilistic Automata. New York: Academic, 1971.
13. J. Sklansky, “Learning systems for automatic control,” IEEE Trans. Automat. Contr., vol. AC-11, pp. 6-19, Jan. 1966.
14. G. N. Saridis, “On-line learning control algorithms,” Learning Systems—A Symposium of the 1973 Joint Automatic Control Conf. (ASME publication), pp. 19–52, 1973.
15. Ya. Z. Tsypkin, Adaptation and Learning in Automatic Systems. New York: Academic, 1971.
16. J. Von Neumann and O. Morgenstern, Theory of Games and Economic Behavior. Princeton, N.J.: Princeton Univ., 1947.
17. S. L. Ginzburg and M. L. Tsetlin, “Some examples of simulation of the collective behavior of automata,” Probl. Peredachi Informatsii, vol. 1, no. 2, pp. 54–62, 1965.
18. V. Yu Krylov and M. L. Tsetlin, “Games between automata,” Avtomat. Telemekh., vol. 24, pp. 975–987, July 1963.
19. L. E. Radyuk and A. F. Terpugov, “Effectiveness of applying automata with linear tactic in signal detection systems,” Avtomat. Telemekh., vol. 32, pp. 99–107, Apr. 1971.
20. V. L. Stefanyuk, “Example of a problem in the joint behavior of two automata,” Avtomat. Telemekh., vol. 24, pp. 781–784, June 1963.
21. N. L. Stefanyuk and M. L. Tsetlin, “Power regulation in a group of radio stations,” Probl. Peredachi Informatsii, vol. 3, no. 4, pp. 49–57, 1967.
22. M. L. Tsetlin, “On the behavior of finite automata in random media,” Avtomat. Telemekh., vol. 22, pp. 1345–1354, Oct. 1961.
23. H. Tsuji, M. Mizumoto, J. Toyoda, and K. Tanaka, “An automaton in the non-stationary random environment,” Inform. Sci., vol. 6, no. 2, pp. 123–142, Apr. 1973.
24. E. M. Vaisbord, “Game of two automata with differing memory depths,” Avtomat. Telemekh., vol. 29, pp. 91–102, Mar. 1968.
25. —, “Game of many automata with various depths of memory,” Avtomat. Telemekh., vol. 29, pp. 52–58, Dec. 1968.
26. V. I. Varshavskii, M. V. Meleshina, and M. L. Tsetlin, “Behavior of automata in periodic random media and the problem of synchronization in the presence of noise,” Probl. Peredachy Informatsii, vol. 1, no. 1, pp. 65–71, 1965.
27. —, “Priority organization in queuing systems using a model of collective behavior,” Probl. Peredachi Informatsii, vol. 4, no. 1, pp. 73–76, 1968.
28. V. I. Varshavskii, “Collective behavior and control problems,” in Machine Intelligence, 3, D. Michie, Ed. Edinburgh: Edinburgh Univ., 1968.
29. —, “The organization of interaction in collectives of automata,” Machine Intelligence, 4, B. Meltzer and D. Michie, Eds. Edinburgh: Edinburgh Univ., 1969.
30. —, “Some effects in the collective behavior of automata,” Machine Intelligence, 7, B. Meltzer and D. Michie, Eds. Edinburgh: Edinburgh Univ., 1972.
31. —, “Automata games and control problems,” presented at the 5th World Congr. IFAC, Paris, France, June 12–17, 1972.
32. B. Chandrasekaran and D. W. C. Shen, “On expediency and convergence in variable-structure automata,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-4, pp. 52–60, Mar. 1968.
33. —, “Adaptation of stochastic automata in nonstationary environments,” Proc. NEC, vol. 23, pp. 39–44, 1967.
34. —, “Stochastic automata games,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-5, pp. 145–149, Apr. 1969.
35. L. D. Cockrell and K. S. Fu, “On search techniques in switching environment,” Proc. 9th Symposium Adaptive Processes, Austin, Tex., Dec. 1970.
36. T. M. Cover and M. E. Hellman, “The two-armed bandit problem with time-invariant finite memory,” IEEE Trans. Inform. Theory, vol. IT-16, pp. 185–195, Mar. 1970.
37. K. S. Fu and G. J. McMurtry, “A study of stochastic automata as models of adaptive and learning controllers,” Purdue Univ., Lafayette, Ind., Tech. Rep., TR-EE 65–8, 1965.
38. K. S. Fu and R. W. McLaren, “An application of stochastic automata to the synthesis of learning systems,” Purdue Univ., Lafayette, Ind., Tech. Rep. TR-EE 65–17, 1965.
39. K. S. Fu, “Stochastic automata, stochastic languages and pattern recognition,” J. Cybern., vol. 1, no. 3, pp. 31–49, July-Sept. 1971.
40. K. S. Fu and T. J. Li, “Formulation of learning automata and automata games,” Inform. Sci., vol. 1, no. 3, pp. 237–256, July 1969.
41. —, “On stochastic automata and languages,” Inform. Sci., vol. 1, no. 4, pp. 403–419, Oct. 1969.
42. K. S. Fu, “Stochastic automata as models of learning systems,” Computer and Information Sciences II, J. T. Tou, Ed. New York: Academic, 1967.
43. R. M. Glorioso, G. R. Grueneich, and J. C. Dunn, “Self organization and adaptive routing for communication networks,” 1969 EASCON Rec., pp. 243–250.
44. R. M. Glorioso and G. R. Grueneich, “A training algorithm for systems described by stochastic transition matrices,” IEEE Trans. Syst., Man, Cybern., vol. SMC-1, pp. 86–87, Jan. 1971.
45. B. R. Gaines, “Stochastic computing systems,” in Advances in Information Sciences 2. New York: Plenum, 1969, pp. 37–172.
46. R. A. Jarvis, “Adaptive global search in a time-invariant environment using a probabilistic automaton,” Proc. IREE, Australia, pp. 210–226, 1969.
47. —, “Adaptive global search in a time-variant environment using a probabilistic automaton with pattern recognition supervision,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-6, pp. 209–217, July 1970.
48. S. Lakshmivarahan and M. A. L. Thathachar, “Optimal nonlinear reinforcement schemes for stochastic automata,” Inform. Sci., vol. 4, pp. 121–128, 1972.
49. —, “Bayesian learning and reinforcement schemes for stochastic automata,” Proc. Int. Conf. Cybernetics and Society, Washington, D.C., Oct. 1972.
50. —, “Absolutely expedient learning algorithms for stochastic automata,” IEEE Trans. Syst. Man, Cybern., vol. SMC-3, pp. 281–286, May 1973.
51. S. Lakshmivarahan, “Learning algorithms for stochastic automata,” Ph.D. dissertation, Dep. Elec. Eng., Indian Institute of Science, Bangalore, India, Jan. 1973.
52. S. Lakshmivarahan and M. A. L. Thathachar, “Hypothesis testing using variable-structure stochastic automata,” in 1973 IEEE Decision and Control Conf., Preprints, San Diego, Calif.
53. G. Langholtz, “Behavior of automata in a nonstationary environment,” Electron. Lett., vol. 7, no. 12, pp. 348–349, June 17, 1971.
54. A. A. Lorentz and V. I. Lenin, Eds., Probabilistic Automata and Their Applications (in Russian). Riga, Latvia, USSR: Zinatne, 1971.
55. T. J. Li and K. S. Fu, “Automata games, stochastic automata and formal languages,” Purdue Univ., Lafayette, Ind., Tech. Rep. TR-EE69-1, Jan. 1969.
56. R. W. McLaren, “A stochastic automaton model for synthesis of learning systems,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-2, pp. 109–114, Dec. 1966.
57. —, “A stochastic automaton model for a class of learning controllers,” in 1967 Joint Automatic Control Conf., Preprints.
58. G. J. McMurtry and K. S. Fu, “A variable-structure automaton used as a multimodal search technique,” IEEE Trans. Automat. Contr., vol. AC-11, pp. 379–387, July 1966.
59. L. G. Mason, “Self optimizing allocation systems,” Ph.D. dissertation, University of Saskatchewan, Saskatoon, Sask., Canada, 1972.
60. —, “An optimal learning algorithm for S-model environments,” IEEE Trans. Automat. Contr., vol. AC-18, pp. 493–496, Oct. 1973.
61. K. S. Narendra and R. Viswanathan, “A two-level system of stochastic automata for periodic random environments,” IEEE Trans. Syst., Man, Cybern., vol. SMC-2, pp. 285–289, Apr. 1972.
62. —, “Learning models using stochastic automata,” in Proc. 1972 Int. Conf. Cybernetics and Society, Washington, D.C., Oct. 9–12.
63. M. F. Norman, “On linear models with two absorbing barriers,” J. Math. Psychol., vol. 5, pp. 225–241, 1968.
64. —, “Some convergence theorems for stochastic learning models with distance diminishing operators,” J. Math. Psychol., vol. 5, pp. 61–101, 1968.
65. K. S. Narendra, S. S. Tripathi, and L. G. Mason, “Application of learning automata to telephone traffic routing problems,” Becton Center, Yale Univ., New Haven, Conn., Tech. Rep. CT-60, Jan. 1974.
66. J. S. Riordon, “Optimal feedback characteristics from stochastic automaton models,” IEEE Trans. Automat. Contr., vol. AC-14, pp. 89–92, Feb. 1969.
67. —, “An adaptive automaton controller for discrete-time Markov processes,” Automatica (IFAC Journal), vol. 5, no. 6, pp. 721–730, Nov. 1969.
68. I. J. Shapiro and K. S. Narendra, “Use of stochastic automata for parameter self-optimization with multimodal performance criteria,” IEEE Trans. Syst. Sci. Cybern., vol. SSC-5, pp. 352–360, Oct. 1969.
69. I. J. Shapiro, “The use of stochastic automata in adaptive control,” Ph.D. dissertation, Dep. Eng. and Applied Sci., Yale Univ., New Haven, Conn., 1969.
70. Y. Sawaragi and N. Baba, “A consideration on the learning behavior of variable structure stochastic automata,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 644–647, Nov. 1973.
71. Ya. Z. Tsypkin and A. S. Poznyak, “Finite learning automata,” Eng. Cybern., vol. 10, pp. 478–490, May-June 1972.
72. V. I. Varshavskii and I. P. Vorontsova, “On the behavior of stochastic automata with variable structure,” Avtomat. Telemekh., vol. 24, pp. 353–360, Mar. 1963.
73. I. P. Vorontsova, “Algorithms for changing stochastic automata transition probabilities,” Probl. Peredachi Informatsii, vol. 1, no. 3, pp. 122–126, 1965.
74. R. Viswanathan and K. S. Narendra, “Stochastic automata models with applications to learning systems,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, Jan. 1973.
75. —, “A note on the linear reinforcement scheme for variable-structure structure stochastic automata,” IEEE Trans. Syst., Man, Cybern., vol. SMC-2, pp. 292–294, Apr. 1972.
76. R. Viswanathan, “Learning automata: models and applications,” Ph.D. dissertation, Dep. Eng. and Applied Sci., Yale Univ., New Haven, Conn., 1972.
77. R. Viswanathan and K. S. Narendra, “Comparison of expedient and optimal reinforcement schemes for learning systems,” J. Cybern., vol. 2, no. 1, pp. 21–37, 1972.
78. —, “Competitive and cooperative games of variable-structure structure stochastic automata,” in Joint Automatic Control Conf., Preprints, Aug. 1972; also available as Becton Center, Yale Univ., New Haven, Conn., Tech. Rep. CT-44, Nov. 1971.
79. —, “Application of stochastic automata models to learning systems with multimodal performance criteria,” Becton Center, Yale Univ., New Haven, Conn., Tech. Rep. CT-40, June 1971.
80. —, “Expedient and optimal variable-structure stochastic automata,” Dunham Lab., Yale Univ., New Haven, Conn., Tech. Rep. CT-31, Apr. 1970.
81. —, “Simulation studies of stochastic automata models, Part I—Reinforcement schemes,” Becton Center, Yale Univ., New Haven, Conn., Tech. Rep. CT-45, Dec. 1971.
82. I. H. Witten, “Comments on ‘Use of stochastic automata for parameter self-optimization with multimodal performance criteria,’” IEEE Trans. Syst., Man, Cybern., vol. SMC-2, Apr. 1972.
83. —, “Finite-time performance of some two-armed bandit controllers,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 194–197, Mar. 1973.