Neuronlike Adaptive Elements That Can Solve Difficult Learning Control Problems

IEEE Transactions on Systems, Man and Cybernetics

Volumn SMC-13, Issue 5, 1983, Pages 834-846

  Barto, Andrew G ^a   Sutton, Richard S ^a   Anderson, Charles W ^a  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; CENTRAL NERVOUS SYSTEM; FEEDBACK SYSTEM; HUMAN; INFORMATION SCIENCE; LEARNING; MATHEMATICAL MODEL; NERVE CELL NETWORK; NERVOUS SYSTEM; NONBIOLOGICAL MODEL; NONHUMAN; REINFORCEMENT; REVIEW;

SYSTEMS SCIENCE AND CYBERNETICS - LEARNING SYSTEMS;

BIOMEDICAL ENGINEERING;

EID: 0020970738     PISSN: 00189472     EISSN: 21682909     Source Type: Journal    
DOI: 10.1109/TSMC.1983.6313077     Document Type: Article

References (42)

1. B. Chandrasekaran, “Natural and social system metaphors for distributed problem solving: Introduction to the issue,” IEEE Trans. Syst., Man., Cybern., vol. SMC-11, pp. 1-5, 1981.
2. V. R. Lesser and D. D. Corkill, “Functionally-accurate, cooperative distributed systems,” IEEE Trans. Syst., Man, Cybern., vol. SMC-11, pp. 81-96, 1981.
3. D. Michie and R. A. Chambers, “BOXES: An experiment in adaptive control,” in Machine Intelligence 2, E. Dale and D. Michie, Eds. Edinburgh: Oliver and Boyd, 1968, pp. 137-152.
4. D. Michie and R. A. Chambers, “‘Boxes’ as a model of pattern-formation,” in Towards a Theoretical Biology, vol. 1, Prolegomena, C. H. Waddington, Ed. Edinburgh: Edinburgh Univ. Press, 1968, pp. 206-215.
5. A. G. Barto, C. W. Anderson, and R. S. Sutton, “Synthesis of nonlinear control surfaces by a layered associative search network,” Biol. Cybern., vol. 43, pp. 175-185, 1982.
6. A. G. Barto and R. S. Sutton, “Landmark learning: An illustration of associative search,” Biol. Cybern., vol. 42, pp. 1-8, 1981.
7. A. G. Barto and R. S. Sutton, “Simulation of anticipatory responses in classical conditioning by a neuron-like adaptive element,” Behavioral Brain Res., vol. 4, pp. 221-235, 1982.
8. A. G. Barto, R. S. Sutton, and P. S. Brouwer, “Associative search network: A reinforcement learning associative memory,” Biol. Cybern., vol. 40, pp. 201-211, 1981.
9. R. S. Sutton and A. G. Barto, “Toward a modern theory of adaptive networks: Expectation and prediction,” Psychol. Rev., vol. 88, pp. 135-171, 1981.
10. R. S. Sutton and A. G. Barto, “An adaptive network that constructs and uses an internal model of its world,” Cognition and Brain Theory, vol. 4, pp. 213-246, 1981.
11. A. H. Klopf, “Brain function and adaptive systems–A heterostatic theory,” Air Force Cambridge Res. Lab. Res. Rep., AFCRL-72-0164, Bedford, MA, 1972. (A summary appears in Proc. Int. Conf. Syst., Man, Cybern., 1974).
12. A. H. Klopf, The Hedonistic Neuron: A Theory of Memory, Learning, and Intelligence. Washington, DC: Hemisphere, 1982.
13. B. Widrow, N. K. Gupta, and S. Maitra, “Punish/reward: learning with a critic in adaptive threshold systems,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 455-465, 1973.
14. B. Widrow and F. W. Smith, “Pattern-recognizing control systems,” in Computer and Information Sciences, J. T. Tow and R. H. Wilcox, Eds. Clever Hume Press, 1964, pp. 288-317.
15. F. Rosenblatt, Principles of Neurodynamics. New York: Spartan, 1962.
16. B. Widrow and M. E. Hoff, “Adaptive switching circuits,” in 1960 WESCON Conv. Record, part IV, 1960, pp. 96-104.
17. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
18. P. R. Cohen and E. A. Feigenbaum, The Handbook of Artificial Intelligence, vol 3. Los Altos, CA: Kauffman, 1982.
19. D. O. Hebb, Organization of Behavior. New York: Wiley, 1949.
20. B. G. Farley and W. A. Clark, “Simulation of self-organizing systems by digital computer,” IRE Trans. Inform. Theory, vol. PGIT-4, pp. 76-84, 1954.
21. M. L. Minsky, “Theory of neural-analog reinforcement systems and its application to the brain-model problem,” Ph.D. dissertation, Princeton Univ., Princeton, NJ, 1954.
22. G. M. Edelman, “Group selection and phasic reentrant signaling: A theory of higher brain function,” in The Mindful Brain: Cortical Organization and the Group-Selective Theory of Higher Brain Function, G. M. Edelman and V. B. Mountcastle, Eds. Cambridge, MA: MIT Press, 1978.
23. M. L. Minsky, “Steps toward artificial intelligence,” Proc. IRE, vol. 49, pp. 8-30, 1961.
24. A. L. Samuel, “Some studies in machine learning using the game of checkers,” IBM J. Res. Develop., vol. 3, pp. 210-229, 1959.
25. J. Doran, “An approach to automatic problem solving,” in Machine Intelligence, vol. 1, E. L. Collins and D. Michie, Eds. Edinburgh: Oliver and Boyd, 1967, pp. 105-123.
26. J. H. Holland, “Adaptive algorithms for discovering and using general patterns in growing knowledge-bases,” Int. J. Policy Anal. Inform. Syst., vol. 4, pp. 217-240, 1980.
27. I. H. Witten, “An adaptive optimal controller for discrete-time Markov environments,” Inform. Contr., vol. 34, pp. 286-295, 1977.
28. T. D. Dietterich and B. G. Buchanan, “The role of the critic in learning systems,” Stanford Univ. Tech. Rep., STAN-CS-81-891, 1981.
29. W. S. McCulloch and W. H. Pitts, “A logical calculus of the ideas immanent in nervous activity,” Bull. Math. Biophys., vol. 5, pp. 115-133, 1943.
30. K. S. Narendra and M. A. L. Thatachar, “Learning automata–A survey,” IEEE Trans. Syst., Man, Cybern., vol. SMC-4, pp. 323-334, 1974.
31. S. Lakshmivarahan, Learning Algorithms Theory and Applications. New York: Springer-Verlag, 1981.
32. R. R. Bush and F. Mosteller, Stochastic Models for Learning. New York: Wiley, 1958.
33. M. L. Tsetlin, Automaton Theory and Modelling of Biological Systems. New York: Academic, 1973.
34. J. A. Albus, Brains, Behavior, and Robotics. Peterborough, NH: BYTE Books, 1981.
35. D. Marr, “A theory of cerebellar cortex,” J. Physiol., vol. 202, pp. 437-470, 1969.
36. G. E. Hinton and J. A. Anderson, Eds., Parallel Models of Associative Memory. Hillsdale, NJ: Erlbaum, 1981.
37. T. Kohonen, Associative Memory: A System Theoretic Approach. Berlin, Germany: Springer, 1977.
38. R. A. Rescorla, Pavlovian Second-Order Conditioning: Studies in Associative Learning. Hillsdale, NJ: Erlbaum, 1980.
39. R. A. Rescorla and A. R. Wagner, “A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement,” in Classical Conditioning II: Current Research and Theory, A. H. Black and W. R. Prokasy, Eds. New York: Appleton-Century-Crofts, 1972.
40. A. Dickinson, Contemporary Animal Learning Theory. Cambridge: Cambridge Univ. Press, 1980.
41. L. Stein and J. D. Belluzzi, “Beyond the reflex arc: A neuronal model of operant conditioning,” in Changing Concepts of the Nervous System. New York: Academic, 1982.
42. R. H. Cannon, Dynamics of Physical Systems. New York: McGraw-Hill, 1967.