Hybrid architecture for situated learning of reactive sequential decision making

Applied Intelligence

Volumn 11, Issue 1, 1999, Pages 109-127

  Sun, Ron ^a   Peterson, Todd ^b   Merrill, Edward ^b  

^a NEC LABORATORIES AMERICA   (United States)

^b UNIVERSITY OF ALABAMA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COGNITIVE SYSTEMS; COMPUTER ARCHITECTURE; DECISION MAKING; NEURAL NETWORKS;

COGNITIVE MODELING; HYBRID MODELS; REINFORCEMENT LEARNING;

LEARNING SYSTEMS;

EID: 0033165135     PISSN: 0924669X     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1008332731824     Document Type: Article

References (109)

1. Sutton and Barto (1981).
2. A. Schultz, "Using a genetic algorithm to learn strategies for collisionavoidance and local navigation," in Proc. of 7th International Symp. on Unmanned Untethered Submersible Technology, University of New Hampshire, Durham, pp. 213-225, 1991.
3. R. Sun and T. Peterson, "A hybrid learning model of reactive sequential decision making," in The Working Notes of The IJ-CAI Workshop on Connectionist-Symbolic Integration, edited by R. Sun and F. Alexandre, 1995.
4. P. Rosenbloom, J. Laird, and A. Newell, The SOAR Papers: Research on Integrated Intelligence, MIT Press: Cambridge, MA, 1993.
5. R. Nosofsky, T. Palmeri, and S. McKinley, "Rule-plus-exception model of classification learning," Psychological Review, vol. 101, no. 1, pp. 53-79, 1994.
6. G. Widmer and M. Kubat, "Learning in the presence of concept drift and hidden context," Machine Learning, vol. 23, no. 1, 1996.
7. R. Sutton, "Learning to predict by the methods of temporal difference," Machine Learning, vol. 3, pp. 9-44, 1988.
8. R. Sutton, "Integrated architectures for learning, planning, and reacting based on approximating dynamic programming," in Proc. of Seventh International Conference on Machine Learning, Morgan Kaufmann: San Mateo, CA, 1990.
9. S. Mahadevan and J. Connell, "Automatic programming of behavior-based robot with reinforcement learning," vol. 55, pp. 311-365, 1992.
10. L. Lin, "Self-improving reactive agents based on reinforcement learning, planning, and teaching," Machine Learning, vol. 8, pp. 293-321, 1992.
11. L. Kaelbling, M. Littman, and A. Moore, "Reinforcement learning: A survey," Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
12. T. Dietterich, Hierarchical reinforcement learning with MAXQ value function decomposition, 1997, ftp://www.cs.orst.edu.
13. J. Holland, N. Nisbitt, P. Thagard, and J. Holyoak, Induction: A Theory of Learning and Development, MIT Press: Cambridge, MA, 1986.
14. J. Grefenstette, "The evolution of strategies for multiagent environments," Adaptive Behavior, vol. 1, no. 1, pp. 65-90, 1992.
15. L. Meeden, "An incremental approach to developing intelligent neural network controllers for robots," Adaptive Behavior, 1995.
16. R. Michalski, "A theory and methodology of inductive learning," Artificial Intelligence, vol. 20, pp. 111-161, 1983.
17. R. Quinlan, "Inductive learning of decision trees," Machine Learning, vol. 1, pp. 81-106, 1986.
18. R. Quinlan, "Learning logical definition from relations," Machine Learning, vol. 5, pp. 239-266, 1990.
19. T. Mitchell, "Generalization as search," Artificial Intelligence, vol. 18, pp. 203-226, 1982.
20. M. Lebowitz, "Experiments with incremental concept formation: UNIMEM," Machine Learning, vol. 2, pp. 103-138, 1987.
21. D. Fisher, "Knowledge acquisition via incremental conceptual clustering," Machine Learning, vol. 2, pp. 139-172, 1987.
22. P. Utgoff, "Incremental induction of decision trees," Machine Learning, vol. 4, pp. 161-186, 1989.
23. H. Hirsh, "Generalizing version spaces," Machine Learning, vol. 17, pp. 5-46, 1994.
24. P. Clark and T. Niblett, "The CN2 induction algorithm," Machine Learning, vol. 3, pp. 261-284, 1989.
25. R. Sun and T. Peterson, "Some experiments with a hybrid model for learning sequential decision making," Information Sciences, vol. 14, pp. 83-107, 1998.
26. R. Sun and T. Peterson, "Autonomous learning of sequential tasks: Experiments and analyses," IEEE Transaction on Neural Networks, vol. 9, no. 6, pp. 1217-1234, 1998.
27. J. Anderson, "Acquisition of cognitive skill," Psychological Review, vol. 89, pp. 369-406, 1982.
28. J. Anderson, Rules of the Mind, Lawrence Erlbaum Associates: Hillsdale, NJ, 1993.
29. J. Anderson and C. Lebiere, The Atomic Components of Thought, Lawrence Erlbaum Associates: Mahwah, NJ, 1998.
30. F. Keil, Concepts, Kinds, and Cognitive Development, MIT Press: Cambridge, MA, 1989.
31. A. Damasio et al., "Neural regionalization of knowledge access," Cold Spring Harbor Symp. on Quantitative Biology, CSHL Press, vol. LV, 1990.
32. R. Sun, Integrating Rules and Connectionism for Robust Commonsense Reasoning, John Wiley and Sons: New York, NY, 1994.
33. J. Anderson, The Architecture of Cognition, Harvard University Press: Cambridge, MA, 1983.
34. P. Fitts and M. Posner, Human Performance, Brooks/Cole, Monterey, CA, 1967.
35. H. Dreyfus and S. Dreyfus, Mind Over Machine, The Free Press: New York, NY, 1987.
36. P. Smolensky, "On the proper treatment of connectionism," Behavioral and Brain Sciences, vol. 11, no. 1, pp. 1-74, 1988.
37. W. James, The Principles of Psychology, Dover: New York, 1890.
38. R. Sun, "Learning, action, and consciousness: A hybrid approach towards modeling consciousness," Neural Networks, special issue on consciousness, vol. 10, no. 7, pp. 1317-1331, 1997.
39. D. Willingham, M. Nissen, and P. Bullemer, "On the development of procedural knowledge," Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 15, pp. 1047-1060, 1989.
40. R. Sun, "Robust reasoning: Integrating rule-based and similarity-based reasoning," Artificial Intelligence, vol. 75, no. 2, pp. 241-296, 1995.
41. J. Hendler, "Marker passing and microfeature," in Proc. 10th IJCAI, Morgan Kaufmann: San Mateo, CA, 1987, pp. 151-154.
42. J. Gelfand, D. Handelman, and S. Lane, "Integrating knowledge-based systems and neural networks for robotic skill acquisition," in Proc. IJCAI, Morgan kaufmann: San Mateo, CA, 1989, pp. 193-198.
43. W. Schneider and W. Oliver, "An instructable connectionist/control architecture," in Architectures for Intelligence, edited by K. VanLehn, Erlbaum: Hillsdale, NJ, 1991.
44. R. Sun, "A Connectionist model for commonsense reasoning incorporating rules and similarities," Knowledge Acquisition, vol. 4, pp. 293-321, 1992.
45. M. Erickson and J. Kruschke, Rules and Examplars in Category Learning, manuscript, 1997.
46. A. Reber, "Implicit learning and tacit knowledge," Journal of Experimental Psychology: General, vol. 118, no. 3, pp. 219-235, 1989.
47. R. Dominowski, How do People Discover Concepts?, manuscript, 1975.
48. D. Medin, W. Wattenmaker, and R. Michalski, "Constraints and preferences in inductive learning: An experimental study of human and machine performance," Cognitive Science, vol. 11, pp. 299-339, 1987.
49. C. Watkins, "Learning with Delayed Rewards," Ph.D. Thesis, Cambridge University, Cambridge, UK, 1989.
50. D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific: Belmont, MA, 1996.
51. R. Parr and S. Russell, "Reinforcement learning with hierarchies of machines," Advances in Neural Information Processing Systems, MIT Press: Cambridge, MA, 1997.
52. D. Precup, R. Sutton, and S. Singh, "Multi-time models for temporary abstract planning," Advances in Neural Information Processing Systems 10, MIT Press: Cambridge, MA, 1998.
53. T. Tesauro, "Practical issues in temporal difference learning," Machine Learning, vol. 8, pp. 257-277, 1992.
54. J. Boyan and A. Moore, "Generalization in reinforcement learning: Safely approximating the value function," in Neural Information Processing Systems, edited by J. Tesauro and D. Touretzky, and T. Leen, MIT Press: Cambridge, MA, pp. 369-376, 1995.
55. R. Sun, "On variable binding in connectionist networks," Connection Science, vol. 4, no. 2, pp. 93-124, 1992.
56. L.M. Fu, "Rule learning by searching on adapted nets," in Proc. of AAAI'91, 1991, pp. 590-595.
57. R.C. Lacher, "Expert networks: Paradigmatic conflict, technological rapprochment," Minds and Machines, vol 3, pp. 53-71, 1993.
58. G. Towell and J. Shavlik, "Extracting refined rules from knowledge-based neural networks," Machine Learning, vol. 13, no. 1, pp. 71-101, 1993.
59. R. Michalski, I. Mozetic, J. Hong, and N. Lavrac, "The multipurpose incremental learning system AQ15," in Proc. of AAAI-86, Morgan Kaufmann: San Mateo, CA, 1986, pp. 1041-1045.
60. A. McCallum, "Learning to use selective attention and short-term memory in sequential tasks," in Proc. Conference on Simulation of Adaptive Behavior, MIT Press: Cambridge, MA, 1996, pp. 315-324.
61. L. Breiman, "Bagging predictors," Machine Learning, vol. 24, no. 2, pp. 123-140, 1996.
62. Y. Freund and R. Schapire, "Experiments with a new boosting algorithm," in Proc. of ICML'97, Morgan Kaufmann, San Francisco, CA, 1996, pp. 148-156.
63. R. Brooks, "Intelligence without representation," Artificial Intelligence, vol. 47, pp. 139-160, 1991.
64. P. Tadepalli and T. Dietterich, "Hierarchical explanation-based reinforcement learning," in Proc. International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, 1997, pp. 358-366.
65. D. Gordon, A. Schultz, J. Grefenstette, J. Ballas, and M. Perez, User's Guide to the Navigation and Collision Avoidance Task, Naval Research Lab, Washington, DC, 1994.
66. R. Sun, E. Merrill, and T. Peterson, "A bottom-up model of skill learning," in Proc. of 20th Cognitive Science Society Conference, Lawrence Erlbaum Associates: Mahwah, NJ, pp. 1037-1042, 1998.
67. R. Sun, E. Merrill, and T. Peterson, "Skill learning using a bottom-up hybrid model," in Proc. of The Second European Conference on Cognitive Modeling, Nottingham University Press: Nottingham, UK, pp. 23-29, April 1998.
68. E. Smith and D. Medin, Categories and Concepts, Cambridge, MA: Harvard University Press, 1981.
69. E. Rosch, "Principles of categorization," in Cognition and Categorization, edited by E. Rosch and B. Lloyd, Erlbaum: Hillsdale, NJ, 1978.
70. C. Giraud-Carrier and T. Martinez, "An integrated framework for learning and reasoning," Journal of Artificial Intelligence Research, vol. 3, pp. 147-185, 1995.
71. D. Waltz, "How to build a robot," in Proc. of Conf. on Simulation of Adaptive Behaviors, edited by S. Wilson, MIT Press: Cambridge, MA, 1991.
72. R. Sun, E. Merrill, and T. Peterson, "A bottom-up model of skill learning," in Proc. of 20th Cognitive Science Society Conference, Lawrence Erlbaum Associates: Mahwah, NJ, pp. 1037-1042, 1998.
73. M. Lebowitz, "Experiments with incremental concept formation: UNIMEM," Machine Learning, vol. 2, pp. 103-138, 1987.
74. D. Fisher, "Knowledge acquisition via incremental conceptual clustering," Machine Learning, vol. 2, pp. 139-172, 1987.
75. Stepp and Michalski (1983).
76. W. Shen, "Discovery as autonomous learning from the environment," Machine Learning, vol. 12, pp. 143-165, 1993.
77. E. Wisniewski and D. Medin, "On the interaction of data and theory in concept learning," Cognitive Science, vol. 18, pp. 221-281, 1994.
78. L. Rips, "Similarity, typicality, and categorization," in Similarity and Analogical Reasoning, edited by S. Vosniadou and A. Ortony, Cambridge University Press: New York, NY, 1989.
79. R. Andrews and J. Diederich (Eds.), Proceedings of the NIPS'96 Workshop on Rule Extraction From Trained Artificial Neural Networks, NIPS Foundation, 1996.
80. R. Miikkulainen and M. Dyer, "Natural language processing with modular PDP networks and distributed lexicons," Cognitive Science, vol. 15, no. 3, pp. 343-399, 1991.
81. D. Gordon and D. Subramanian, "A cognitive model of learning to navigate," in Proc. of 18th Cognitive Science Conference, Lawrence Erlbaum: Mahwah, NJ, 1997, pp. 271-276.
82. T. Johnson, J. Zhang, and H. Wang, "A hybrid learning model of abductive reasoning," in Connectionist-Symbolic Integration, edited by R. Sun and F. Alexandre, Lawrence Erlbaum Associates: Mahwah, NJ, 1998.
83. V. Ajjanagadde and L. Shastri, "From simple association to systematic reasoning," Tech. Report MS-CIS-90-05, University of Pennsylvania, Philadelphia, PA, 1990.
84. J. Barnden, "The right of free association: Relative-position encoding for connectionist data structures," in Proc. 10th Conference of Cognitive Science Society, Lawrence Erlbaum Associates: Hillsdale, NJ, pp. 503-509, 1988.
85. J. Kruschke, "ALCOVE: an examples-based connectionist model of category learning," Psychological Review, vol. 99, pp. 22-44, 1992.
86. D. Touretzky and G. Hinton, "Symbols among neurons," in Proc. 9th IJCAI, Morgan Kaufmann, 1987, pp. 238-243.
87. C.L. Giles and M. Gori, "Adaptive processing of sequences and data structures," Lecture Notes in Artificial Intelligence, Springer Verlag, 1998.
88. J. Barnden, "The right of free association: Relative-position encoding for connectionist data structures," in Proc. 10th Conference of Cognitive Science Society, Lawrence Erlbaum Associates: Hillsdale, NJ, pp. 503-509, 1988.
89. E. Gat, "Integrating planning and reacting in a heterogeneous architecture," in Proc. AAAI, Morgan Kaufmann: San Mateo, CA, 1992, pp. 809-815.
90. W. Schneider and W. Oliver, "An instructable connectionist/control architecture," in Architectures for Intelligence, edited by K. VanLehn, Erlbaum: Hillsdale, NJ, 1991.
91. R. Maclin and J. Shavlik, "Incorporating advice into agents that learn from reinforcements," in Proc. of AAAI-94, Morgan Kaufmann: San Meteo, CA, 1994.
92. G. Drescher, Made-up Minds, MIT Press: Cambridge, MA, 1991.
93. D. Broadbent, P. Fitsgerald, and M. Broadbent, "Implicit and explicit knowledge in the control of complex systems," British Journal of Psychology, vol. 77, pp. 33-50, 1986.
94. G. Logan, "Toward an instance theory of automatization," Psychological Review, vol. 95, no. 4, pp. 492-527, 1988.
95. W. Stanley, R. Mathews, R. Buss, and S. Kotler-Cope, "Insight without awareness: On the interaction of verbalization, instruction and practice in a simulated process control task," Quarterly Journal of Experimental Psychology, vol. 41 A, no. 3, pp. 553-577, 1989.
96. D. Gordon and D. Subramanian, "A multistrategy learning scheme for agent knowledge acquisition," Informatica, vol. 17, pp. 331-346, 1993.
97. E. Thorndike, Animal Intelligence, Hafner: Darien, Connecticutt, 1911.
98. K. VanLehn, "Cognitive skill acquisition," in Annual Review of Psychology, edited by J. Spence, J. Darly, and D. Foss, Annual Reviews, Palo Alto, CA, vol. 47, pp. 513-539, 1996.
99. S. Whitehead and D. Ballard, "Learning to perceive and act by trial and error," Artificial Intelligence, vol. 7, pp. 45-83, 1991.
100. J. Schlimmer, "Incremental adjustment of representations for learning," in Proc. of 4th Workshop on Machine Learning, Morgan Kaufmann: San Mateo, CA, pp. 502-507, 1987.
101. P. Agre and D. Chapman, "What are plans for?," in Designing Autonomous Agents, edited by P. Maes, Elsevier: New York, 1990.
102. A. Barto, R. Sutton, and C. Watkins, "Learning and sequential decision-making," in Learning and Computational Neuroscience, edited by M. Gabriel and J. Moors, MIT Press: Cambridge, MA.
103. J. LeDoux, "Brain mechanisms of emotion and emotional learning," Current Opinion in Neurobiology, vol. 2, no. 2, pp. 191-197, 1992
104. R. Shiffrin and W. Schneider, "Controlled and automatic human information processing II," Psychological Review, vol. 84, pp. 127-190, 1977.
105. R. Stepp and R. Michalski, "Conceptual clustering," in Machine Learning, II, edited by R. Michalski et al., Morgan Kaufmann: Los Altos, CA, 1986.
106. R. Sun and L. Bookman (Eds.), Computational Architectures Integrating Neural and Symbolic Processes, Kluwer Academic Publishers: Norwell, MA, 1994.
107. D. Touretzky and G. Hinton, "Symbols among neurons," in Proc. 9th IJCAI, Morgan Kaufmann, 1987, pp. 238-243.
108. K. VanLehn, "Rule acquisiton events in the discovery of problem-solving strategies," Cognitive Science, vol. 15, pp. 1-47, 1991.
109. S. Whitehead and L. Lin, "Reinforcement learning of non-Markov decision processes," Artificial Intelligence, vol. 73, nos. 1-2, pp. 271-306, 1995.