The metacognitive loop I: Enhancing reinforcement learning with metacognitive monitoring and control for improved perturbation tolerance

Journal of Experimental and Theoretical Artificial Intelligence

Volumn 18, Issue 3, 2006, Pages 387-411

  Anderson, Michael L ^a,d   Oates, Tim ^a,c   Chong, Waiyian ^b   Perlis, Don ^a,b  

^a UNIVERSITY OF MARYLAND   (United States)

^b University of Maryland Baltimore County ^*  (United States)

^c University of Marylandw Baltimore County   (United States)

^d FRANKLIN AND MARSHALL COLLEGE   (United States)

Author keywords

Autonomous agents; Commonsense reasoning; Learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; COGNITIVE SYSTEMS; DECISION MAKING; LEARNING SYSTEMS; UNCERTAIN SYSTEMS;

COMMONSENSE REASONING; INTELLIGENT SYSTEMS; METACOGNITIVE MONITORING; PERTURBATION TOLERANCE;

AUTONOMOUS AGENTS;

EID: 33748702264     PISSN: 0952813X     EISSN: 13623079     Source Type: Journal    
DOI: 10.1080/09528130600926066     Document Type: Article

References (34)

1. J.F. Allen, L.K. Schubert, G. Ferguson, P. Heeman, C.H. Hwang, T. Kato, M. Light, N. Martin, B. Miller, M. Poesio and D.R. Traum, "The TRAINS project: a case study in building a conversational planning agent", J. Exp. Theor. Artifi. Int., 8, pp. 7-48, 1995.
2. E. Amir, "Toward a formalization of elaboration tolerance: Adding and deleting axioms", in Frontiers of Belief Revision, M. Williams and H. Rott, Eds, Dordrecht: Kluwer, 2000, pp. 147-162.
3. M.L. Anderson, D. Josyula and D. Perlis, "Talking to computers", in Proceedings of the Workshop on Mixed Initiative Intelligent Systems, Acapulco, Mexico: IJCAI-03, 2003, pp. 1-8.
4. M.L. Anderson and D. Perlis, "Logic, self-awareness and self-improvement: The metacognitive loop and the problem of brittleness", J. Logic Comput., 15, pp. 21-40, 2005.
5. D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming, Nashua, NH: Athena Scientific, 1996.
6. J. Elgot-Drapkin and D. Perlis, "Reasoning situated in time I: Basic concepts", J. Exp. Theor. Artif. Int., 2, pp. 75-98, 1990.
7. J. Elgot-Drapkin, S. Kraus, M. Miller, M. Nirkhe and D. Perlis, "Active logics: A unified formal approach to episodic reasoning", Technical Report UMIACS-TR # 99-65, CS-TR # 4072, College Park, MD: University of Maryland, College Park, 1993.
8. J. Elgot-Drapkin, "Step-logic: Reasoning Situated in Time". PhD thesis, Department of Computer Science, University of Maryland, College Park, 1988.
9. K. Hennacy, N. Swamy and D. Perlis, "RGL study in a hybrid real-time system", in Proceedings of IASTED NCI 2003, Mexico, Cancun, 2003, pp. 203-208.
10. D. Josyula, M.L. Anderson and D. Perlis, "Towards domain-independent, task-oriented, conversational adequacy", Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003, pp. 1637-1638.
11. H.H. Kendler and T.S. Kendler, "Vertical and horizontal processes in problem solving". Psychol. Rev., 69, pp. 1-16, 1962.
12. H.H. Kendler and T.S. Kendler, "Reversal-shift behavior: Some basic issues", Psychol. Bull., 72, pp. 229-232, 1969.
13. S. Kraus, M. Nirkhe and D. Perlis, "Deadline-coupled real-time planning", Proceedings of 1990 DARPA workshop on Innovative Approaches to Planning, Scheduling and Control, 1990, pp. 100-108.
14. J. McCarthy, "Elaboration tolerance", in Proceedings of the Fourth Symposium on Logical Formalizations of Commonsense Reasoning, 1998.
15. M. Miller and D. Perlis, "Presentations and this and that: Logic in action", in Proceedings of the 15th Annual Conference of the Cognitive Science Society, 1993.
16. A.W. Moore and C.G. Atkeson, "Prioritized sweeping: Reinforcement learning with less data and less time", Mach. Learn., 13, pp. 103-130, 1993.
17. T.O. Nelson and J. Dunlosky, "Norms of paired-associate recall during multitrial learning of Swahili-english translation equivalents", Memory, 2, pp. 325-335, 1994.
18. T.O. Nelson, J. Dunlosky, A. Graf and L. Narens, "Utilization of metacognitive judgments in the allocation of study during multitrial learning". Psychol. Sci., 4, pp. 207-213, 1994.
19. T.O. Nelson, "Consciousness and metacognition", Am. Psychol., 51, pp. 102-116, 1996.
20. M. Nirkhe, S. Kraus, M. Miller and D. Perlis, "How to (plan to) meet a deadline between now and then", J. Logic Comput., 7, pp. 109-156, 1997.
21. M. Nirkhe, "Time-situated reasoning within tight deadlines and realistic space and computation bounds", PhD thesis, Department of Computer Science, University of Maryland, College Park, 1994.
22. D. Perlis, K. Purang and C. Andersen, "Conversational adequacy: mistakes are the essence", International Journal of Human-Computer Studies, 48, pp. 553-575, 1998.
23. D. Perlis, "On the consistency of commonsense reasoning", Comput. Intelli., 2, pp. 180-190, 1986.
24. G. Priest, R. Routley, J. Norman, "Paraconsistent Logic: Essays on the Inconsistent", Philosophia Verlag, Munchen, 1989.
25. G. Priest, "Paraconsistent logic", in Handbook of Philosophical Logic, 2nd ed., D. Gabbay and F. Guenther, Eds, Dordrecht: Kluwer Academic Publishers, 2002, pp. 287-393.
26. K. Purang, "Systems that detect and repair their own mistakes", PhD thesis, Department of Computer Science, University of Maryland, College Park, 2001.
27. R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1995.
28. I. Szita, B. Takács and A. Lörincz, "ε-MDPs: Learning in varying environments", Journal of Machine Learning Research, 3, pp. 145-174, 2002.
29. D. Traum and C. Andersen, "Representations of dialogue state for domain and task independent metadialogue", Proceedings of the IJCAI99 workshop: Knowledge And Reasoning in Practical Dialogue Systems, pp. 113-120, 1999.
30. D. Traum, C. Andersen, Y. Chong, D. Josyula, M. O'Donovan-Anderson, Y. Okamoto, K. Purang and D. Perlis, "Representations of dialogue state for domain and task independent meta-dialogue", Electronic Transactions on Artificial Intelligence, 3, pp. 125-152, 1999.
31. K. Tsumori and S. Ozawa, "Incremental learning in dynamic environments using neural network with long-term memory", Proceedings of the International Conference on Neural Networks, pp. 2583-2588, 2003.
32. C.J.C.H. Watkins and P. Dayan, "Q-learning", Mach. Learn., 8, pp. 279-292, 1992.
33. C.J.C.H. Watkins, "Learning from Delayed Rewards", PhD thesis, Cambridge University, Cambridge, England, 1989.
34. M.A. Wiering, "Reinforcement learning in dynamic environments using instantiated information", Proceedings of the Eighth International Conference on Machine Learning, pp. 585-592, 2001.