Connectionist models of reinforcement, imitation, and instruction in learning to solve complex problems

IEEE Transactions on Autonomous Mental Development

Volumn 1, Issue 2, 2009, Pages 110-121

  Dandurand, Frédéric ^b,c   Shultz, Thomas R ^a,b  

^a Université Laval   (Canada)

^b MCGILL UNIVERSITY   (Canada)

^c CNRS   (France)

Author keywords

Cognitive science; Learning systems; Neural networks; Problem solving

Indexed keywords

COGNITIVE SCIENCE; COMPLEX PROBLEMS; COMPLEX SOLUTION; COMPUTATIONAL MODEL; CONNECTIONIST MODELS; HUMAN PERFORMANCE; LEARNING BY IMITATION; SELECTION CRITERIA;

COGNITIVE SYSTEMS; EDUCATION; KNOWLEDGE BASED SYSTEMS; NEURAL NETWORKS; REINFORCEMENT;

PROBLEM SOLVING;

EID: 77955678386     PISSN: 19430604     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAMD.2009.2031234     Document Type: Article

References (43)

1. J. H. Holland, K. J. Holyoak, R. E. Nisbett, and P. Thagard, Induction -Processes of Inference, Learning and Discovery. Cambridge, MA: MIT Press, 1986.
2. K. J. Holyoak and P. Thagard, Mental Leaps-Analogy in Creative Thought. Cambridge, MA: MIT Press, 1996.
3. A. Newell, Unified Theories of Cognition. Cambridge, MA: Harvard University Press, 1990.
4. G. Polya, How to Solve It, 2nd ed. Princeton, NJ: Princeton University Press, 1957.
5. J. R. Anderson, D. Bothell, M. D. Byrne, D. Douglass, C. Lebiere, and Y. Qin, "An integrated theory of mind," Psychol. Rev., vol.111, pp. 1036-1060, 2004.
6. G. Gunzelmann and J. R. Anderson, "Problem solving: Increased planning with practice," Cogn. Syst. Res., vol.4, pp. 57-76, 2003.
7. W.-T. Fu and J. R. Anderson, "From recurrent choice to skill learning: A reinforcement-learning model," J. Exper. Psychol.: General, vol.135, pp. 184-206, 2006.
8. S. Nason and J. E. Laird, "Soar-RL: Integrating reinforcement learning with Soar," in Proc. Sixth Int. Conf. Cogn. Modeling , Mahwah, NJ, 2004, pp. 208-213, Erlbaum.
9. A. Newell and H. A. Simon, Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall, 1972.
10. G. Katona, Organizing and Memorizing. New York: Columbia University Press, 1940.
11. M. Carpenter, J. Call, and M. Tomasello, "Understanding "prior intentions" enables two-year-olds to imitatively learn a complex task," Child Dev., vol.75, pp. 1431-1441, 2002.
12. R. H. Cuijpers, H. T. Schie, M. Koppen, W. Erlhagen, and H. Bekkering, "Goals and means in action observation: A computational approach," Neural Netw., vol.19, pp. 311-322, 2006.
13. F. Dandurand, M. Bowen, and T. R. Shultz, "Learning by imitation, reinforcement and verbal rules in problem solving tasks," in Proc. Third Int. Conf. Dev. Learning: Dev. Social Brains, J. Triesch and T. Jebara, Eds., La Jolla, CA, 2004, pp. 88-95, University of California, San Diego, Institute for Neural Computation.
14. D. C. Noelle and G. W. Cottrell, "A connectionist model of instruction following," in Proc. 17th Annu. Conf. Cogn. Sci. Soc., J. D. Moore and J. F. Lehman, Eds., Hillsdale, NJ, 1995, pp. 369-374, Lawrence Erlbaum.
15. G. Biele, J. Rieskamp, and R. Gonzalez, "Computational models for the combination of advice and individual learning," Cogn. Sci., vol.33, pp. 206-242, 2009.
16. G. B. Kaplan and C. Güzelis, "Hopfield networks for solving Tower of Hanoi problems," ARI: An Interdiscipl. J. Phys. Eng. Sci., vol.52, pp. 23-29, 2001.
17. R. W. Parks and J. Cardoso, "Parallel distributed processing and executive functioning: Tower of Hanoi neural network model in healthy controls and left frontal lobe patients," Int. J. Neurosci., vol.89, pp. 217-240, 1997.
18. R. W. Parks, D. S. Levine, and D. L. Long, Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neuroscience. Cambridge, MA: MIT Press, 1998, p. 428.
19. H. A. Simon, "The structure of ill-structured problems," Artif. Intell., vol.4, pp. 181-202, 1973.
20. M. L. Simmel, "The coin problem: A study in thinking," Amer. J. Psychol., vol.66, pp. 229-241, 1953.
21. L. Halbeisen and N. Hungerbuhler, "The general counterfeit coin problem," Discr. Math., vol.147, pp. 139-150, 1995.
22. F. Dandurand, T. R. Shultz, and K. H. Onishi, "Strategies, heuristics and biases in complex problem solving," in Proc. Twenty-Ninth Meet. Cogn. Sci. Soc. (CogSci 2007), New York, 2007, pp. 917-922, Lawrence Erlbaum Associates.
23. N. Chater and P. Vitányi, "Simplicity: A unifying principle in cognitive science?," Trends Cogn. Sci., vol.7, pp. 19-22, 2003.
24. J. Freyd and B. Tversky, "Force of symmetry in form perception," Amer. J. Psychol., vol.97, pp. 109-126, 1984.
25. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2. Los Altos, CA: Morgan Kaufmann, 1990, pp. 524-532.
26. T. R. Shultz, Computational Developmental Psychology. Cambridge, MA: MIT Press, 2003.
27. S. Baluja and S. E. Fahlman, Reducing Network Depth in the Cascade-Correlation Carnegie Mellon University. Pittsburgh, PA, CMU-CS-94-209, 1994.
28. F. Dandurand, V. Berthiaume, and T. R. Shultz, "A systematic comparison of flat and standard Cascade-Correlation using a student-teacher network approximation task," Connect. Sci., vol.19, pp. 223-244, 2007.
29. S. E. Fahlman, "Faster-learning variations on back-propagation: An empirical study," in Proc. 1988 Connect. Models Summer School, T. J. Sejnowski, G. E. Hinton, and D. S. Touretzky, Eds., San Mateo, CA, 1988, Morgan Kaufmann.
30. T. R. Shultz and F. Rivest, "Knowledge-based cascade-correlation: Using knowledge to speed learning," Connect. Sci., vol.13, pp. 43-72, 2001.
31. J. P. Thivierge, F. Dandurand, and T. R. Shultz, "Transferring domain rules in a constructive network: Introducing RBCC," in Proc. 2004 IEEE Int. Joint Conf. Neural Netw. (IJCNN '04), 2004, vol.2, pp. 1403-1408, IEEE.
32. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
33. F. Dandurand, T. R. Shultz, and F. Rivest, "Complex problem solving with reinforcement learning," in Proc. Sixth IEEE Int. Conf. Dev. Learning (ICDL-2007), 2007, pp. 157-162, IEEE.
34. R. E. Suri and W. Schultz, "A neural network model with dopaminelike reinforcement signal that learns a spatial delayed response task," Neuroscience, vol.91, pp. 871-890, 1999.
35. J. C. Houk, J. L. Adams, and A. G. Barto, "A model of how the basal ganglia generate and use neural signals that predict reinforcement," in Models of Information Processing in the Basal Ganglia, J. C. Houk, J. L. Davis, and D. G. Beiser, Eds. Cambridge, MA: MIT Press, 1995, pp. 249-270.
36. K. Samejima, Y. Ueda, K. Doya, and M. Kimura, "Representation of action-specific reward values in the striatum," Science, vol.310, pp. 1337-1340, 2005.
37. N. Cowan, "The magical number 4 in short-term memory: A reconsideration of mental storage capacity," Behav. Brain Sci., vol.24, pp. 87-125, 2000.
38. G. A. Miller, "The magical number seven, plus or minus two: Some limits on our capacity for processing information," Psychologic. Rev., vol.63, pp. 81-97, 1956.
39. R. V. Hogg and A. T. Craig, Introduction to Mathematical Statistics. Upper Saddle River, NJ: Prentice-Hall, 1995.
40. T. R. Shultz, F. Rivest, L. Egri, J.-P. Thivierge, and F. Dandurand, "Could knowledge-based neural learning be useful in developmental robotics? The case of KBCC," Int. J. Human. Robot., vol.4, pp. 245-279, 2007.
41. D. H. Holding, The Psychology of Chess Skill. New York: Lawrence Erlbaum , 1985.
42. D. H. Holding, "Theories of chess skill," Psychologic. Res., vol.54, pp. 10-16, 1992.
43. J. Baxter, A. Tridgell, and L. Weaver, "TDLeaf(lambda): Combining temporal difference learning with game-tree search," in Proc. Ninth Australian Conf. Neural Netw., 1998, pp. 168-172.