Shaping robot behavior using principles from instrumental conditioning

Robotics and Autonomous Systems

Volumn 22, Issue 3-4, 1997, Pages 231-249

  Saksida, Lisa M ^a,c   Raymond, Scott M ^b   Touretzky, David S ^b,c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Carnegie Mellon University   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Autonomous mobile robots; Instrumental learning; Operant conditioning; Reinforcement learning; Shaping

Indexed keywords

COMPUTATION THEORY; COMPUTER SIMULATION; LEARNING ALGORITHMS; MATHEMATICAL MODELS;

INSTRUMENTAL LEARNING; REINFORCEMENT LEARNING;

MOBILE ROBOTS;

EID: 0031336564     PISSN: 09218890     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0921-8890(97)00041-9     Document Type: Article

References (39)

1. M. Asada, S. Noda, S. Tawaratsumida, K. Hosoda, Purposive behavior acquisition for a real robot by vision-based reinforcement learning, Machine Learning 23 (2-3) (1996) 279-303.
2. S.A. Barnett, Modern Ethology, Oxford University Press, Oxford, 1981.
3. D.A. Baxter, D.V. Bounomano, J.L. Raymond, D.G. Cook, F.M. Kuenzi, T.J. Carew, J.H. Byrne, Empirically derived adaptive elements and networks simulate associative learning, in: Neural Network Models of Conditioning and Action, Lawrence Erlbaum, Hillsdale, NJ, 1991, pp. 13-52.
4. B. Blumberg, Action - selection in hamsterdam: Lessons from ethology, in: Proceedings of the Third International Conference on the Simulation of Adaptive Behavior, Brighton, 1994.
5. B.M. Blumberg, P.M. Todd, Pattie Maes, No bad dogs: Ethological lessons for learning in hamsterdam, in: Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior, 1996.
6. K. Breland, M. Breland, The misbehavior of organisms, American Psychologist 16 (1961) 681-684.
7. P.L. Brown, H.M. Jenkins, Auto-shaping of the pigeon's keypeck, Journal of the Experimental Analysis of Behavior 11 (1968) 1-8.
8. T.J. Bussey, J.L. Muir, T.W. Robbins, A novel automated touchscreen procedure for assessing learning in the rat using computer graphic stimuli, Neuroscience Research Communications 15 (2) (1994) 103-109.
9. CCI. The CCI Program. Canine Companions for Independence, Santa Rosa, CA, 1995. Informational page available at http://grunt.berkeley.edu/cci/cci.html.
10. J.A. Clouse, P.E. Utgoff, A teaching method for reinforcement learning, in: Proceedings of the Ninth Conference on Machine Learning, Morgan Kaufmann, Los Altos, CA, 1992.
11. M. Colombetti, M. Dorigo, G. Borghi, Behavior analysis and training: A methodology for behavior engineering, IEEE Transactions on Systems, Man, and Cybernetics -Part B 26 (3) (1996) 365-380.
12. A. Dickinson, Instrumental conditioning, in: N.J. Mackintosh (Ed.), Handbook of Perception and Cognition, vol. 9, Academic Press, Orlando, FL, 1995.
13. A. Dickinson, Actions and habits: The development of behavioral autonomy, Philosophical Transactions of the Royal Society of London, Series B 308 (1985) 67-78.
14. M. Dorigo, M. Colombetti, Robot shaping: Developing autonomous agents through learning, Artificial Intelligence 70 (2) (1994) 321-370.
15. G.L. Drescher, Made-Up Minds, MIT Press, Cambridge, MA, 1991.
16. C.R. Gallistel, The Organization of Action, Lawrence Erlbaum, Hillsdale, NJ, 1980.
17. D. Gordon, D. Subramanian, A multistrategy learning scheme for agent knowledge acquisition, Informatica 17 (1994) 331-346.
18. R.E. Hampson, C.J. Heyser, S.A. Deadwyler, Hippocampal cell firing correlates of delayed-match-to-sample performance in the rat, Behavioral Neuroscience 107 (5) (1993) 715-739.
19. E. Hearst, H.M. Jenkins, Sign tracking: The Stimulus-Reinforcer Relation and Directed Action, The Psychonomic Society, Austin, 1975.
20. H.M. Jenkins, B.R. Moore, The form of the autoshaped response with food or water reinforcers, Journal of the Experimental Analysis of Behavior 20 (1973) 163-181.
21. L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: A survey, Journal of Artificial Intelligence Research 4 (1996) 237-285.
22. L.-J. Lin, Self-improving reactive agents based on reinforcement learning, planning, and teaching, Machine Learning 8 (1992) 293-321.
23. R. Maclin, J.W. Shavlik, Creating advice-taking reinforcement learners, Machine Learning 22 (1-3) (1996) 251-281.
24. J. del R. Millán, Learning efficient reactive behavioral sequences from basic reflexes in a goal-directed autonomous robot, in: From Animals to Animates 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior, MIT Press, Cambridge, MA, 1994, pp. 266-274.
25. J. del R. Millán, Rapid, safe, and incremental learning of navigation strategies, IEEE Transactions on Systems, Man, and Cybernetics - Part B 26 (3) (1996) 408-420.
26. J.M. Pearce, G. Hall, A model for Pavlovian learning: Variations in effectiveness of conditioned but not unconditioned stimuli, Psychological Review 87 (6) (1980) 532-552.
27. S.M. Pellis, D.P. O'Brien, V.C. Pellis, P. Teitelbaum, D.L. Wolgin, S. Kennedy, Escalation of feline predation along a gradient from avoidance through play to killing, Behavioral Neuroscience 102 (5) (1988) 760-777.
28. S. Perkins, G. Hayes, Robot shaping - principles, methods, and architectures, in: Workshop on Learning in Robots and Animals, AISB'96, 1996.
29. K. Pryor, Lads Before the Wind, Harper and Row, New York, 1975.
30. J.L. Raymond, D.A. Baxter, D.V. Buonomano, J.H. Byrne, A learning rule based on empirically derived activity-dependent neuromodulation supports operant conditioning in a small network, Neural Networks 5 (5) (1992) 789-803.
31. R.A. Rescorla, A.R. Wagner, A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement, in: A.H. Black, W.F. Prokasy (Eds.), Classical Conditioning II: Theory and Research, Appleton-Century-Crofts, New York, 1972.
32. G.S. Reynolds, A Primer of Operant Conditioning, Scott, Foresman, Glenview, IL, 1968.
33. F. Rosenblatt, Principles of Neurodynamics, Spartan, New York, 1962.
34. R. Simmons, Structured control for autonomous robots, IEEE Transactions on Robotics and Automation 10 (1) (1994) 34-43.
35. R. Simmons, R. Goodwin, K. Haigh, S. Koenig, J. O'Sullivan, A modular architecture for office delivery robots, in: The Proceedings of the First International Conference on Autonomous Agents, February 1997.
36. S.P. Singh, Transfer of learning across sequential tasks, Machine Learning 8 (1992) 323-339.
37. R.S. Sutton, A.G. Barto, Toward a modern theory of adaptive networks: Expectation and prediction, Psychological Review 88 (1981) 135-170.
38. P. Utgoff, J. Clouse, Two kinds of training information for evaluation function learning, in: Proceedings of the Ninth National Conference on Artificial Intelligence (AAAI-91), AAAI Press, 1991.
39. C.J.C.H. Watkins, Learning from Delayed Rewards, Ph.D. thesis, Cambridge University, Cambridge, UK, 1989.