Shaping: A multiple contingencies analysis and its relevance to behaviour-based robotics

Connection Science

Volumn 13, Issue 3, 2001, Pages 199-234

  Savage, Tony ^a  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Critical situations; Evolutionary learning; Fitness functions; Multiple contingencies; Shaping; Successive approximations

Indexed keywords

CONTINGENCIES ANALYSIS;

APPROXIMATION THEORY; AUTONOMOUS AGENTS; BEHAVIORAL RESEARCH; LEARNING SYSTEMS;

ROBOTICS;

EID: 0035545047     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/09540090110096196     Document Type: Article

References (47)

1. Balsam, P. D., Deich, J. D., Ohyama, T., and Stokes, P. D., 1998, Origins of new behavior. In W. O'Donohue (ed.) Learning and Behavior Therapy (Needham Heights MA: Allyn and Bacon).
2. Banzhaf, W., Nordin, P., Keller, R., and Francone, F., 1998, Genetic Programming: An Introduction (San Francisco: Morgan Kaufmann).
3. Bindra, D., 1976, A Theory of Intelligent Behavior (New York: Wiley).
4. Blumberg, B., Todd, P., and Maes, P., 1996, No bad dogs: ethological lessons for learning in hamsterdam. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S. W. Wilson (eds) From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
5. Brown, P. L., and Jenkins, H. M., 1968, Auto-shaping of the pigeon's key peck. Journal of the Experimental Analysis of Behavior, 11: 1-8.
6. Colombetti, M., Dorigo, M., and Borghi, G., 1996, Behavior analysis and training: a methodology for behavior engineering. IEEE Transactions on Systems, Man and Cybernetics - Part B, 26: 365-380.
7. Dickinson, A., Hall, G., and Mackintosh, N.J., 1976, Surprise and the attenuation of blocking. Journal of Experimental Psychology. 37B: 397-416.
8. Dickinson, A., and Shanks, D., 1995, Instrumental action and causal representation. In D. Sperber. D. Premack and A. J. Premack (eds) Causal Cognition: A Multidisciplinary Debate (Oxford: Clarendon Press).
9. Dorigo, M., and Colombetti, M., 1998. Robot Shaping: An Eexperiment in Behavior Engineering (Cambridge MA: MIT Press).
10. Dorigo, M., and Colombetti, M., 1994, Robot shaping: developing autonomous agents through learning. Artificial Intelligence, 71 (2): 321-370.
11. Frick, R. W., 1986. Is the force of response shaped by reinforcement? Perceptual and Motor Skills. 62: 267-277.
12. Galbicka, G., 1994, Shaping in the 21st century: moving percentile schedules into applied settings. Journal of Applied Behavior Analysis, 27: 739-760.
13. Galbicka, G., 1988, Differentiating the behavior of organisms. Journal of the Experimental Analysis of Behavior, 50: 343-354.
14. Galbicka, G., and Platt, J., 1986, Parametric manipulation of inter response-time contingency independent of reinforcement rate. Journal of Experimental Psychology: Animal Behavior Processes, 12: 371-380.
15. Gaussier, P., Joulain, C., Revel, A., and Banquet, J., 1996. Are shaping techniques the correct answer for the control of an autonomous robot? In UKACC International Conference on Control '96. University of Exeter, IEE.
16. Harvey, I., Husbands, P., and Cliff, D., 1993. Issues in evolutionary robotics. In J.-A. Meyers, H. L. Roitblat and S. W. Wilson (eds) From Animals to Animals 2, Proceedings of the Second International Conference on the Simulation of Adaptive Behavior (Cambridge MA.: The MIT Press/Bradford Books).
17. Harvey, I., Husbands, P., Cliff, D.,Thompson, A., and Jakobi, N., 1997, Evolutionary robotics: the Sussex approach. Robotics and Autonomous Systems, 20: 205-224.
18. Lee, W.-P., 1999, Evolving complex robot behaviors. Information Sciences, 121: 1-25.
19. Lund, H., and Parisi, D., 1995, Pre-adaptations in populations of neural networks evolving in a changing environment. Artificial Life, 2 (2): 179-197.
20. Malott, R. W., Whaley, D. L., and Malott, M. E., 1997, Elementary Principles of Behavior (Englewood Cliffs NJ: Prentice Hall).
21. Martin, G., and Pear, J., 1999, Behavior Modification: What it is and How to do it, 6th edn (Englewood Cliffs NJ: Prentice Hall).
22. Midgley, M., Lea, S. E. G., and Kirby, R., 1989, Algorithmic shaping and misbehavior in the acquisition of token deposit by rats. Journal of the Experimental Analysis of Behavior, 52: 27-40.
23. Moore, B. R., 1973, The role of directed Pavlovian reactions in simple instrumental learning in the piegeon. In R. A. Hinde and J. Stevenson-Hinde (eds) Constraints on Learning (London: Academic Press).
24. Morgan, M. J., and Nicholas, D. J., 1979, Discrimination between reinforced action patterns in the rat. Learning and Motivation, 10: 1-22.
25. Nolfi, S., 1998, Evolutionary robotics: exploiting the full power of self-organization. Connection Science, 10: 167-184.
26. Nolfi, S., 1997a, Using emergent modularity to develop control systems for mobile robots. Adaptive Behavior, 3-4: 343-364.
27. Nolfi, S., 1997b, Evolving non-trivial behaviors on real robots: a garbage collecting robot. Robotics and Autonomous Systems, 22: 187-198.
28. Nolfi, S., and Floreano, D., 1999. Learning and evolution. Autonomous Robots, 7: 89-113.
29. Nolfi, S. and Parisi, D. 1995 'Genotypes' for neural networks. In M. A. Arbib (ed.) The Handbook of Brain Theory and Neural Networks (Cambridge, MA: MIT Press).
30. Pear, J. J., and Legris, J. A., 1987, Shaping by automated tracking of an arbitrary operant response. Journal of the Experimental Analysis of Behavior, 47: 241-247.
31. Platt, J. R., 1984, Motivational and response factors in temporal differentiation. In J. Gibbon and L. Allen (eds) Timing and Time Perception (New York: New York Academy of Sciences).
32. Platt, J. R., 1973, Percentile reinforcement: paradigms for experimental analysis of response shaping. Psychology of Learning and Motivation. 7: 271-296.
33. Saksida, L. M., Raymond, S. M., and Toutetzky, D. S., 1997, Shaping robot behavior using principles from instrumental conditioning. Robotics and Autonomous Systems, 22: 231-249.
34. Savage, T., 1998, Shaping: the link between rats and robots. Connection Science, 10: 321-340.
35. Savage, T., and Ziemke, T., 2001, Learning and unlearning mechanisms in animals and animals. Adaptive Behavior, 8 (2): 93-124.
36. Scutt, T., and Damper, R., 1997, Designing a nervous system for an adaptive robot. In A. J. Brown (ed.) Neural Network Perspectives on Cognition and Adaptive Robotics (Bristol: IOP Press).
37. Shanks, D. R., 1994, Human associative learning. In N. J. Mackintosh (ed.) Animal Learning and Cognition (London: Academic Press).
38. Sharkey, N., and Heemskerk, J., 1997, The neural mind and the robot. In A. J. Brown (ed.) Neural Network Perspectives on Cognition and Adaptive Robotics (Bristol: IOP Press).
39. Skinner, B. F., 1938, The Behavior of Organisms (New York: Appleton-Century).
40. Sladdon, J. E. R., 1983, Adaptive Behavior and Learning (New York: Cambridge University Press).
41. Stein, L., Xue, B. G., and Belluzzi, J. D., 1993, A cellular analogue of operant conditioning. Journal of Experimental Analysis of Behavior. 60: 41-53.
42. Stokes, P. D., 1995, Learned variability. Animal Learning and Behavior, 23: 164-176.
43. Stokes, P. D., and Balsam, P. D., 1991, Effects of reinforcing preselected approximations on the topography of the rat's bar press. Journal of Experimental Analysis of Behavior, 55: 213-231.
44. Timberlake, A., and Lucas, G., 1989, Behavior systems and learning: from misbehavior to general principles. In S. B. Klein and R. R. Mower (eds) Contemporary Learning Theories: Instrumental Conditioning Theory and the Impact of Biological Constraints on Learning (Hillsdale NJ: LEA).
45. Timberlake, W., Wahl, G., and King, D., 1982, Stimulus and response contingencies in the misbehavior of rats. Journal of Experimental Psychology: Animal Behavior Processes, 8: 62-85.
46. Urzelai, J., Floreano, D., Dorigo, M., and Colombetti, M., 1998. Incremental robot shaping. Connection Sciencem, 10: 341-360.
47. Williams, D. R., and Williams, H., 1969, Auto-maintenance in the pigeon: sustained pecking despite contingent non reinforcement. Journal of the Experimental Analysis of Behavior, 12: 511-520.