Measurement and the explanation of adaptive and novel behaviors in real and artificial creatures

Cognitive Systems Research

Volumn 5, Issue 1, 2004, Pages 3-39

  Savage, Tony ^a  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Behavior scales; Flexibility; Novel behaviors; Shaping techniques

Indexed keywords

ADAPTIVE BEHAVIOR; ANIMAL BEHAVIOR; ARTICLE; BEHAVIOR MODIFICATION; CONDITIONING; LEARNING; MAZE TEST; NONHUMAN; PRIORITY JOURNAL; RAT; REINFORCEMENT; ROBOTICS; STIMULUS RESPONSE; TASK PERFORMANCE;

EID: 2442476030     PISSN: 13890417     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cogsys.2003.08.001     Document Type: Article

References (74)

1. Adams C.D. Dickinson A. Instrumental responding following reinforcement devaluation Quarterly Journal of Experimental Psychology 33B 1981 109-122
2. Arbib M.A. Schema theory Arbib M.A. The handbook of brain theory and neural networks 1998 MIT Press Cambridge, MA
3. Arkin R. Motor schema-based mobile robot navigation International Journal of Robotics Research 8 4 1989 92-112
4. Arkin, R. C. (1990). Integrating behavioral, perceptual, and world knowledge in reactive navigation. In P. Maes (Ed.), Robotics and autonomous systems: Vol. 6. Special issue on designing autonomous agents, 105-122
5. Balleine B. Instrumental performance following a shift in primary motivation depends on incentive learning Journal of Experimental Psychology: Animal Behavior Processes 18 1992 236-250
6. Balleine B. Garner C. Gonzalez F. Dickinson A. Motivational control of heterogeneous instrumental chains Journal of Experimental Psychology: Animal behavior processes 21 1995 203-217
7. Balsam P.D. Deich J.D. Ohyama T. Stokes P.D. Origins of new behavior O'Donohue W. Learning and Behavior Therapy 1998 Allyn and Bacon Needham Heights, MA
8. Baum W.M. Understanding behaviorism 1994 Harper Collins NY
9. Blumberg B. Todd P. Maes P. No bad dogs: Ethological lessons for learning in hamsterdam Maes P. Mataric M. Meyer J.-A. Pollack J. Wilson S.W. From animals to animats 4, Proceedings of the fourth international conference on the simulation of adaptive behavior 1996 The MIT Press/Bradford Books Cambridge, MA
10. Boakes R.A. Poli M. Lockwood M.J. Goodall G.A. A study of misbehavior: Token reinforcement in the rat Journal of the Experimental Analysis of Behavior 29 1978 115-134
11. Brooks R. Intelligence without representation Artificial Intelligence 47 1991 139-159
12. Brown P.L. Jenkins H.M. Auto-shaping of the pigeon's key peck Journal of the Experimental Analysis of Behavior 11 1968 1-8
13. Colombetti M. Dorigo M. Borghi G. Behavior analysis and training. A methodology for behavior engineering IEEE Transactions on Systems, Man, and Cybernetics 26 3 1996 365-380
14. Dorigo M. Colombetti M. Robot shaping: an experiment in behaviour engineering 1998 MIT Press/Bradford Books
15. Dorigo M. Schnepf U. Organization of robot behavior through genetic learning processes Fifth International Conference on Advanced Robotics 2 1991 1456-1460
16. Epstein A. Instinct and motivation as explanations for complex behavior Pfaff D.W. The physiological mechanisms of motivation 1982 Springer-Verlag NY
17. Epstein R. Kirshnit C.E. Lanza R.P. Rubin L.C. 'Insight' in the pigeon: Antecedents and determinants of an intelligent performance Nature 308 1984 61-62
18. Floreano D. Mondada F. Evolution of homing navigation in a real mobile robot IEEE Transactions on Systems, Man and Cybernetics 26 1996 396-407
19. Galbicka G. Shaping in the 21st century: Moving percentile schedules into applied settings Journal of Applied Behavior Analysis 27 1994 396-407
20. Galbicka G. Platt J.R. Parametric manipulation of interresponse-time contingency independent of reinforcement rate Journal of Experimental Psychology: Animal behavior Processes 12 1986 371-380
21. Gullapalli V. Reinforcement learning of complex behavior through shaping Donahoe J.W. Packard Dorsel V. Neural-network models of cognition 1997 Elsevier NY
22. Harvey I. Husbands P. Cliff D. Thompson A. Jakobi N. Evolutionary robotics: The Sussex approach Robotics and Autonomous Systems 20 1997 205-224
23. Heikkonen J. Koikkalainen P. Self-organization and autonomous robots Omidvar O. van der Smagt P. Neural systems for robotics 1998 Academic Press San Diego, CA
24. Hugues, L., & Drogoul, A. (2001). Shaping of robot behaviors by demonstrations. In: First international workshop on epigenetic robotics: Modeling cognitive development in robotic systems. Lund University Cognitive Studies 85
25. Jakobi N. Half-baked, ad-hoc and noisy: Minimal simulations for evolutionary robotics Husbands P. Harvey I. Fourth European conference on artificial life 1997 The MIT Press Cambridge, MA
26. Jordan M. Jacobs R. Hierarchical mixtures of experts and the EM algorithm Neural Computation 6 1994 181-214
27. Kaplan F. Oudeyer P.-Y. Kubinyi E. Miklosi A. Robotic clicker training Robotics and Autonomous Systems 38 2002 197-206
28. Kohler W. The mentality of apes 1925 Routledge and Kegan Paul London
29. MacFarland D.A. The role of kinesthesis in maze learning University of California Publications in Psychology 4 1930 277-305
30. Mackenzie D. Arkin R. Cameron J. Multiagent mission specification and execution Autonomous Robots 4 1997 29-52
31. Mahadevan S. Connell J. Automatic programming of behaviour-based robots using reinforcement learning Artificial Intelligence 55 1992 311-365
32. Malott R.W. Whaley D.L. Malott M.A. Elementary principles of behavior 1997 Prentice Hall NJ
33. Martin G. Pear J. Behavior modification: What it is and how to do it sixth ed. 1999 Prentice Hall NJ
34. Mataric M. Integration of representation in goal-driven behavior-based robots IEEE Transactions on Robotics and Automation 8 3 1992 304-312
35. Mataric M. Reward functions for accelerated learning Proceedings conference on simulation of adaptive behavior 1995 MIT Press Cambridge, MA
36. Mataric M. Sensory-motor primitives as a basis for imitation: Linking perception to action and biology to robotics Dautenhahn K. Nehaniv C. Imitation in animals and artifacts 2000 The MIT Press Cambridge, MA
37. Mataric M. Learning in behavior-based multi-robot systems: Policies, models, and other agents Cognitive Systems Research 2 1 2001 81-93
38. Midgley M. Lea S.E.G. Kirby R. Algorithmic shaping and misbehavior in the acquisition of token deposit by rats Journal of the Experimental Analysis of Behavior 52 1989 27-40
39. Morgan M.J. Nicholas D.J. Discrimination between reinforced action patterns in the rat Learning and Motivation 10 1979 1-22
40. Nehmzow U. McGonigle B. Achieving rapid adaptations in robots by means of external tuition Cliff D. Husbands P. Meyers J.-A. Wilson S. From animals to animats 3, Proceedings of the third international conference on the simulation of adaptive behavior 1994 The MIT Press/Bradford Books Cambridge, MA
41. Nolfi, S. (1996). Adaptation as a more powerful tool than decomposition and integration. Technical Report 96-03. Rome, Italy: Department of neural systems and Artificial Life
42. Nolfi S. Evolving non-trivial behaviors on real robots: A garbage collecting robot Robotics and Autonomous Systems 22 1997 197-198
43. Nolfi S. Using emergent modularity to develop control systems for mobile robots Adaptive Behavior 5 1997 343-363
44. Nolfi S. Floreano D. Co-evolving predator and prey robots: Do 'arms races' arise in artificial evolution? Artificial Life 4 1998 311-335
45. Nolfi S. Miglioni O. Studying the emergence of grounded representations: exploiting the power and the limits of sensory-motor coordination Fetzer J.H. Evolving Consiousness 2002 John Benjamins Amsterdam
46. Nolfi S. Parisi D. Learning to adapt to changing environment in evolving neural networks Adaptive Behavior 1997 99-105
47. Perkins S. Hayes G. Evolving complex visual behaviours using genetic programming and shaping Demiris J. Birk A. Interdisciplinary approaches to robot learning 2000 World Scientific
48. Platt J.R. Percentile reinforcement: Paradigms for experimental analysis of response shaping Psychology of Learning and Motivation 7 1973 271-296
49. Pryor K.W. Hagg R. O'Reilly J. The creative porpoise: Training for novel behavior Journal of The Experimental Analysis of Behavior 12 1969 653-661
50. Ram A. Boone G. Arkin R. Pearce M. Using genetic algorithms to learn reactive control parameters for autonomous robotic navigation Adaptive Behavior 2 1994 277-305
51. Rizzolatti G. Gadiga L. Gallese V. Fogassi L. Pre-motor cortex and the recognition of motor actions Cognitive Brain Research 3 1996 131-141
52. Ronald E. Sipper M. Capcarrere M. Design, observation, surprise! A test of emergence Artificial Life 5 1999 225-239
53. Savage T. Shaping: A multiple contingencies analysis and its relevance to behaviour-based robotics Connection Science 2001 199-234
54. Savage T. The grounding of motivation in artificial animals: Indices of motivational behavior Cognitive Systems Research 4 2003 23-55
55. Schwartz B. Robbins S.J. Psychology of learning and behavior 1995 Norton - Co NY
56. Scott R.W. Peters R.D. Gillespie W.J. Blanchard E.B. Edmunson E.D. Young L.D. The use of shaping and reinforcement in the operant acceleration and deceleration of heart rate Behavior Research and Therapy 11 1973 179-185
57. Sharkey N. Heemskerk J. The neural mind and the robot Brown A.J. Neural network perspectives on cognition and adaptive robotics 1997 IOP Press Bristol, UK
58. Skinner B.F. The behavior of organisms 1938 Appleton-Century NY
59. Skinner B.F. Science and human behavior 1953 Macmillan NY
60. Sporns O. Almassy N. Edelman G. Plasticity in value systems and its role in adaptive behavior Adaptive Behavior 8 2000 129-148
61. Staddon J.E.R. Adaptive behavior and learning 1983 Cambridge University Press NY
62. Steels L. The artificial life roots of artificial intelligence Langton C.G. Artificial life: An overview 1995 The MIT Press Cambridge, MA
63. Stein L. Bao G.X. Belluzzi J.D. A cellular analogue of operant conditioning Journal of Experimental Analysis of Behavior 60 1993 41-53
64. Stokes P.D. Learned variability Animal Learning and Behavior 23 1995 164-176
65. Stokes P.D. Balsam P.D. Effects of reinforcing preselected approximations on the topography of the rat's bar press Journal of Experimental Analysis of Behavior 55 1991 213-231
66. Sun R. Peterson T. Multi-agent reinforcement learning: Weighting and partitioning Neural Networks 12 1999 727-753
67. Sun R. Sessions C. Self-segmentation of sequences: Automatic formation of hierarchies of sequential behaviors IEEE Transactions on Systems, Man and Cybernetics 30 3B 2000 403-418
68. Timberlake A. Lucas G. Behavior systems and learning: From misbehavior to general principles Klein S.B. Mower R.R. Contemporary learning theories: Instrumental conditioning theory and the impact of biological constraints on learning 1989 LEA Hillsdale, NJ
69. Tyrrell T. An evaluation of Maes's bottom-up mechanism for behavior selection Adaptive Behavior 2 1994 307-348
70. Uhl C. Garcia E. Comparison of omission and extinction in response elimination in rats Journal of Comparative and Physiological Psychology 69 1969 554-562
71. Urzelai J. Floreano D. Dorigo M. Colombetti M. Incremental robot shaping Connection Science 10 1998 341-360
72. Watkins C. Q-learning Machine Learning 8 1992 279-292
73. Wiering M. Schmidhuber J. HQ-learning Adaptive Behavior 6 1996 219-246
74. Wynn C. Animal Cognition 2001 Palgrave Press New York, NY