Learning and unlearning mechanisms in animats and animals

Adaptive Behavior

Volumn 8, Issue 2, 2000, Pages 97-128

  Savage, Tony ^a   Ziemke, Tom ^b  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^b UNIVERSITY OF SKÖVDE   (Sweden)

Author keywords

Causality; Dual contingencies; Learning unlearning; Motivation; Strengthening and weakening mechanisms

Indexed keywords

EID: 23044526280     PISSN: 10597123     EISSN: None     Source Type: Journal    
DOI: 10.1177/105971230000800202     Document Type: Article

References (79)

1. Adams, C.D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. The Quarterly Journal of Experimental Psychology, 34B, pp.77-98.
2. Adams, C.D. and Dickinson, A. (1981). Instrumental responding following reinforcement devaluation. Quarterly Journal of Experimental Psychology, 33B, pp.109-22.
3. Amsel, A. (1958). The role of frustrative non reward in non continuous reward situations. Psychological Bulletin, 55, pp.102-119.
4. Baeyens, F., Helen, P. and Crombez, G. (1995). Pavlovian associations are forever: On classical conditioning and extinction. Journal of Psychophysiology, 9, pp.127-141.
5. Balkenius, C. (1996). Generalization in instrumental learning. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
6. Balkenius, C. (1995). Natural Intelligence in Artificial Creatures. Sweden: Lund University.
7. Balkenius, C. (1994). Natural Intelligence for Autonomous Agents. LUCS 29. Sweden: Lund University.
8. Barnes, J.M., and Underwood, B.J. (1959). Fate of first-list associations in transfer theory. Journal of Experimental Psychology, 58, pp.97-105.
9. Blumberg, B., Todd, P. and Maes, P. (1996). No bad dogs: ethological lessons for learning in hamsterdam. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
10. Bouton, M.E. (1993). Context, time and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114 (1), pp.80-99.
11. Bouton, M.E. (1991). Context and retrieval in extinction and in other examples of interference in simple associative learning. In L. Dachowski and C.F. Flaherty (Eds.), Current topics in animal learning: Brain, emotion, and cognition. Hillsdale, NJ: LEA.
12. Bouton, M.E. and Swartzentruber, D. (1989). Slow reacquisition following extinction: Context, encoding, and retrieval mechanisms. Journal of Experimental Psychology: Animal Behavior Processes, 15, pp.43-53.
13. Brown, P.L. and Jenkins, H.M. (1968). Auto-shaping of the pigeon's key peck. Journal of the Experimental Analysis of Behavior, 11, pp.1-8.
14. Cecconi, F. and Parisi, D. (1993). Neural networks with motivational units. In J.-A. Meyer, H.L. Roitblat and S.W. Wilson (Eds.), From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
15. Chatlosh, D.L., Neunaber, D.J., and Wasserman, E.A. (1985). Response-outcome contingency: behavioral and judgmental effects of appetitive and aversive outcomes with college students. Learning and Motivation, 16, pp.1-34.
16. Chang, C. and Gaudiano, P. (1998). Application of biological learning theories to mobile robot avoidance and approach behaviors. Journal of Complex Systems, 1, pp.79-114.
17. Cheng, P.W. and Holyoak, K.J. (1995). Adaptive systems as intuitive statisticians: Causality, contingency, and prediction. In J.-A. Meyer and H. Roitblat (Eds.), Comparative approaches to cognition. Cambridge, MA: The MIT Press.
18. Colwill, R.M. (1994). Associative representations of instrumental contingencies. The Psychology of Learning and Motivation, 31, pp.1-72.
19. Colwill, R.M. (1993). Signalling the omission of a response-contingent outcome reduces discriminative control. Animal Learning and Behavior, 21, pp.337-345.
20. Colwill, R.M. and Rescorla, R.A. (1990). Evidence for the hierarchical structure of instrumental learning. Animal Learning and Behavior, 18 (1), pp.71-82.
21. Colwill, R.M. and Rescorla, R.A. (1986). Associative structures in instrumental learning. In G.H. Bower (Ed.) The Psychology of Learning and Motivation, 20, pp.55-104.
22. Davey, G. (1989). Ecological learning theory. London: Routledge.
23. Dickinson, A. (1994). Instrumental conditioning. In N. J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press.
24. Dickinson, A. and Mulatero, C.W. (1989). Reinforcer specificity of the suppression of instrumental performance on a non-contingent schedule. Behavioural Processes, 19, pp.167-180.
25. Dickinson, A. and Shanks, D.R. (1995). Instrumental action and causal representation. In D. Sperber, D. Premack and A.J. Premack (Eds.), Causal Cognition: A multidisciplinary debate. Oxford: Clarendon Press.
26. Domjan, M. (1993). The Principles of Learning and Behavior (3rd Edition). CA: Brooks/Cole.
27. Donahoe, J.W, Burgos, J.E., and Palmer, D.C. (1993). A selectionist approach to reinforcement. Journal of the Experimental Analysis of Behavior, 60, pp.17-40.
28. Gabora, L.M. (1993). Should I stay or should I go: Co-ordinating biological needs with continuously-updated assessments of the environment. In J.-A. Meyer, H.L. Roitblat and S.W. Wilson (Eds.), From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
29. Galbicka, G. (1994). Shaping in the 21st century: moving percentile schedules into applied settings. Journal of Applied Behavior Analysis, 27, pp.739-760.
30. Gaudiano, P., Zalama, E., Chang, C. and Coronado, J.L. (1996). A model of operant conditioning for adaptive obstacle avoidance. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
31. Groves, P.M. and Thompson, R.F. (1970). Habituation: a dual-process theory. Psychological Review, 77, pp.419-450.
32. Guha, D., Dutta, S.N. and Pradhan, S.N. (1974). Conditioning of gastric secretion by epinephrine in rats. Proceedings of the Society for Experimental Biology and Medicine, 147, pp.817-919.
33. Halperin, J.R.P. (1995). Cognition and emotion in animals and machines. In H.L. Roitblat and J.-A. Meyer (Eds.), Comparative Approaches to Cognitive Science. Cambridge, MA: The MIT Press.
34. Halperin, J.R.P. (1991). Machine Motivation. In J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats, Proceedings of the First International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
35. Hammond, L.J. (1980). The effects of contingencies upon appetitive conditioning of free-operant behavior. Journal of the Experimental Analysis of Behavior, 34, pp.297-304.
36. Hebb, D.O. (1949). The Organization of Behavior. NY: Wiley.
37. Herrnstein, R.J. (1990). Levels of stimulus control: A functional approach. Cognition, 37, pp.133-166.
38. Holland, P.C. (1983). "Occasion-setting" in conditional discriminations. In M.L. Commons, R.J. Herrnstein and A.R. Wagner (Eds.), Quantitative analyses of behavior: Discrimination processes (Vol. 4). NY: Ballinger.
39. Hull, C. L. (1951). Essentials of behavior. New Haven, CT: Yale University Press.
40. Kamin, L.J. (1969). Predictability, surprise, attention, and conditioning. In B.A. Campbell and R.M. Church (Eds.), Punishment and aversive behavior. NY: Appleton-Century-Crofts.
41. Lawrence, D.H. (1952). The transfer of a discrimination along a continuum. Journal of Comparative and Physiological Psychology, 45, pp.511-16.
42. Lieberman, D.A., McIntosh, D.C. and Thomas, G.V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 5, pp.224-242.
43. Lorenz, K. (1950). The comparative method in studying innate behavior patterns. Symp. Soc. Exp. Biol., 4, pp. 221-268.
44. Mackintosh, N.J. (1983). Conditioning and associative learning. Oxford: Oxford University Press.
45. Mackintosh, N.J. (1974). The Psychology of Animal Learning. London: Academic Press.
46. Mackintosh, N.J. and Dickinson, A. (1979). Instrumental (Type II) conditioning. In A. Dickinson and R.A. Boakes (Eds.), Mechanisms of learning and motivation. Hillsdale, NJ: Erlbaum.
47. Maes, P. (1994). Modelling Adaptive Autonomous Agents. Artificial Life, 1, pp.135-62.
48. Maes, P. (1991). Learning behavior networks from experience. In F.J. Varela and P. Bourgine (Eds.), Towards a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life. Cambridge, MA: The MIT Press.
49. McFarland, D. J. and Sibly, R. M. (1975). The behavioural final common path. Philosophical Transactions of the Royal Society, 270, B.907, pp. 265-293.
50. Midgley, M., Lea, S.E.G. and Kirby, R. (1989). Algorithmic shaping and misbehavior in the acquisition of token deposit by rats. Journal of the Experimental Analysis of Behavior, 52, pp.27-40.
51. Pearce, J.M. (1994). Discrimination and categorisation. In N.J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press.
52. Rescorla, M.E., and Wagner, A.R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. Black and W. Prokasy (Eds.), Classical Conditioning II, Current Research and Theory. NY: Appleton-Century-Crofts.
53. Roitblat, H.L. (1994). Mechanism and process in animal behavior: models of animals, animals as models. In D. Cliff, D, P. Husbands, J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animals 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
54. Saksida, L.M., Raymond, S.M., Touretzky, D.S. (1997). Shaping robot behavior using principles from instrumental conditioning. Robotics and Autonomous Systems, 22, pp.231-249.
55. Salomon, R. (1998). Improving the DAC architecture by using proprioceptive sensors. In R. Pfeifer, B. Blumberg, J-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats 5, Proceedings of the Fifth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
56. Savage, T. (2000a). Artificial motives: a review of motivation in artificial creatures. Connection Science, 12, pp.1-67.
57. Savage, T. (2000b). Shaping: a multi-contingency model. Submitted to Connection Science.
58. Schmajuk, N. (1997). Animal learning and cognition: a neural network approach. Cambridge: Cambridge University Press.
59. Seely, R., Ramsey, D. and Woods, S. (1997). Regulation of food intake: interactions between learning and physiology. In M.E. Bouton and M.S. Fanselow (Eds.), Learning, Motivation and Cognition. American Psychological Association.
60. Shanks, D.R. (1995). The Psychology of Associative Learning. Cambridge: Cambridge University Press.
61. Shanks, D.R. (1994). Human Associative Learning. In N.J. Mackintosh (Ed.), Animal Learning and Cognition. London: Academic Press.
62. Shanks, D.R. and Dickinson, A. (1991). Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Memory and Cognition, 19, pp.353-360.
63. Shanks, D.R. and Dickinson, A. (1990). Contingency awareness in evaluative conditioning: A comment on Baeyens, Eelen and van den Berg. Cognition and Emotion, 49 (1), pp.19-30.
64. Siegal, S. (1989). Pharmacological conditioning and drug effects. In A.J. Goudie and M.W. Emmett-Oglesby (Eds.), Psychoactive drugs: Tolerance and sensitization. Clifton, NJ: Humana Press.
65. Skinner, B. F. (1938) The Behavior of Organisms. New York: Appleton-Century-Crofts.
66. Solomon, R.L. and Corbit, J.D. (1974). An opponent-process theory of motivation: I. The temporal dynamics of affect. Psychological Review, 81, pp.119-145.
67. Spence, K.W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press.
68. Spier, E. and McFarland, D. J. (1996). A finer-grained motivational model of behaviour sequencing. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S.W. Wilson (Eds.), From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
69. Spier, E. and McFarland, D.J. (1998). Learning to do without cognition. In R. Pfeifer, B. Blumberg, J.-A. Meyer and S.W. Wilson (Eds.), From Animals to Animats 5, Proceedings of the Fifth International Conference on the Simulation of Adaptive Behavior. Cambridge, MA: The MIT Press/Bradford Books.
70. Stewart, J., and Eikelbloom, R. (1987). Conditioned drug effects. In L.L. Iversen and S.H. Snyder (Eds.), Handbook of Psychopharmacology, 19, pp.1-57.
71. Tait, R. and Saladin, M.E. (1986). Concurrent development of excitatory and inhibitory associations during backward conditioning. Animal Learning and Behavior, 14, pp.133-137.
72. Toates, F. (1986). Motivational systems. Cambridge: Cambridge University Press.
73. Thorndike, E.L. (1911). Animal intelligence: Experimental studies. NY: Macmillan.
74. Tinbergen, N. (1951). The study of instinct. London: Oxford University Press.
75. Toates, F. (1995). Animal motivation and cognition, In H. Roitblat and J.-A. Meyer (Eds.), Comparative approaches to cognitive science. Cambridge, MA: The MIT Press.
76. Uhl, C. and Garcia, E. (1969). Comparison of omission and extinction in response elimination in rats. Journal of Comparative and Physiological Psychology, 69, pp.554-562.
77. Wagner, A.R. and Brandon, S.E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In S.B. Klein and R.R. Mower (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Erlbaum.
78. Wasserman, E.A., Elek, S.M. Chalosh, D.L. and Baker, A.G. (1993). Rating causal relations: The role of probability in judgements of response-outcome contingency. Journal of Experimental Psychology: Learning, Memory and Cognition, 19, pp.174-88.
79. Williams, D.R., and Williams, H. (1969). Auto-maintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, pp.511-520.