Artificial motives: A review of motivation in artificial creatures

Connection Science

Volumn 12, Issue 3-4, 2000, Pages 211-277

  Savage, Tony ^a  

^a QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Affect; Animats; Drive; Incentive; Instinct; Motivation; Value

Indexed keywords

EID: 0034345039     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/095400900750060131     Document Type: Article

References (117)

1. Adams, C. D., 1982, Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 34B: 77-98.
2. Adams, C. D., and Dickinson, A., 1981a, Instrumental responding following reinforcement devaluation. Quarterly Journal of Experimental Psychology, 33B: 109-22.
3. Adams, C., and Dickinson, A., 1981b, Actions and habits: variations in associative representations during instrumental learning. In N. E. Spear and R. R. Miller (eds) Information Processing in Animals: Memory Mechanisms (Hillsdale NJ: Lawrence Erlbaum).
4. Amsel, A., and Roussell, J., 1952, Motivational properties of frustration: I. Effect on a running response of the addition of frustration to the motivational complex. Journal of Experimental Psychology, 43: 363-68.
5. Arbib, A., and Lieblich, I., 1977, Motivational learning of spatial behaviour. In J. Metzler (ed.) Systems Neuroscience (New York: Academic Press).
6. Baerends, G. P., 1976, The functional organisation of behavior. Animal Behavior, 24: 726-738.
7. Balkenius, C., 1995, Natural Intelligence in Artificial Creatures (Sweden: Lund University).
8. Balkenius, C., 1994, Natural Intelligence for Autonomous Agents. LUCS 29 (Sweden: Lund University).
9. Balleine, B., 1992, Instrumental performance following a shift in primary motivation depends on incentive learning. Journal of Experimental Psychology: Animal Behavior Processes, 18, 236-250.
10. Balleine, B., and Dickinson, A., 1998, The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety. Animal Learning and Behavior, 26: 46-59.
11. Balleine, B., Garner, C., Gonzalez, F., and Dickinson, A., 1995, Motivational control of heterogeneous instrumental chains. Journal of Experimental Psychology: Animal Behavior Processes, 21: 203-217.
12. Baxter, D., and Zambie, E., 1982, Reinforcer and response specificity in appetitive transfer of control. Animal Learning and Behavior, 10: 201-210.
13. Bindra, D., 1978, How adaptive behavior is produced: a perceptual-motivational alternative to response reinforcement. Behavioral and Brain Sciences, 1: 41-91.
14. Bindra, D., 1976, A Theory of Intelligent Behavior (New York: Wiley).
15. Bindra, D., 1974, Neuropsychological interpretation of the effects of drive and incentive-motivation on general activity and instrumental behavior. Psychological Review, 75: 1-22.
16. Blumberg, B., 1994, Action-selection in Hamsterdam: lessons from ethology. In D. Cliff, P. Husbands, J. Meyer and S. W. Wilson (eds) From Animals to Animats 3, Proceedings of the Third International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
17. Blumberg, B., Todd, P. and Macs, P., 1996, No bad dogs: ethological lessons for learning in hamsterdam. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S. W. Wilson (eds) From Animals to Animals 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
18. Bolles, R. C., 1975 Theory of Motivation, 2nd edn (New York: Harper & Row).
19. Bolles, R. C., and Morlock. H., 1960, Some asymmetrical drive summation phenomena. Psychological Reports, 6: 373-378.
20. Bouton, M. E., 1991, Context and retrieval in extinction and in other examples of interference in simple associative learning. In L. Dachowski and C. F. Flaherty (eds) Current Topics in Animal Learning (Hillsdale NJ: Lawrence Erlbaum).
21. Brooks, R., 1986, A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 6:14-23.
22. Brooks, R., 1991, Intelligence without representation. Artificial Intelligence, 47: 139-159.
23. Campbell, B., and Sheffield, F., 1953, Relation of random activity to food deprivation. Journal of Comparative and Physiological Psychology, 46: 320-322.
24. Capaldi, E., Miller, D. J., and Alptekin, S., 1989, Multiple-food-unit-incentive effect: non conservation of weight of food reward by rats. Journal of Experimental Psychology: Animal Behavior Processes, 15 (1): 75-80.
25. Collier, G., and Johnson, D., 1997, Motivation as a function of animal versus experimenter control. In M. E. Bouton and M. S. Fanselow (eds) Learning Motivation and Cognition (Washington: American Psychological Association).
26. Colwill, R. M., and Rescorla, R. A., 1985, Post-conditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behavior Processes, 11: 120-132.
27. Dethier, V., and Stellar, E., 1970, Animal Behavior (New Jersey: Prentice-Hall).
28. Deutsch, J., 1979, Drive - another point of view. Trends in Neuro Sciences, 2: 242-244.
29. Deutsch, J. A., 1960, The Structural Basis of Behaviour (Cambridge: Cambridge University Press).
30. Dickinson, A., 1994, Instrumental conditioning. In N. J. Mackintosh (ed.) Animal Learning and Cognition (London: Academic Press).
31. Dickinson, A., and Balleine, B., 1994, Motivational control of goal-directed action. Animal Learning and Behavior, 22: 1-18.
32. Dickinson, A., and Balleine, B., 1990, Motivational control of instrumental performance following a shift from thirst to hunger. Quarterly Journal of Experimental Psychology, 42B: 413-431.
33. Dickinson, A., and Dawson, G. R., 1989, Incentive learning and the motivational control of instrumental performance. Quarterly Journal of Experimental Psychology, 41B: 99-112.
34. Dickinson, A., and Dawson, G. R., 1988, Motivational control of instrumental performance: the role of prior experience of the reinforcer. Quarterly Journal of Experimental Psychology, 40B: 113-134.
35. Dickinson, A., and Dawson, G. R., 1987, Pavlovian processes in the motivational control of instrumental performance. Quarterly Journal of Experimental Psychology, 39B: 201-213.
36. Domjan, M., 1998, The Principles of Learning and Behavior (California: Brooks/Cole).
37. Epstein, A., 1982, Instinct and motivation as explanations for complex behavior. In D. W. Pfaff (ed.) The Physiological Mechanisms of Motivation (New York: Springer).
38. Epstein, A., 1967, Oropharyngeal factors in feeding and drinking. In C. F. Code (ed.) Handbook of Physiology (Washington, DC: American Physiological Society).
39. Gabora, L. M., 1993, Should I stay or should I go: co-ordinating biological needs with continuously-updated assessments of the environment. In J.-A. Meyers, H. L. Roitblat and S. W. Wilson (eds) From Animals to Animats 2, Proceedings of the Second International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
40. Glickman, S., and Schiff, B., 1967, A biological theory of reinforcement. Psychological Review, 74: 81-109.
41. Grey, J., 1975, Elements of a Two-process Theory of Learning (London: Academic Press).
42. Gricc, G., and Davis, J., 1957, Effect of irrelevant thirst motivation on a response learned with food reward. Journal of Experimental Psychology, 53: 347-352.
43. Grossberg, S., 1988, Neural Networks and Natural Intelligence (Cambridge MA: The MIT Press).
44. Grossberg, S., and Levine, D. S., 1987, Neural dynamics of attentionally modulated Pavlovian conditioning: blocking, inter-stimulus interval and secondary reinforcement. Applied Optics, 26: 5015-5030.
45. Guazzelli, A., Bota, M., Corbacho, F., and Arbib, M., 1998, Affordances, motivations and the world Graph theory. Adaptive Behavior, 6: 435-471.
46. Gutman, A., and Maier, S., 1978, Operant and Pavlovian in cross-response transfer of inhibitory stimulus control. Learning and Motivation, 9: 231-254.
47. Halperin, J. R. P., 1995, Cognition and emotion in animals and machines. In H. L. Roitblat and J.-A. Meyers (eds) Comparative Approaches to Cognitive Science (Cambridge MA: The MIT Press/Bradford Books).
48. Halperin, J. R. P., 1991, Machine Motivation. In J.-A. Meyers and S. W. Wilson (eds) From Animals to Animats, Proceedings of the First International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
49. Hoebel, B., and Teitelbaum, P., 1962, Hypothalamic control of feeding and self-stimulation. Science, 135: 375-377.
50. Holland, J., and Reitman, J., 1978, Cognitive systems based on adaptive algorithms. In D. Waterman and F. Hayes-Roth (eds) Pattern-directed Inference Systems (New York: Academic Press).
51. Holland, P. C., and Rescorla, R. A., 1975, The effects of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. Journal of Experimental Psychology: Animal Behavior Processes, 1: 355-363.
52. Holman, G., 1968, Intragastric reinforcement effect. Journal of Comparative and Physiological Psychology, 69: 432-441.
53. Hull. C. L., 1951, Essentials in Behavior (New Haven CT: Yale University Press).
54. Klopf, A. H., and Morgan, J. S., 1990, The role of time in natural intelligence: implications of classical and instrumental conditioning for neuronal and neural-network modelling. In M. Gabriel and J. Moore (eds) Learning and Computational Neuroscience: Foundations of Adaptive Networks (Cambridge MA: The MIT Press).
55. Konorski, J., 1967, Integrative Activity of the Brain (Chicago: University of Chicago Press).
56. Konorski, J., 1948, Conditioned Reflexes and Neuron Organisation (Cambridge: Cambridge University Press).
57. Lack, D., 1943, The Life of the Robin (London: H.F. & G. Witherby).
58. Leyhausen, P. 1956, Das Verhalten der Katzen (Felidae). Handbuch Zoologie Berlin, 8: 1-34.
59. Levine, D. S., 1991, Introduction to Neural and Cognitive Modelling (Hillsdale NJ: Lawrence Erlbaum).
60. Lieblich, I., and Arbib, M., 1982, Multiple representations of space underlying behavior. The Behavioral and Brain Sciences, 5: 627-659.
61. Lorenz, K., and Tinbergen, N., 1957, Taxis and instinct action in the egg-retrieving behavior of the greylag goose. In C. Schiller (ed.) Instinctive Behavior (New York: International Universities Press).
62. Lorenz, K., 1981, The Foundations of Ethology (New York: Springer).
63. Lorenz, K., 1950, The comparative method in studying innate behavior patterns. Symp. Soc. Exp. Biology, 4: 221-268.
64. Lovibond, P., 1983, Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. Journal of Experimental Psychology: Animal Behavior Processes, 9: 225-247.
65. MacFarlane, D. A., 1930, The role of kinesthesis in maze learning. University of California Publications in Psychology, 4: 277-305.
66. Mackintosh, N. J., 1983, Conditioning and Associative Learning (Oxford: Oxford University Press).
67. Mackintosh, N. J., 1974, The Psychology of Animal Learning (London: Academic Press).
68. Maes, P., 1994, Modelling adaptive autonomous agents. Artificial Life, 1: 135-162.
69. Marler, P., and Hamilton, W. J., 1967, Mechanisms of Animal Behavior (New York: Wiley).
70. Mataric, M. J., 1996, Designing and understanding adaptive group behavior. Adaptive Behavior, 4:51-80.
71. Mataric, M. J., 1995, Issues and approaches in the design of collective autonomous agents. Robotics and Autonomous Systems, 16: 321-331.
72. Mataric, M. J., 1992, Integration of representation in goal-driven behavior-based robots. IEEE Transactions on Robotics and Automation, 8 (3): 304-312.
73. Mataric, M. J., 1991, Navigating with a rat brain: a neurobiologically-inspired model for robot spatial representation. In J.-A. Meyer and S. W. Wilson (eds) From Animals to Animats, Proceedings of the First International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
74. McDougall, W., 1914, An Introduction to Social Psychology (Boston: Luce).
75. McFarland, D. J., 1999, Animal Behaviour, 3rd edn (London: Longman).
76. McFarland, D. J., 1969, Separation of satiating and rewarding consequences of drinking. Physiology and Behavior, 4: 987-989.
77. McFarland, D. J., 1966, On the causal and functional significance of displacement activities. Zeitschrift für Tierpsychologie, 23: 217-235.
78. McFarland, D. J., and Bosser, U., 1993, Intelligent Behavior in Animals and Robots (Cambridge MA: The MIT Press).
79. McLaren, I. P., and Dickinson, A., 1990, The conditioning connection. Philosophical Transactions of the Royal Society, B329: 179-186.
80. Meachum, C., 1990, The role of response-contingent incentives in lithium chloride-mediated suppression of an operant response. Quarterly Journal of Experimental Psychology, 42B: 175-195.
81. Michaud, F., Lachiver, G., and Le Dinh, C., 1996, A new control architecture combining reactivity, planning, deliberation and motivation for situated autonomous agents. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S. W. Wilson (eds) From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
82. Millan, J. R., 1996, Rapid, safe, and incremental learning of navigation strategies. IEEE Transactions on Systems, Man, and Cybernetics - Part B, 26: 408-420.
83. Miller, N. E., 1956, Effects of drugs on motivation: the value of using a variety of measures. Annals of the New York Academy of Sciences, 65: 318-333.
84. Miller, N. E., 1955, Shortcomings of food consumption as a measure of hunger: results from other behavioral techniques. Annals of the New York Academy of Sciences, 63: 141-143.
85. Miller, N., and Kessen, M., 1952, Reward effects of food via stomach fistula compared with those of food via mouth. Journal of Comparative and Physiological Psychology, 45: 555-564.
86. Mook, D., Gonder-Frederick, L., Keats, P., and Mangione, R., 1984, Desatiation produced by liquid diet in rats satiated for glucose solution. Physiology and Behavior, 32: 875-877.
87. Morgan, M., 1979, The concept of drive. Trends in Neuro Sciences, 2: 240-242.
88. Murray, H. A., 1938, Explorations in Personality (New York: Oxford University Press).
89. Pelchat, M., Grill, H., Rozin, P., and Jacobs, J., 1983, Quality of acquired responses to taste by Ratus norvegicus depends upon type of associated discomfort. Journal of Comparative and Physiological Psychology, 97: 140-153.
90. Pfaff, D. W., 1982, Motivational concepts: definitions and distinctions. In D. W. Pfaff (ed.) The Physiological Mechanisms of Motivation (New York: Springer).
91. Potegal, M., 1982, The special nature of spatial information. The Behavioral and Brain Sciences, 5: 647-648.
92. Pryor, K. W., Hagg, R., and O'Reilly, J., 1969, The creative porpoise: training for novel behavior. Journal of the Experimental Analysis of Behavior, 12: 653-661.
93. Savage, T., and Ziemke, T., 2000 (forthcoming) Learning and unlearning mechanisms in animats and animals. Adaptive Behavior.
94. Schleidt, W., Yakalis, G., Donnelly, M., and McGarry, J., 1984, A proposal for a standard ethogram, exemplified by an ethogram of the blue-breasted quail. Zeitsschrift für Tierpsychologie, 64: 193-220.
95. Schnepf, H., 1991, Robot ethology: a proposal for the research into intelligent autonomous systems. In J.-A. Meyers and S. W. Wilson (eds) From Animals to Animats, Proceedings of the First International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
96. Sheffield, F. D., Roby, T. B., and Campbell, B. A., 1954, Drive reduction verses consummatory behavior as determinants of reinforcement. Journal of Comparative and Physiological Psychology, 47: 471-481.
97. Siegel, S., 1989, Pharmacological conditioning and drug effects. In A.J. Goudie and M. W. Emmett-Oglesby (eds) Psychoactive Drugs: Tolerance and Sensitization (Clifton NJ: Humana Press).
98. Solomon, R., and Corbit, J., 1974, An opponent-process theory of motivation: I. The temporal dynamics of affect. Psychological Review, 81: 119-145.
99. Spetch, M. L., Wilkie, D. M., and Pine, J. P. J., 1981, Backward conditioning: a re-valuation of the empirical evidence. Psychological Bulletin, 89: 163-175.
100. Steels, L., 1995, The artificial life roots of artificial intelligence. In C. G. Langton (ed.) Artificial Life: An Overview (Cambridge MA: The MIT Press).
101. Sutton, R. S., 1988, Learning to predict by the methods of temporal differences. Machine Learning, 3: 9-44.
102. Sutton, R. S., and Barto, A. G., 1990, Time-derivative models of Pavlovian reinforcement. In M. Gabriel and J. Moore (eds) Learning and Computational Neuroscience: Foundations of Adaptive Networks (Cambridge MA: The MIT Press).
103. Tinbergen, N., 1951, The Study of Instinct (London: Oxford University Press).
104. Toates, F., 1995, Animal motivation and cognition. In H. Roitblat and J.-A. Meyer (eds) Comparative Approaches to Cognition (Cambridge MA: The MIT Press).
105. Toates, F., 1986, Motivational Systems (Cambridge: Cambridge University Press).
106. Tolman, E. C., 1955, Principles of performance. Psychological Review, 62: 315-326.
107. Tolman, E. C. 1949, There is more than one kind of learning. Psychological Review, 56: 144-155.
108. Tolman, E. C., 1938, The determinants of behavior at a choice point. Psychological Review, 45: 1-41.
109. Trapold, M., and Overmier, J., 1972, The second learning process in instrumental learning. In A. Black and W. Prokasy (eds) Classical Conditioning II: Current Research and Theory (New York: Appleton-Century-Crofts).
110. Tyrell, T., 1993, The use of hierarchies for action selection. In J.-A. Meyers and S. W. Wilson (eds) From Animals to Animats 2, Proceedings of the Second International Conference on the Simulation of Adaptive Behavior, (Cambridge MA: The MIT Press/Bradford Books).
111. Urcuioli, P. and DeMarse, T., 1996, Associative processes in differential outcome discriminations. Journal of Experimental Psychology: Animal Behavior Processes, 22: 192-204.
112. Verplanck, W. S., and Hayes, J. R., 1953, Eating and drinking as a function of maintenance schedule. Journal of Comparative and Physiological Psychology, 46: 327-333.
113. Watkins, C., 1992, Q-learning. Machine Learning, 8: 279-292.
114. Williams, D., and Williams, H., 1969, Automaintenance in the pigeon: sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12: 511-520.
115. Woods, S., Vasselli, J., and Milam, K., 1997, Iron appetite and latent learning in rats. Physiology and Behavior, 19: 623-626.
116. Wright, I., 1996, Reinforcement learning and animat emotions. In P. Maes, M. Mataric, J.-A. Meyer, J. Pollack and S. W. Wilson (eds) From Animals to Animats 4, Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior (Cambridge MA: The MIT Press/Bradford Books).
117. Zrehen, S., and Gaussier, P., 1997, Building grounded symbols for localisation using motivations. In P. Husbands and I. Harvey (eds) Proceeding of the Fourth European Conference on Artificial Life ECAL97, pp. 299-308 (Cambridge, MA: MIT Press).