Affect, anticipation, and adaptation: Affect-controlled selection of anticipatory simulation in artificial adaptive agents

Adaptive Behavior

Volumn 15, Issue 4, 2007, Pages 397-422

  Broekens, Joost ^a,b   Kosters, Walter A ^a   Verbeek, Fons J ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

^b Niels Bohrweg 1   (Netherlands)

Author keywords

Action selection; Affect; Anticipatory simulation; Simulated adaptive agents; Simulation selection; Working memory

Indexed keywords

EID: 35548939964     PISSN: 10597123     EISSN: 17412633     Source Type: Journal    
DOI: 10.1177/1059712307084686     Document Type: Article

References (61)

1. Ashby, F.G., Isen, A.M., & Turken, U. (1999). A neuropsychological theory of positive affect and its influence on cognition. Psychological Review, 106, 529 - 550.
2. Avila-Garcia, O., & Caamero, L. (2004). Using hormonal feedback to modulate action selection in a competitive scenario. From Animals to Animats 8: Proceedings of the 8th International Conference on Simulation of Adaptive Behavior (pp. 243-252). Cambridge, MA: MIT Press.
3. Aylett, R. (2006). Emotion as an integrative process between non-symbolic and symbolic systems in intelligent agents. Proceedings of the AISB'06 Symposium on Architecture of Brain and Mind (pp. 43-47). AISB Press.
4. Baddeley, A. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4, 417 - 423.
5. Belavkin, R.V. (2004). On relation between emotion and entropy. Proceedings of the AISB'04 Symposium on Emotion, Cognition and A fective Computing (pp. 1-8). AISB Press.
6. Berridge, K.C. (2003). Pleasures of the brain. Brain and Cognition, 52, 106 - 128.
7. Bickhard, M.H. (1998). Levels of representationality. Journal of Experimental and Theoretical Artificial Intelligence, 10, 179 - 215.
8. Blanchard, A.J., & Caamero, L. (2006). Modulation of exploratory behavior for adaptation to the context. Proceedings of the AISB'06 Symposium on Biologically Inspired Robotics (Biro-net) (pp 131-137). AISB Press.
9. Botelho, L.M., & Coelho, H. (1998). Information processing, motivation and decision making. Proceedings of the 4th International Workshop on Artificial Intelligence in Economics and Management, (pp. 233-250).
10. Broekens, J. (2005). Internal simulation of behavior has an adaptive advantage. Proceedings of the CogSci'05 Conference (pp. 342-347). Mahwah, NJ: Lawrence Erlbaum Associates.
11. Broekens, J., & DeGroot, D. (2004). Emergent representations and reasoning in adaptive agents. Proceedings of the Third International Conference on Machine Learning and Applications (pp. 207-214). Los Alamitos, CA: IEEE Press.
12. Broekens, J., Kosters, W.A., & Verbeek, F.J. (2007). On affect and self-adaptation: Potential benefits of valence-controlled action-selection. In: J. Mira and J.R. lvarez (Eds.), IWINAC 2007, Part I, LNCS 4527 (pp. 357-366). Berlin: Springer.
13. Broekens, J., & Verbeek, F.J. (2005). Simulation, emotion and information processing: Computational investigations of the regulative role of pleasure in adaptive behavior. Proceedings of the Workshop on Modeling Natural Action Selection (pp. 166-173). AISB Press.
14. Butz, M.V., Sigaud, O., & Gerard, P. (2003). Internal models and anticipations in adaptive learning systems. In: LNAI 2684: Anticipatory behavior in adaptive learning systems (pp. 86 - 109). Berlin: Springer.
15. Caamero, D. (2000). Designing emotions for activity selection. Department of Computer Science Technical Report DAIMI PB 545. University of Aarhus, Denmark.
16. Clore, G.L., & Gasper, K. (2000). Feeling is believing: Some affective influences on belief. In: N. Frijda, A. S. R. Manstead, & S. Bem (Eds.), Emotions and beliefs. Cambridge, UK: Cambridge University Press.
17. Cohen, J.D., & Blum, K.I. (2002) Reward and decision. Neuron, 36, 193 - 198.
18. Cos-Aguilera, I., Caamero, L., Hayes, G.M., & Gillies, A. (2005). Ecological integration of affordances and drives for behaviour selection. Proceedings of the Workshop on Modeling Natural Action Selection, Edinburgh: AISB Press, (pp. 225-228).
19. Cotterill, R.M.J. (2001). Cooperation of the basal ganglia, cerebellum, sensory cerebrum and hippocampus: Possible implications for cognition, consciousness, intelligence and creativity. Progress in Neurobiology, 64, 1 - 33.
20. Craig, S.D., Graesser, A.C., Sullins, J., & Gholson, B. (2004). Affect and learning: An exploratory look into the role of affect in learning with AutoTutor. Journal of Educational Media, 29, 241 - 250.
21. Custers, R., & Aarts, H. (2005). Positive affect as implicit motivator: On the nonconscious operation of behavioral goals. Journal of Personality and Social Psychology, 89, 129 - 142.
22. Damasio, A.R. (1994). Descartes' error: Emotion, reason, and the human brain. New York: Penguin Putnam.
23. Davidson, R.J. (2000). Cognitive neuroscience needs affective neuroscience (and vice versa). Brain and Cognition, 42, 89 - 92.
24. Dayan, P., & Balleine, B.W. (2002) Reward, motivation, and reinforcement learning. Neuron 36, 285 - 298.
25. Deheane, S., Sergent, C., & Changeux, J-P. (2003). A neuronal network model linking subjective reports and objective physiological data during conscious perception. Proceedings of the Natural Academy of Sciences, 100, 8520 - 8525.
26. Demiris, Y., & Johnson, M. (2003). Distributed, predictive perception of actions: A biologically inspired robotics architecture for imitation and learning. Connection Science, 15, 231 - 243.
27. Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15, 495 - 506.
28. Dreisbach, G., & Goschke, K. (2004). How positive affect modulates cognitive control: Reduced perseveration at the cost of increased distractibility. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 343 - 353.
29. Forgas, J.P. (2000). Feeling is believing? The role of processing strategies in mediating affective influences in beliefs. In: N. Frijda, A. S. R. Manstead, & S. Bem (Eds.), Emotions and beliefs. Cambridge, UK: Cambridge University Press.
30. Foster, D.J., & Wilson, M.A. (2006). Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature, 440, 680 - 683.
31. Frijda, N.H., Manstead, A.S.R., & Bem. S. (2000). The influence of emotions on beliefs. In: N. Frijda, A. S. R. Manstead, & S. Bem (Eds.), Emotions and beliefs. Cambridge, UK: Cambridge University Press.
32. Frijda, N.H., & Mesquita, B. (2000). Beliefs through Emotions. In: N. Frijda, A. S. R. Manstead, & S., Bem, (Eds.), Emotions and beliefs. Cambridge, UK: Cambridge University Press.
33. Gadanho, S.C. (2003). Learning behavior-selection by emotions and cognition in a multi-goal robot task. Journal of Machine Learning Research, 4, 385 - 412.
34. Griffith, P.E. (1999). Modularity & the psychoevolutionary theory of emotion. Mind and Cognition: An Anthology. Oxford: Blackwell.
35. Hesslow, G. (2002). Conscious thought as simulation of behaviour and perception. Trends in Cognitive Sciences, 6, 242 - 247.
36. Hoffmann, H., & Mller, R. (2004). Action selection and mental transformation based on a chain of forward models. Proceedings of the 8th International Conference on the Simulation of Adaptive Behavior (pp. 213-222). Cambridge, MA: MIT Press.
37. Hogewoning, E., Broekens, J., Eggermont, J., & Bovenkamp, E.G.P. (2007). Strategies for affect-controlled action-selection in Soar-RL. In: J. Mira and J.R. lvarez (Eds.), IWINAC 2007, Part II, LNCS 4528 (pp. 501-510). Berlin: Springer.
38. Kaelbling, L.P., Littman, M.L., & Moore, A.W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237 - 285.
39. Lahnstein, M. (2005). The emotive episode is a composition of anticipatory and reactive evaluations. Proceedings of the AISB'05 Symposium on Agents that Want and Like (pp. 62-69). AISB Press.
40. McCallum, A. (1995). Instance-based utile distinctions for reinforcement learning with hidden state. Proceedings of the Twelfth International Conference on Machine Learning, San Francisco: Morgan Kaufmann, (pp. 387-395).
41. McMahon, A., Scott, D., Baxter, P., & Browne, W. (2006). An autonomous explore/exploit strategy. Proceedings of the AISB'06 Symposium on Nature Inspired Systems (pp. 192- 201). AISB Press.
42. Montague, P.R., Hyman S.E., & Cohen, J.D. (2004). Computational roles for dopamine in behavioural control. Nature, 431, 760 - 767.
43. Nason, S., & Laird, J.E. (2005). Soar-RL, integrating reinforcement learning with soar. Cognitive Systems Research, 6, 51 - 59.
44. Phaf, R.H., & Rotteveel, M. (2005). Affective modulation of recognition bias. Emotion, 5, 309 - 318.
45. Picard, R.W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover, T., Resnick, M., Roy, D., & Strohecker, C. (2004). Affective learning-A manifesto. BT Technology Journal, 22, 253 - 269.
46. Rolls, E.T. (2000). Prcis of The brain and emotion. Behavioral and Brain Sciences, 23, 177 - 191.
47. Rose, S.A., Futterweit, L.R., & Jankowski, J.J. (1999). The relation of affect to attention and learning in infancy. Child Development, 70, 549 - 559.
48. Russell, J.A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145 - 172.
49. Salichs, M.A., & Malfaz, M. (2006). Using emotions on autonomous agents. The role of happiness, sadness and fear. Proceedings of the AISB'06 Symposium on Integrative Approaches to Machine Consciousness (pp. 157-164). AISB Press.
50. Scherer, K.R. (2001). Appraisal considered as a process of multilevel sequential checking. In: K. R. Scherer, A. Schott, & T. Johnstone (Eds.) Appraisal processes in emotion: Theory, Methods, Research. New York: Oxford Univ. Press,.
51. Schultz, W., Dayan, P., & Montague, P.R. (1997). A neural substrate of prediction and reward. Science, 275, 1593 - 1599.
52. Schweighofer, N., & Doya, K. (2003). Meta-learning in reinforcement learning. Neural Networks, 16, 5 - 9.
53. Shanahan, M. (2006). A cognitive architecture that combines internal simulation with a global workspace. Consciousness and Cognition, 15, 433 - 449.
54. Sutton, R.S. (1990). Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. Proceedings of the Seventh International Conference on Machine Learning (pp. 216-224). Morgan Kaufmann.
55. Sutton, R.S., & Barto, A.G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
56. Tyrell, T. (1993). Computational mechanisms for action selection. Ph.D. Thesis, University of Edinburgh.
57. van Dartel, M.F., & Postma, E.O. (2005) Symbol manipulation by internal simulation of perception and behaviour. Proceedings of the 5th International Workshop on Epigenetic Robotics. Lund University Cognitive Studies Vol. 123 (pp. 121 - 124). Lund: Lund University.
58. van Dartel, M., Postma, E. & van den Herik, J. (2004) Categorisation through internal simulation of perception and behaviour. Proceedings of the 16th Belgium-Netherlands Conference on Artificial Intelligence (pp. 187-194).
59. von Hecker, U., & Meiser, T., (2005). Defocused attention in depressed mood: Evidence from source monitoring. Emotion, 5, 456 - 463.
60. Velasquez, J.D. (1998). A computational framework for emotion-based control. In: SAB'98 Work-shop on Grounding Emotions in Adaptive Systems, 5th International Conference of the Society for Adaptive Behavior, Zurich.
61. Ziemke, T., Jirenhed, D., & Hesslow, G. (2005). Internal simulation of perception: A minimal neuro-robotic model. Neurocomputing, 68, 85 - 104.