What is intrinsic motivation? A typology of computational approaches

Frontiers in Neurorobotics

Volumn 3, Issue NOV, 2009, Pages

  Oudeyer, Pierre Yves ^a,b   Kaplan, Frederic ^c  

^a SONY COMPUTER SCIENCE LABORATORY   (France)

^b INRIA   (France)

^c EPFL   (Switzerland)

Author keywords

Artificial intelligence; Cognitive development; Computational modeling; Curiosity; Developmental robotics; Exploration; Intrinsic motivation; Reinforcement learning; Reward

Indexed keywords

EID: 84891105730     PISSN: None     EISSN: 16625218     Source Type: Journal    
DOI: 10.3389/neuro.12.006.2007     Document Type: Article

References (47)

1. Arkin, R. (2005). Moving up the food chain: motivation and emotion in behavior based robots. In Who Needs Emotions: The Brain Meets the Robot, J. Fellous and M. Arbib, eds (Oxford University Press), pp. 245-270.
2. Arkin, R., Cervantes-Perez, F., and Weitzenfeld, A. (1998). Ecological robotics: a schema-theoretic approach. In Intelligent Robots: Sensing, Modelling and Planning, R. Bolles, H. Bunke and H. Noltemeier, eds (Singapore, World Scientific), pp. 377-393.
3. Barto, A., and Simsek, O. (2005). Intrinsic motivation for reinforcement learning systems. In Proceedings of the Thirteenth Yale Workshop on Adaptive and Learning Systems, New Haven, CT, Yale University.
4. rd International Conference on Development and Learning (ICDL 2004). Salk Institute, San Diego.
5. Berlyne, D. (1960). Conflict, Arousal and Curiosity. New York, NY, McGraw-Hill.
6. Bonarini, A., Lazaric, A., and Restelli, M. (2006). Self-development frame work for reinforcement learning agents. Proceedings of the Fifth International Conference on Development and Learning, Bloomington, IN, USA.
7. Braitenberg, V. (1984). Vehicles: Experiments in Synthetic Psychology. Cambridge, MA, Bradford Books/MIT Press.
8. Breazeal, C. (2002). Designing Sociable Robots. Cambridge, MA, Bradford Books/MIT Press.
9. Crutchfield, J. P. (1990). Information and its metric. In Nonlinear Structures in Physical Systems - Pattern Formation, Chaos, and Waves, L. Lam and H. C. Morris, eds (New York, NY, Springer Verlag), pp. 119-130.
10. Csikszentmihalyi, M. (1991). Flow: The Psychology of Optimal Experience. New York, NY, Harper Perennial.
11. De Charms, R. (1968). Personal Causation: The Internal Affective Determinants of Behavior. New York, NY, Academic Press.
12. Deci, E., and Ryan, R. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. New York, NY, Plenum Press.
13. Dember, W. N., and Earl, R. W. (1957). Analysis of exploratory, manipulatory and curiosity behaviors. Psychol. Rev. 64, 91-96.
14. Endo, Y., and Arkin, R. (2001). Implementing tolman's schematic sowbug: behavior-based robotics in the 1930's. Proceedings of the IEEE International Conference on Robotics and Automation, Seoul, Korea.
15. Fedorov, V. (1972). Theory of Optimal Experiment. New York, NY, Academic Press.
16. Festinger, L. (1957). A Theory of Cognitive Dissonance. Evanston, Row, Peterson.
17. Fujita, M., Costa, G., Takagi, T., Hasegawa, R., Yokono, J., and Shimomura, H. (2001). Experimental results of emotionally grounded symbol acquisition by four-legged robot. In Proceedings of Autonomous Agents 2001, J. Muller, ed. Montreal, Canada.
18. Gogate, L. J., and Bahrick, L. (1998). Intersensory redundancy of kinematic primitives for visual speech perception facilitates learning of arbitrary relations between vowel sounds and objects in seven-month-old infants. J. Exp. Child Psychol. 69, 133-149.
19. Harlow, H. (1950). Learning and satiation of response in intrinsically motivated complex puzzle performances by monkeys. J. Comp. Physiol. Psychol. 43, 289-294.
20. Hershey, J., and Movellan, J. (2000). Audio-vision: using audio-visual synchrony to locate sounds. In Advances in Neural Information Processing Systems 12, T. Solla and K.-R. Muller, eds (Cambridge, MA, MIT Press).
21. Huang, X., and Weng, J. (2002). Novelty and reinforcement learning in the value system of developmental robots. In Proceedings of the 2nd International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Lund University Cognitive Studies, Vol. 94, C. Prince, Y. Demiris, Y. Marom, H. Kozima and C. Balkenius, eds (Edinburgh, Scotland, Lund University), pp. 47-55.
22. Huang, X., and Weng, J. (2004). Motivational system for human-robot interaction in Proceedings of the ECCV International Workshop on Human-Computer Interaction, Prague.
23. Hull, C. L. (1943). Principles of Behavior: An Introduction to Behavior Theory. New York, NY, Appleton-Century-Croft.
24. Hunt, J. M. (1965). Intrinsic motivation and its role in psychological development. Nebr. Symp. Motiv. 13, 189-282.
25. Kagan, J. (1972). Motives and development. J. Pers. Soc. Psychol. 22, 51-66.
26. Kaplan, F., and Oudeyer, P.-Y. (2003). Motivational principles for visual know-how development. In Proceedings of the 3rd International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Lund University Cognitive Studies, Vol. 101, C. Prince, L. Berthouze, H. Kozima, D. Bullock, G. Stojanov and C. Balkenius, eds (Boston, USA, Lund University), pp. 73-80.
27. Kaplan, F., and Oudeyer, P.-Y. (2007a). In search of the neural circuits of intrinsic motivation. Front. Neurosci. 1, 225-236.
28. Kaplan, F., and Oudeyer, P.-Y. (2007b). The progress-drive hypothesis: an interpretation of early imitation. In Models and Mechanisms of Imitation and Social Learning: Behavioural, Social and Communication Dimensions, C. Nehaniv and K. Dautenhahn, eds (New York, Cambridge University Press), pp. 361-377.
29. Konidaris, G., and Barto, A. (2006). An adaptive robot motivational system. In From Animals to Animats 9: Proceedings of the 9th International Conference on Simulation of Adaptive Behavior (Roma, Italy, SAB-06).
30. rd International Conference on Development and Learning (ICDL 2004). Salk Institute, San Diego.
31. McFarland, D., and Bosser, T. (1994). Intelligent Behavior in Animals and Robots. Cambridge, MA, MIT Press.
32. Merrick, K., and Maher, M.-L. (2008). Motivated learning from interesting events: adaptive, multitask learning agents for complex environments. Adapt. Behav. (in press).
33. Montgomery, K. (1954). The role of exploratory drive in learning. J. Comp. Physiol. Psychol. 47, 60-64.
34. Oudeyer, P.-Y., and Kaplan, F. (2006). Discovering communication. Connect. Sci. 18, 189-206.
35. Oudeyer, P.-Y., Kaplan, F., and Hafner, V. (2007). Intrinsic motivation systems for autonomous mental development. IEEE Trans. Evol. Comput. 11, 265-286.
36. Oudeyer, P.-Y., Kaplan, F., Hafner, V. V., and Whyte, A. (2005). The playground experiment: task-independent development of a curious robot. In Proceedings of the AAAI Spring Symposium on Developmental Robotics, 2005, D. Bank and L. Meeden, eds (Stanford, AAAI), pp. 42-47.
37. Prince, C., Hollich, G., Helder, N., Mislivec, E., Reddy, A., Salunke, S., and Memon, N. (2003). Taking synchrony seriously: a perceptual-level model of infant synchrony detection. In Proceedings of the Fourth International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems, Lund University Cognitive Studies, Vol. 117, L. Berthouze, H. Kozima, C. Prince, G. Sandini, G. Stojanov, G. Metta and C. Balkenius, eds (Edinburgh, Scotland, Lund University).
38. Rochat, P., and Striano, T. (2000). Perceived self in infancy. Infant Behav. Dev. 23, 513-530.
39. Roy, N., and McCallum, A. (2001). Towards optimal active learning through sampling estimation of error reduction. In Proceedings of the 18th International Conference on Machine Learning. Williamstown, MA, USA, Morgan Kaufmann Publishers Inc.
40. Ryan, R. M., and Deci, E. L. (2000). Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp. Educ. Psychol. 25, 54-67.
41. Schmidhuber, J. (1991). Curious model-building control systems. In Proceedings of the International Joint Conference on Neural Networks, Vol. 2. Singapore, IEEE, pp. 1458-1463.
42. Skinner, B. (1953). Science and Human Behavior. New York, NY, Macmillan.
43. Sutton, R., and Barto, A. (1998). Reinforcement Learning: An Introduction. Cambridge, MA, MIT Press.
44. Sutton, R., Precup, D., and Singh, S. (1999). Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artif. Intell. 112, 181-211.
45. Thrun, S. (1995). Exploration in active learning. In Handbook of Brain Science and Neural Networks, M. Arbib, ed (Cambridge, MA, MIT Press).
46. Watson, J. S. (1972). Smiling, cooing, and the game. Merrill Palmer Q. 18, 323-339.
47. White, R. (1959). Motivation reconsidered: the concept of competence. Psychol. Rev. 66, 297-333.