R-IAC: Robust intrinsically motivated exploration and active learning

IEEE Transactions on Autonomous Mental Development

Volumn 1, Issue 3, 2009, Pages 155-169

  Baranès, Adrien ^a   Oudeyer, Pierre Yves ^a  

^a FLOWERS Team   (France)

Author keywords

Active learning; Artificial curiosity; Developmental robotics; Exploration; Intrinsic motivation; Sensorimotor learning

Indexed keywords

ACTIVE LEARNING; ACTIVE-LEARNING ALGORITHM; CONTROL SCHEMES; DEVELOPMENTAL MECHANISMS; DEVELOPMENTAL ROBOTICS; DEVELOPMENTAL STAGE; FORWARD MODELS; INTRINSIC MOTIVATION; OPEN SOURCES; SELF-ORGANIZE;

MOTIVATION; ROBOTICS; ROBOTS;

LEARNING ALGORITHMS;

EID: 77955701366     PISSN: 19430604     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAMD.2009.2037513     Document Type: Article

References (63)

1. J. Weng, J. McClelland, A. Pentland, and O. Sporns et al., "Autonomous mental development by robots and animals," Science, vol.291, pp. 599-600, 2001.
2. M. Lungarella, G. Metta, R. Pfeifer, and G. Sandini, "Developmental robotics: A survey," Connect. Sci., vol.15, no.4, pp. 151-190, 2003.
3. S. Calinon, F. Guenter, and A. Billard, "On learning, representing and generalizing a task in a humanoid robot," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.37, no.2, pp. 286-298, Mar. 2007. (Pubitemid 46523219)
4. M. Lopes, F. S. Melo, and L. Montesano, "Affordance-based imitation learning in robots," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., San Diego, CA, 2007, pp. 1015-1021.
5. P. Abbeel and A. Y. Ng, "Apprenticeship learning via inverse reinforcement learning," in Proc. 21st Int. Conf. Mach. Learn., Alberta, Canada, 2004, pp. 1-8.
6. C. G. Atkeson and S. Schaal, "Robot learning from demonstration," in Proc. 14th Int. Conf. Mach. Learn., Nashville, TN, 1997, pp. 12-20.
7. A. Alissandrakis, C. L. Nehaniv, and K. Dautenhahn, "Action, state and effect metrics for robot imitation," in Proc. 15th IEEE Int. Symp. Robot Human Interact. Comm., Hatfield, U.K., 2006, pp. 232-237.
8. B. Argall, S. Chernova, and M. Veloso, "A survey of robot learning from demonstration," Robot. Autonom. Syst., vol.57, no.5, pp. 469-483, 2009.
9. M. Asada, M. Ogino, S. Matsuyama, and J. Oga, "Imitation learning based on visuo-somatic mapping," in Proc. 9th Int. Symp. Exp. Robot., Germany, 2006, vol.21, pp. 269-278.
10. P. Andry, P. Gaussier, S. Moga, J. P. Banquet, and J. Nadel, "Learning and communication via imitation: An autonomous robot perspective," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol.31, no.5, pp. 431-442, Sep. 2001.
11. Y. Demiris and A. Meltzoff, "The robot in the crib: A developmental analysis of imitation skills in infants and robots," Inf. Child Develop., vol.17, pp. 43-53, 2008.
12. M. Pardowitz, S. Knoop, R. D. Zollner, and R. Dillmann, "Incremental learning of tasks from user demonstrations, past experiences, and vocal comments," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol.37, no.2, pp. 322-332, Mar. 2007.
13. E. Oztop, M. Kawato, and M. Arbib, "Mirror neurons and imitation: A computationally guided review," Neural Netw., vol.19, no.3, pp. 254-271, 2006.
14. R. Rao, A. Shon, and A. Meltzoff, "A Bayesian model of imitation in infants and robots," in Imitation and Social Learning in Robots, Humans, and Animals. Cambridge, U.K.: Cambridge Univ. Press, 2007, pp. 217-248.
15. R. C. Arkin, "Moving up the food chain: Motivation and emotion in behavior-based robots," in Who Needs Emotions: The Brain Meets the Robot, J. Fellous and M. Arbib, Eds. London, U.K.: Oxford Univ. Press, 2005, pp. 245-269.
16. M. Fellous and M. Arbib, Eds., Who Needs Emotions: The Brain Meets the Robot. London, U.K.: Oxford Univ. Press, 2005.
17. D. McFarland and T. Bosser, Intelligent Behavior in Animals and Robots. Cambridge, MA: MIT Press, 1993.
18. R. Manzotti and V. Tagliasco, "From behaviour-based robots to motivation- based robots," Robot. Autonom. Syst., vol.51, no.2-3, pp. 175-190, 2005.
19. A. Stoytchev and R. Arkin, "Incorporating motivation in a hybrid robot architecture," J. Adv. Comput. Intel. Intell. Informat., vol.8, no.3, pp. 269-274, 2004.
20. R. C. Arkin, M. Fujita, T. Takagi, and R. Hasegawa, "An ethological and emotional basis for human-robot interaction," Robot. Autonom. Syst., vol.42, no.3, pp. 191-201, 2003.
21. R. White, "Motivation reconsidered: The concept of competence," Psychology, vol.66, pp. 297-333, 1959.
22. D. Berlyne, "Curiosity and exploration," Science, vol.153, no.3731, pp. 25-33, 1966.
23. E. Deci and R. Ryan, Intrinsic Motivation and Self-Determination in Human Behavior. New York: Plenum, 1985.
24. W. Schultz, "Getting formal with dopamine and reward," Neuron, vol.36, pp. 241-263, 2002.
25. P. Dayan and B. Balleine, "Reward, motivation and reinforcement learning," Neuron, vol.36, pp. 285-298, 2002.
26. P. Redgrave and K. Gurney, "The short-latency dopamine signal: A role in discovering novel actions?," Nature Rev. Neurosci., vol.7, no.12, pp. 967-975, 2006.
27. P.-Y. Oudeyer, F. Kaplan, and V. Hafner, "Intrinsic motivation systems for autonomous mental development," IEEE Trans. Evol. Comput., vol.11, no.2, pp. 265-286, Apr. 2007.
28. A. Barto, S. Singh, and N. Chentanez, "Intrinsically motivated learning of hierarchical collections of skills," in Proc. 3rd Int. Conf. Develop. Learn., San Diego, CA, 2004, pp. 112-119.
29. A. Blanchard and L. Cañamero, "Modulation of exploratory behavior for adaptation to the context," in Proc. Biol. Inspired Robot. AISB'06: Adapt. Artif. Biol. Syst., Bristol, U.K., 2006.
30. R. Der, M. Herrmann, and R. Liebscher, "Homeokinetic approach to autonomous learning in mobile robots," in Proc. Robotik, R. Dillman, R. D. Schraft, and H. Wörn, Eds., Dusseldorf, Germany, 2002, pp. 301-306, VDI.
31. D. S. Blank, D. Kumar, L. Meeden, and J. Marshall, "Bringing up robot: Fundamental mechanisms for creating a self-motivated, self-organizing architecture," Cybern. Syst., vol.36, no.2, pp. 125-150, 2005.
32. X. Huang and J. Weng, "Novelty and reinforcement learning in the value system of developmental robots," in Proc. 2nd Int. Workshop Epig. Robot.: Modeling Cogn. Develop. Robot. Syst., Edinburgh, Scotland, Aug. 10-11, 2002, pp. 47-55.
33. J. Schmidhuber, "Curious model-building control systems," in Proc. Int. Joint Conf. Neural Netw., Singapore, 1991, vol.2, pp. 1458-1463.
34. P.-Y. Oudeyer and F. Kaplan, "Discovering communication," Connect. Sci., vol.18, no.2, pp. 189-206, 2006.
35. M. Schembri, M. Mirolli, and G. Baldassarre, "Evolution and learning in an intrinsically motivated reinforcement learning robot," in Proc. Euro. Conf. Artif. Life, 2007, pp. 294-303.
36. F. Kaplan and P.-Y. Oudeyer, "Intrinsically motivated machines," in 50 Years of AI, Festschrift, M. Lungarella, F. Iida, J. Bongard, and R. Pfeifer, Eds. : Springer-Verlag, 2007, vol.4850, LNAI, pp. 304-315.
37. V. Fedorov, Theory of Optimal Experiment. New York: Academic, 1972.
38. E. J. Gibson, Principles of Perceptual Learning and Development. New York: Appleton-Century-Crofts, 1969.
39. D. Berlyne, Conflict, Arousal, and Curiosity. New York: McGraw- Hill, 1960.
40. M. Csikszentmihalyi, Creativity-Flow and the Psychology of Discovery and Invention. New York: Harper Perennial, 1996.
41. D. Cohn, Z. Ghahramani, and M. Jordan, "Active learning with statistical models," J. Artif. Intell. Res., vol.4, pp. 129-145, 1996.
42. M. Hasenjager and H. Ritter, "Active learning in neural networks," in New Learning Paradigms in Soft Computing. Berlin, Germany: Physica-Verlag GmbH, 2002, pp. 137-169.
43. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. Hoboken, NJ: Wiley, 2006.
44. S. Vijayakumar and S. Schaal, "LWPR: An O(n) algorithm for incremental real time learning in high dimensional space," in Proc. 17th Int. Conf. Mach. Learn., Stanford, CA, 2000, pp. 1079-1086.
45. A. D'Souza, S. Vijayakumar, and S. Schaal, "Learning inverse kinematics," in IEEE Int. Conf. Intell. Robots Syst., Piscataway, NJ, 2001, pp. 298-303.
46. J. Peters and S. Schaal, "Learning to control in operational space," Int. J. Robot. Res., vol.27, pp. 197-212, 2008.
47. C. Salaün, V. Padois, and O. Sigaud, "Control of redundant robots using learned models: An operational space control approach," in Proc. IEEE Int. Conf. Intell. Robots Syst., Piscataway, NJ, 2009, pp. 878-885.
48. D. Y. Yeung and Y. Zhang, "Learning inverse dynamics by Gaussian process regression under the multi-task learning framework," in The Path to Autonomous Robots, G. S. Sukhatme, Ed. New York: Springer-Verlag, 2009, pp. 131-142.
49. Z. Ghahramani, "Solving inverse problems using an EM approach to density estimation," in Proc. 1993 Connectionist Models Summer School, M. C. Mozer, P. Smolensky, D. S. Toureztky, J. L. Elman, and A. S. Weigend, Eds. Hillsdale, NJ: Erlbaum Associates, 1993, pp. 316-323.
50. C. E. Rasmussen, "Evaluation of Gaussian Process and Other Methods for Non-Linear Regression," PhD thesis, Dept. Comp. Sci., Univ. Toronto, Toronto, ON, Canada, 1996.
51. S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, "An optimal algorithm for approximate nearest neighbor searching," J. Assoc. Comput. Mach., vol.45, pp. 891-923, 1998.
52. S. Maneewongvatana and D. M. Mount, "Analysis of approximate nearest neighbor searching with clustered point sets, data structures, near neighbor searches, and methodology," in Fifth and Sixth DIMACS Implementation Challenges, M. H. Goldwasser, D. S. Johnson, and C. C. McGeoch, Eds., 2002, vol.59, DIMACS Series in Discr. Math. and Theor. Comput. Sci., pp. 105-123, AMS.
53. D. Filliat, "A visual bag of words method for interactive qualitative localization and mapping," in Proc. Int. Conf. Robot. Autom., 2007, pp. 3921-3926.
54. P. I. Corke, "A robotics toolbox for matlab," IEEE Robot. Autom. Mag., vol.1, no.3, pp. 24-32, Sep. 2006.
55. P.-Y. Oudeyer and F. Kaplan, "How can we define intrinsic motivation?," in Proc. 8th Int. Conf. Epig. Robot.: Modeling Cogn. Develop. Robot. Syst., Brighton, 2008.
56. Y. Kuniyoshi, Y. Yorozu, M. Inaba, and H. Inoue, "From visuo-motor self learning to early imitation-a neural architecture for humanoid learning," in Proc. IEEE Int. Conf. Robot. Autom., 2003, vol.3, pp. 3132-3139.
57. M. Lopes, F. Mello, L. Montesano, and J. Santos-Victor, "Cognitive processes in imitation: Overview and computational approaches," in From Motor to Interaction Learning in Robots, O. Sigaud and J. Peters, Eds. New York: Springer LNCS, (unpublished).
58. A. L. Thomaz and C. Breazeal, "Experiments in socially guided exploration: Lessons learned in building robots that learn with and without human teachers," Connect. Sci., vol. 20, no. 2&3, pp. 91-110, 2008.
59. F. Kaplan, P.-Y. Oudeyer, and B. Bergen, "Computational models," Debate Lang. Learnability, Inf. Child Develop., vol.17, no.1, pp. 55-80, 2008.
60. D. Koch and A. Billard, "Gaussian mixture regression and its application in robotics," in Proc. Workshop Regress. Robot., 2009.
61. D. Nguyen-Tuong and J. Peters, "Local Gaussian processes regression for real-time model-based robot control," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Piscataway, NJ, 2008, pp. 380-385.
62. D. Y. Yeung and Y. Zhang, "Learning inverse dynamics by Gaussian process regression under the multi-task learning framework," in The Path to Autonomous Robots, G. S. Sukhatme, Ed. New York: Springer-Verlag, 2009, pp. 131-142.
63. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines 2001 [Online]. Available: http://www.csie.ntu.edu.tw/ cjlin/ libsvm