Fast damage recovery in robotics with the T-resilience algorithm

International Journal of Robotics Research

Volumn 32, Issue 14, 2013, Pages 1700-1723

  Koos, Sylvain ^a   Cully, Antoine ^a   Mouret, Jean Baptiste ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Damage recovery; evolutionary algorithm; fault tolerance; hexapod locomotion; learning; long term autonomy; resilience; transferability

Indexed keywords

DAMAGE RECOVERY; LEARNING; LONG-TERM AUTONOMY; RESILIENCE; TRANSFERABILITY;

EVOLUTIONARY ALGORITHMS; FAULT TOLERANCE; RECOVERY; ROBOTICS; ROBOTS;

ALGORITHMS;

EID: 84882424717     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364913499192     Document Type: Article

References (93)

1. Argall B, Chernova S, Veloso M, et al. A survey of robot learning from demonstration. Robotics and Autonomous Systems. 2009 ; 57 (5). 469-483
2. Barfoot T, Earon E, D'Eleuterio G. Experiments in learning distributed control for a hexapod robot. Robotics and Autonomous Systems. 2006 ; 54 (10). 864-872
3. Bellingham JG, Rajan K. Robotics in remote and hostile environments. Science. 2007 ; 318 (5853). 1098-1102
4. Berenson D, Estevez N, Lipson H. Hardware evolution of analog circuits for in-situ robotic fault-recovery. Proceedings of NASA/DoD conference on evolvable hardware ; 2005 ; 2005. 12.
5. Bongard J. Action-selection and crossover strategies for self-modeling machines. Proceedings of genetic and evolutionary computation conference (GECCO) ; 2007 ; 2007. 198.
6. Bongard J, Lipson H. Nonlinear system identification using coevolution of models and tests. IEEE Transactions on Evolutionary Computation. 2005 ; 9 (4). 361-384
7. Bongard J, Zykov V, Lipson H. Resilient machines through continuous self-modeling. Science. 2006 ; 314 (5802). 1118-1121
8. Caccavale F, Villani L Fault Diagnosis and Fault Tolerance for Mechatronic Systems: Recent Advances. New York, NY: Springer ; 2002 :
9. Cantu-Paz E Efficient and Accurate Parallel Genetic Algorithms. Norwell, MA: Kluwer Academic Publishers ; 2000 :
10. Chang C, Lin C. Libsvm: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology. 2011 ; 2 (3). 27
11. Chernova S, Veloso M. An evolutionary approach to gait learning for four-legged robots. Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2004 ; 2004. 2562.
12. Clune J, Stanley K, Pennock R, et al. On the performance of indirect encoding across the continuum of regularity. IEEE Transactions on Evolutionary Computation. 2011 ; 15 (3). 346-367
13. Connell J, Mahadevan S Robot Learning. New York, NY: Springer ; 1993 :
14. Corbato F. On building systems that will fail. ACM Turing Award Lectures. 2007 ; 34 (9). 72-81
15. Cully A, Mouret JB. Behavioral repertoire learning in robotics. Proceedings of genetic and evolutionary computation conference (GECCO) ; 2013 a ; 2013 a. 175.
16. CullyAMouretJB (2013 b) Learning to walk in every direction. Available at: http://arxiv.org/abs/1308.3689.
17. De Jong K Evolutionary Computation: A Unified Approach. Cambridge, MA: MIT Press ; 2006 :
18. Deb K Multi-Objective Optimization Using Evolutionary Algorithms. New York, NY: John Wiley and Sons ; 2001 :
19. Deb K, Pratap A, Agarwal S, et al. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation. 2002 ; 6 (2). 182-197
20. Delcomyn F. The locomotion of the cockroach Periplaneta americana. Journal of Experimental Biology. 1971 ; 54 (2). 443-452
21. Ding X, Wang Z, Rovetta A, et al. Locomotion analysis of hexapod robot. Proceedings of conference on climbing and walking robots (CLAWAR) ; 2010 ; 2010. 291.
22. Doncieux S, Mouret J-B, Bredeche N, et al. Evolutionary robotics: Exploring new horizons. New Horizons in Evolutionary Robotics: Extended Contributions from the 2009 EvoDeRob Workshop ; 2011 New York, NY: Springer ; 2011. 3.
23. Endres F, Hess J, Engelhard N, et al. An evaluation of the RGB-D SLAM system. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2012 ; 2012..
24. Goldberg K, Chen B. Collaborative control of robot motion: Robustness to error. Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS) ; 2001 ; 2001..
25. Görner M, Hirzinger G. Analysis and evaluation of the stability of a biologically inspired, leg loss tolerant gait for six- and eight-legged walking robots. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2010 ; 2010. 4728.
26. Grefenstette JJ, Schultz AC, Moriarty DE. Evolutionary algorithms for reinforcement learning. Journal of Artificial Intelligence Research. 1999 ; 11: 241-276
27. Hartland C, Bredeche N. Evolutionary robotics, anticipation and the reality gap. IEEE international conference on robotics and biomimetics ; 2006 ; 2006. 1640.
28. Heidrich-Meisner V, Igel C. Neuroevolution strategies for episodic reinforcement learning. Journal of Algorithms. 2009 ; 64 (4). 152-168
29. Hemker T, Stelzer M, Von Stryk O, et al. Efficient walking speed optimization of a humanoid robot. The International Journal of Robotics Research. 2009 ; 28 (2). 303-314
30. Hoffmann M, Marques H, Arieta A, et al. Body schema in robotics: A review. IEEE Transactions on Autonomous Mental Development. 2010 ; 2 (4). 304-324
31. Holland O, Goodman R. Robots with internal models: A route to machine consciousness?. Journal of Consciousness Studies. 2003 ; 10 (4-5). 4-5
32. Hoos HH, Stützle T Stochastic Local Search: Foundations And Applications. Burlington, MA: Morgan Kaufmann ; 2005 :
33. Hornby G, Takamura S, Yamamoto T, et al. Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Transactions on Robotics. 2005 ; 21 (3). 402-410
34. Hornby GS, Lohn JD, Linden DS. Computer-automated evolution of an X-band antenna for NASA's Space Technology 5 mission. Evolutionary Computation. 2011 ; 19 (1). 1-23
35. Jakimovski B, Maehle E Climbing and Walking Robots. Miripour B, ed. Rijeka, Croatia: InTech ; 2010 :
36. Jakobi N, Husbands P, Harvey I. Noise and the reality gap: The use of simulation in evolutionary robotics. Proceedings of the European conference on artificial life (ECAL) ; 1995 ; 1995. 704.
37. Kajita S, Espiau B Handbook of Robotics. Siciliano B Khatib O, ed. New York, NY: Springer ; 2008: 361-389.
38. Katić D, Vukobratović M. Survey of intelligent control techniques for humanoid robots. Journal of Intelligent & Robotic Systems. 2003 ; 37 (2). 117-141
39. Kimura H, Yamashita T, Kobayashi S. Reinforcement learning of walking behavior for a four-legged robot. Proceedings of IEEE conference on decision and control (CDC) ; 2001 ; 2001. 411.
40. Klaus G, Glette K, Tørresen J. A comparison of sampling strategies for parameter estimation of a robot simulator. Simulation, Modeling, and Programming for Autonomous Robots. 2012 ; 7628: 173-184
41. Kober J, Peters J. Imitation and reinforcement learning - practical learning algorithms for motor primitives in robotics. IEEE Robotics and Automation Magazine. 2010 ; 17 (2). 1-8
42. Kober J, Peters J Adaptation, Learning, and Optimization (ALO). New York, NY: Springer ; 2012: 579-610.
43. Kohl N, Stone P. Policy gradient reinforcement learning for fast quadrupedal locomotion. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2004 ; 2004. 2619.
44. Koos S, Mouret JB. Online discovery of locomotion modes for wheel-legged hybrid robots: A transferability-based approach. Proceedings of the conference on climbing and walking robots (CLAWAR) ; 2011 ; 2011. 70.
45. Koos S, Mouret JB, Doncieux S. The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation. 2013 ; 17 (1). 22-145
46. Koren I, Krishna CM Fault-Tolerant Systems. Burlington, MA: Morgan Kaufmann ; 2007 :
47. Lin CM, Chen CH. Robust fault-tolerant control for a biped robot using a recurrent cerebellar model articulation controller. Systems, Man, and Cybernetics, Part B: Cybernetics. 2007 ; 37 (1). 110-123
48. Mahdavi S, Bentley P. An evolutionary approach to damage recovery of robot motion with muscles. Advances in Artificial Life. 2003 2801 ;: 248-255
49. Mahdavi S, Bentley P. Innately adaptive robotics through embodied evolution. Autonomous Robots. 2006 ; 20 (2). 149-163
50. Metzinger T Being No One: The Self-Model Theory Of Subjectivity. Cambridge, MA: MIT Press ; 2004 :
51. Metzinger T. Self models. Scholarpedia. 2007 ; 2 (10). 4174
52. Moore GE. Progress in digital integrated electronics. International electron devices meeting ; 1975 ; 1975. 11.
53. Mostafa K, Tsai C, Her I. Alternative gaits for multiped robots with leg failures to retain maneuverability. International Journal of Advanced Robotic Systems. 2010 ; 7 (4). 31
54. Mouret JB, Doncieux S. Sferes: Evolvin' in the multi-core world. Proceedings of the IEEE congress on evolutionary computation (CEC) ; 2010 ; 2010. 4079.
55. Mouret JB, Doncieux S. Encouraging behavioral diversity in evolutionary robotics: An empirical study. Evolutionary Computation. 2012 ; 20 (1). 91-133
56. Mouret JB, Koos S, Doncieux S. Crossing the reality gap: A short introduction to the transferability approach. Proceedings of ALIFE's workshop 'Evolution in physical systems' ; 2012 ; 2012. 1.
57. Mulder T, Hochstenbach J, Dijkstra P, et al. Born to adapt, but not in your dreams. Consciousness and Cognition. 2008 ; 17 (4). 1266-71
58. Nakamura Y, Mori T, Sato M, et al. Reinforcement learning for a biped robot based on a CPG-actor-critic method. Neural Networks. 2007 ; 20 (6). 723-735
59. Nelson A, Barlow G, Doitsidis L. Fitness functions in evolutionary robotics: A survey and analysis. Robotics and Autonomous Systems. 2009 ; 57 (4). 345-370
60. Nguyen-Tuong D, Peters J. Model learning for robot control: A survey. Cognitive Processing. 2011 ; 12 (4). 319-340
61. Palmer M, Miller D, Blackwell T. An evolved neural controller for bipedal walking: Transitioning from simulator to hardware. Proceedings of IROS workshop on exploring new horizons in evolutionary design of robots ; 2009 ; 2009..
62. Parker G. Punctuated anytime learning to evolve robot control for area coverage. Design and Control of Intelligent Robotic Systems. 2009 ; 177: 255-277
63. Peters J. Policy gradient methods. Scholarpedia. 2010 ; 5 (10). 3698
64. Peters J, Schaal S. Reinforcement learning of motor skills with policy gradients. Neural Networks. 2008 ; 21 (4). 682-697
65. Prassler E, Kosuge K Handbook of Robotics. Siciliano B Khatib O, ed. New York, NY: Springer ; 2008: 1253-1281.
66. Pretorius C, du Plessis M, Cilliers C. Simulating robots without conventional physics: A neural network approach. Journal of Intelligent & Robotic Systems. 2012 ; 71: 319-348
67. Qu Z, Ihlefeld CM, Jin Y, et al. Robust fault-tolerant self-recovering control of nonlinear uncertain systems. Automatica. 2003 ; 39 (10). 1763-1771
68. Quigley M, Conley K, Gerkey BP, et al. ROS: An open-source robot operating system. Proceedings of ICRA's workshop on open source software ; 2009 ; 2009..
69. Ramachandran V, Hirstein W. The perception of phantom limbs. Brain. 1998 ; 121 (9). 1603-1630
70. Saranli U, Buehler M, Koditschek D. Rhex: A simple and highly mobile hexapod robot. The International Journal of Robotics Research. 2001 ; 20 (7). 616-631
71. Schleyer G, Russell A. Adaptable gait generation for autotomised legged robots. Proceedings of Australasian conference on robotics and automation (ACRA) ; 2010 ; 2010..
72. Schmitz J, Dean J, Kindermann T, et al. A biologically inspired controller for hexapod walking: Simple solutions by exploiting physical properties. The Biological Bulletin. 2001 ; 200 (2). 195-200
73. Smola A, Schölkopf B. A tutorial on support vector regression. Statistics and Computing. 2004 ; 14 (3). 199-222
74. Smola A, Vapnik V. Support vector regression machines. Advances in Neural Information Processing Systems. 1997 ; 9: 155-161
75. Sproewitz A, Moeckel R, Maye J, et al. Learning to move in modular robots using central pattern generators and online optimization. The International Journal of Robotics Research. 2008 ; 27 (3-4). 423-443
76. Steingrube S, Timme M, Wörgötter F, et al. Self-organized adaptation of a simple neural circuit enables complex robot behaviour. Nature Physics. 2010 ; 6 (3). 224-230
77. Sturm J, Plagemann C, Burgard W. Adaptive body scheme models for robust robotic manipulation. Robotics: Science and systems. 2008 ;:
78. Sutton R, McAllester D, Singh S, et al. Policy gradient methods for reinforcement learning with function approximation. Advances In Neural Information Processing Systems. 2000 ; 12 (22). 1057-1063
79. Sutton RS, Barto AG Introduction to Reinforcement Learning. Cambridge, MA: MIT Press ; 1998 :
80. Tedrake R, Zhang T, Seung H. Learning to walk in 20 minutes. Proceedings of Yale workshop on adaptive and learning systems ; 2005 ; 2005..
81. Toffolo A, Benini E. Genetic diversity as an objective in multi-objective evolutionary algorithms. Evolutionary Computation. 2003 ; 11 (2). 151-167
82. Togelius J, Schaul T, Wierstra D, et al. Ontogenetic and phylogenetic reinforcement learning. Kuenstliche Intelligenz. 2009 ; 23 (3). 30-33
83. Turing AM. Computing machinery and intelligence. Mind. 1950 ; 59 (236). 433-460
84. Visinsky M, Cavallaro J, Walker I. Robotic fault detection and fault tolerance: A survey. Reliability Engineering & System Safety. 1994 ; 46 (2). 139-158
85. Vogeley K, Kurthen M, Falkai P, et al. Essential functions of the human self model are implemented in the prefrontal cortex. Consciousness and Cognition. 1999 ; 8 (3). 343-63
86. Weingarten J, Lopes G, Buehler M, et al. Automated gait adaptation for legged robots. Proceedings of the IEEE international conference on robotics and automation (ICRA) ; 2004 ; 2004. 2153.
87. Whiteson S Reinforcement Learning: State of the Art. Wiering M van Otterlo M, ed. Berlin: Springer ; 2012: 326-355.
88. Wilson D. Insect walking. Annual Review of Entomology. 1966 ; 11 (1). 103-122
89. Yosinski J, Clune J, Hidalgo D, et al. Evolving robot gaits in hardware: The HyperNEAT generative encoding vs. parameter optimization. Proceedings of the 20th European conference on artificial life (ECAL) ; 2011 ; 2011. 11.
90. Zagal J, Delpiano J, Ruiz-del Solar J. Self-modeling in humanoid soccer robots. Robotics and Autonomous Systems. 2009 ; 57 (8). 819-827
91. Zagal J, Ruiz-del Solar J, Vallejos P. Back to reality: Crossing the reality gap in evolutionary robotics. Proceedings of IFAC symposium on intelligent autonomous vehicles (IAV) ; 2004 ; 2004..
92. ZykovV (2008) Morphological and behavioral resilience against physical damage for robotic systems. PhD Thesis, Cornell University, NY, USA.
93. Zykov V, Bongard J, Lipson H. Evolving dynamic gaits on a physical robot. Proceedings of genetic and evolutionary computation conference, late breaking paper (GECCO) ; 2004 ; 2004..