The transferability approach: Crossing the reality gap in evolutionary robotics

IEEE Transactions on Evolutionary Computation

Volumn 17, Issue 1, 2013, Pages 122-145

  Koos, Sylvain ^a   Mouret, Jean Baptiste ^a   Doncieux, Stéphane ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Evolutionary robotics; reality gap; transfer ability approach

Indexed keywords

CRITICAL ISSUES; EVOLUTIONARY ROBOTICS; FITNESS VALUES; LOCAL SEARCH; MULTI OBJECTIVE EVOLUTIONARY ALGORITHMS; MULTI-OBJECTIVE FORMULATION; NAVIGATION TASKS; OPTIMIZATION METHOD; PHYSICAL ROBOTS; QUADRUPEDAL ROBOT; REALITY GAP; ROBOTIC APPLICATIONS; SIMULATION METHODOLOGY; SURROGATE MODEL; TRANSFER-ABILITY APPROACH; WALKING TASKS;

OPTIMIZATION;

ROBOTS;

EID: 84873297136     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2012.2185849     Document Type: Article

References (59)

1. P. Abbeel, A. Coates, T. Hunter, and A. Ng, "Autonomous autorotation of an RC helicopter," in Experimental Robotics. Berlin, Germany: Springer, 2009, pp. 385-394.
2. P. Abbeel, A. Coates, M. Quigley, and A. Ng, "An application of reinforcement learning to aerobatic helicopter flight," in Proc. NIPS, 2007, p. 1.
3. P. Adigbli, C. Grand, J.-B. Mouret, and S. Doncieux, "Nonlinear attitude and position control of a micro quadrotor using sliding mode and backstepping techniques," in Proc. 7th Eur. MAV Conf., 2007.
4. B. Barshan and H. Durrant-Whyte, "Inertial navigation systems for mobile robots," IEEE Trans. Robot. Autom., vol. 11, no. 3, pp. 328-342, Jun. 1995.
5. A. Boeing and T. Br̈aunl, "Evaluation of real-time physics simulation systems," in Proc. 5th Int. Conf. Comput. Graph. Interactive Tech. Australia Southeast Asia, 2007, pp. 281-288.
6. J. Bongard and H. Lipson, "Once more unto the breach: Co-evolving a robot and its simulator," in Proc. 9th Artif. Life, 2004, p. 57.
7. J. Bongard and H. Lipson, "Nonlinear system identification using coevolution of models and tests," IEEE Trans. Evol. Comput., vol. 9, no. 4, pp. 361-384, Aug. 2005. (Pubitemid 41226911)
8. J. Bongard, V. Zykov, and H. Lipson, "Resilient machines through continuous self-modeling," Science, vol. 314, no. 5802, p. 1118, 2006.
9. J. Borenstein, H. Everett, L. Feng, and D. Wehe, "Mobile robot positioning: Sensors and techniques," J. Robot. Syst., vol. 14, no. 4, pp. 231-249, 1997. (Pubitemid 127709434)
10. J. Campbell, R. Sukthankar, I. Nourbakhsh, and A. Pahwa, "A robust visual odometry and precipice detection system using consumer-grade monocular vision," in Proc. ICRA, 2005, pp. 3421-3427. (Pubitemid 46084314)
11. D. Cliff, P. Husbands, and I. Harvey, "Evolving visually guided robots," in Proc. Conf. From Animals to Animats, vol. 2. 1993, pp. 374-383.
12. C. Coello, "Constraint-handling using an evolutionary multiobjective optimization technique," Civil Eng. Environmental Syst., vol. 17, no. 4, pp. 319-346, 2000.
13. K. Deb, Multi-Objective Optimization Using Evolutionary Algorithms. New York: Wiley, 2001.
14. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182-197, Apr. 2002. (Pubitemid 34555372)
15. S. Doncieux and J.-B. Mouret, "Behavioral diversity measures for evolutionary robotics," in Proc. IEEE CEC, Jul. 2010, pp. 1303-1310.
16. D. Floreano and F. Mondada, "Evolutionary neurocontrollers for autonomous mobile robots," Neural Netw., vol. 11, nos. 7-8, pp. 1461-1478, 1998. (Pubitemid 28527138)
17. D. Floreano and J. Urzelai, "Evolution of plastic control networks," Autonomous Robots, vol. 11, no. 3, pp. 311-317, 2001. (Pubitemid 33109980)
18. A. Gomes, I. Voiculescu, J. Jorge, B. Wyvill, and C. Galbraith, Implicit Curves and Surfaces: Mathematics, Data Structures and Algorithms. Berlin, Germany: Springer-Verlag, 2009.
19. M. Gongora, B. Passow, and A. Hopgood, "Robustness analysis of evolutionary controller tuning using real systems," in Proc. IEEE CEC, May 2009, pp. 606-613.
20. C. Grand, F. BenAmar, F. Plumet, and P. Bidaud, "Decoupled control of posture and trajectory of the hybrid wheel-legged robot Hylos," in Proc. IEEE ICRA, vol. 5. Apr.-May 2004, pp. 5111-5116.
21. C. Hartland and N. Bred̀eche, "Evolutionary robotics, anticipation and the reality gap," in Proc. ROBIO, 2006, pp. 1640-1645.
22. I. Harvey, P. Husbands, and D. Cliff, "Issues in evolutionary robotics," in Proc. SAB, 1992, pp. 364-373.
23. T. Hemker, H. Sakamoto, M. Stelzer, and O. von Stryk, "Hardware-inthe-loop optimization of the walking speed of a humanoid robot," in Proc. CLAWAR, 2006, pp. 614-623.
24. G. Hornby, S. Takamura, J. Yokono, O. Hanagata, T. Yamamoto, and M. Fujita, "Evolving robust gaits with AIBO," in Proc. IEEE ICRA, vol. 3. Apr. 2000, pp. 3040-3045.
25. M. Ḧusken, Y. Jin, and B. Sendhoff, "Structure optimization of neural networks for evolutionary design optimization," Soft Comput. A Fusion Found. Methodol. Applicat., vol. 9, no. 1, pp. 21-28, 2005. (Pubitemid 40097882)
26. N. Jakobi, "Evolutionary robotics and the radical envelope-of-noise hypothesis," Adaptive Behav., vol. 6, no. 2, p. 325, 1997.
27. N. Jakobi, "Minimal simulations for evolutionary robotics," Ph.D. dissertation, School Cogn. Computing Sci., Univ. Sussex, Brighton, U.K., 1998.
28. N. Jakobi, "Running across the reality gap: Octopod locomotion evolved in a minimal simulation," in Lecture Notes in Computer Science, vol. 1468. Berlin, Germany: Springer, 1998, pp. 39-58. (Pubitemid 128122274)
29. N. Jakobi, P. Husbands, and I. Harvey, "Noise and the reality gap: The use of simulation in evolutionary robotics," in Proc. ECAL, 1995, pp. 704-720.
30. S. Jakobsson, B. Andersson, and F. Edelvik, "Rational radial basis function interpolation with applications to antenna design," J. Comput. Appl. Math., vol. 233, no. 4, pp. 889-904, 2009.
31. Y. Jin, "A comprehensive survey of fitness approximation in evolutionary computation," Soft Comput. A Fusion Found. Methodol. Applicat., vol. 9, no. 1, pp. 3-12, 2005. (Pubitemid 40097880)
32. D. Jones, M. Schonlau, and W. Welch, "Efficient global optimization of expensive black-box functions," J. Global Optimization, vol. 13, no. 4, pp. 455-492, 1998. (Pubitemid 128507405)
33. N. Kohl and P. Stone, "Policy gradient reinforcement learning for fast quadrupedal locomotion," in Proc. ICRA, vol. 3. 2004, pp. 2619-2624.
34. S. Koos, J.-B. Mouret, and S. Doncieux, "Automatic system identification based on coevolution of models and tests," in Proc. IEEE CEC, Jul. 2009, pp. 119-126.
35. S. Koos, J.-B. Mouret, and S. Doncieux, "Crossing the reality gap in evolutionary robotics by promoting transferable controllers," in Proc. GECCO, 2010, pp. 560-567.
36. S. Lophaven, H. Nielsen, and J. Sondergaard, "DACE: A MATLAB Kriging Toolbox," IMM, Technical University of Denmark, Tech. Rep. IMM-TR-2002-12, 2002.
37. T. Malvic and M. Durekovic, "Application of methods: Inverse distance weighting, ordinary Kriging and collocated cokriging in porosity evaluation, and comparison of results on the Benicanci and Stari Gradac fields in Croatia," Nafta, vol. 54, no. 9, pp. 331-340, 2003.
38. E. Margerie, J.-B. Mouret, S. Doncieux, and J.-A. Meyer, "Artificial evolution of the morphology and kinematics in a flapping-wing mini-UAV," Bioinsp. Biomimetics, vol. 2, no. 4, p. 65, 2007.
39. J.-A. Meyer, P. Husbands, and I. Harvey, "Evolutionary robotics: A survey of applications and problems," in Lecture Notes in Computer Science, vol. 1468. Berlin, Germany: Springer, 1998, pp. 1-21. (Pubitemid 128122272)
40. O. Miglino, H. Lund, and S. Nolfi, "Evolving mobile robots in simulated and real environments," Artif. Life, vol. 2, no. 4, pp. 417-434, 1995.
41. F. Mondada, M. Bonani, X. Raemy, J. Pugh, C. Cianci, A. Klaptocz, S. Magnenat, J. Zufferey, D. Floreano, and A. Martinoli, "The e-puck, a robot designed for education in engineering," in Proc. 9th Conf. Autonomous Robot Syst. Competitions, vol. 1. 2009, pp. 59-65.
42. J.-B. Mouret and S. Doncieux, "Using behavioral exploration objectives to solve deceptive problems in neuro-evolution," in Proc. GECCO, 2009, pp. 627-634.
43. J.-B. Mouret and S. Doncieux, "Sferes v2: Evolvin' in the multicore world," in Proc. IEEE CEC, Jul. 2010, pp. 1-8.
44. T. Mueller, N. Pusuluri, K. Mathias, P. Cornelius, R. Barnhisel, and S. Shearer, "Map quality for ordinary kriging and inverse distance weighted interpolation," Soil Sci. Soc. Amer. J., vol. 68, no. 6, p. 2042, 2004.
45. D. Nist́er, O. Naroditsky, and J. Bergen, "Visual odometry for ground vehicle applications," J. Field Robot., vol. 23, no. 1, pp. 3-20, 2006.
46. S. Nolfi and D. Floreano, Evolutionary Robotics. Cambridge, MA: MIT Press, 2000.
47. S. Nolfi, D. Floreano, O. Miglino, and F. Mondada, "How to evolve autonomous robots: Different approaches in evolutionary robotics," in Proc. 4th Artif. Life, 1994, pp. 190-197.
48. M. Palmer, D. Miller, and T. Blackwell, "An evolved neural controller for bipedal walking: Transitioning from simulator to hardware," in Proc. IROS Workshop Exploring New Horizons Evol. Des. Robots, 2009 pp. 51-58.
49. T. Pinville, S. Koos, J.-B. Mouret, and S. Doncieux, "How to promote generalisation in evolutionary robotics: The ProGAb approach," in Proc. GECCO, 2011, pp. 259-266.
50. J. Pollack, H. Lipson, S. Ficici, P. Funes, G. Hornby, and R. Watson, "Evolutionary techniques in physical robotics," in Proc. ICES, 2000, pp. 175-186.
51. A. Prieto, F. Bellas, J. Becerra, B. Priego, and R. Duro, "Self-organizing robot teams using asynchronous situated co-evolution," in Proc. SAB, LNCS 6226. 2010, pp. 565-574.
52. W. Regan, F. van Breugel, and H. Lipson, "Toward evolvable hovering flight on a physical ornithopter," in Proc. 10th Artif. Life, vol. 100. 2006, p. 241.
53. F. Saunders, E. Golden, R. White, and J. Rife, "Experimental verification of soft-robot gaits evolved using a lumped dynamic model," Robotica, vol. 29, no. 6, pp. 823-830, 2011.
54. D. Shepard, "A two-dimensional interpolation function for irregularlyspaced data," in Proc. 23rd ACM Natl. Conf., 1968, pp. 517-524.
55. A. Terekhov, J.-B. Mouret, and C. Grand, "Stochastic optimization of a neural network-based controller for aggressive maneuvers on loose surfaces," in Proc. IEEE IROS, Oct. 2010, pp. 4782-4787.
56. A. Thompson, P. Layzell, and R. Zebulum, "Explorations in design space: Unconventional electronics design through artificial evolution," IEEE Trans. Evol. Comput., vol. 3, no. 3, pp. 167-196, Sep. 1999.
57. I. Voutchkov and A. Keane, Multiobjective Optimization Using Surrogates. Berlin, Germany: Springer, 2006.
58. J. Zagal and J. Ruiz-del Solar, "Combining simulation and reality in evolutionary robotics," J. Intell. Robot. Syst., vol. 50, no. 1, pp. 19-39, 2007. (Pubitemid 47228785)
59. V. Zykov, J. Bongard, and H. Lipson, "Evolving dynamic gaits on a physical robot," in Proc. GECCO, vol. 4. 2004.