Evolutionary robotics: What, why, and where to

Frontiers Robotics AI

Volumn 2, Issue MAR, 2015, Pages

  Doncieux, Stephane ^a   Bredeche, Nicolas ^a   Mouret, Jean Baptiste ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^b Gusz) Eiben, Agoston E   (Netherlands)

Author keywords

Embodied intelligence; Evolutionary algorithms; Evolutionary biology; Evolutionary robotics; Robotics

Indexed keywords

EID: 85026858811     PISSN: None     EISSN: 22969144     Source Type: Journal    
DOI: 10.3389/frobt.2015.00004     Document Type: Review

References (197)

1. Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3, 1178-1183. doi: 10.1038/78829
2. Amato, C., Konidaris, G., Cruz, G., Maynor, C. A., How, J. P., and Kaelbling, L. P. (2015). "Planning for decentralized control of multiple robots under uncertainty," in Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA-15)
3. Auerbach, J., and Bongard, J. (2009). "How robot morphology and training order affect the learning of multiple behaviors," in Proceedings of the IEEE Congress on Evolutionary Computation (CEC'09), (IEEE), 39-46
4. Auerbach, J. E., and Bongard, J. C. (2011). "Evolving complete robots with CPPN-NEAT: the utility of recurrent connections," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'11), (ACM), 1475-1482
5. Auerbach, J. E., and Bongard, J. C. (2012). "On the relationship between environmental and morphological complexity in evolved robots," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'12), (New York, NY: ACM Press), 521-528
6. Auerbach, J. E., and Bongard, J. C. (2014). Environmental influence on the evolution of morphological complexity in machines. PLoS Comput. Biol. 10:e1003399. doi:10.1371/journal.pcbi.1003399
7. Bailey, C. H., Giustetto, M., Huang, Y. Y., Hawkins, R. D., and Kandel, E. R. (2000). Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat. Rev. Neurosci. 1, 11-20. doi:10.1038/35036191
8. Bartz-Beielstein, T. (2006). Experimental Research in Evolutionary Computation. Springer
9. Batut, B., Parsons, D. P., Fischer, S., Beslon, G., and Knibbe, C. (2013). In silico experimental evolution: a tool to test evolutionary scenarios. BMC Bioinformatics 14(Suppl. 15):S11. doi:10.1186/1471-2105-14-S15-S11
10. Bedau, M. A., McCaskill, J. S., Packard, N. H., Rasmussen, S., Adami, C., Green, D. G., et al. (2000). Open problems in artificial life. Artif. Life 6, 363-376. doi:10.1162/106454600300103683
11. Beer, R. D. (1995). On the dynamics of small continuous-time recurrent neural networks. Adapt. Behav. 3, 469-510. doi:10.1177/105971239500300405
12. Bellas, F., and Duro, R. (2010). Multilevel Darwinist brain (MDB): artificial evolution in a cognitive architecture for real robots. IEEE Trans. Auton. Ment. Dev. 2, 340-354. doi:10.1109/TAMD.2010.2086453
13. Bernstein, D. S., Givan, R., Immerman, N., and Zilberstein, S. (2002). The complexity of decentralized control of Markov decision processes. Math. Oper. Res. 27, 819-840. doi:10.1287/moor.27.4.819.297
14. Bianco, R., and Nolfi, S. (2004). Toward open-ended evolutionary robotics: evolving elementary robotic units able to self-assemble and self-reproduce. Conn. Sci. 16, 227-248. doi:10.1080/09540090412331314759
15. Bishop, C. M. (1995). Neural Networks for Pattern Recognition. New York, NY: Oxford University Press Inc
16. Bjorklund, D. F. (1997). The role of immaturity in human development. Psychol. Bull. 122, 153-169. doi:10.1037/0033-2909.122.2.153
17. Bongard, J. (2002). "Evolving modular genetic regulatory networks," in Proceedings of the World on Congress on Computational Intelligence, (IEEE), 1872-1877
18. Bongard, J. (2011). Morphological change in machines accelerates the evolution of robust behavior. Proc. Natl. Acad. Sci. U.S.A. 108, 1234-1239. doi:10.1073/pnas.1015390108
19. Bongard, J., Zykov, V., and Lipson, H. (2006). Resilient machines through continuous self-modeling. Science 314, 1118-1121. doi:10.1126/science.1133687
20. Bongard, J. C. (2010). The utility of evolving simulated robot morphology increases with task complexity for object manipulation. Artif. Life 16, 201-223. doi:10.1162/artl.2010.Bongard.024
21. Bongard, J. C. (2013). Evolutionary robotics. Commun. ACM 56, 74. doi:10.1145/2492007.2493883
22. Braitenberg, V. (1984). Vehicles. Cambridge, MA: MIT Press
23. Bredeche, N. (2014). "Embodied evolutionary robotics with large number of robots," in Artificial Life 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems, Number 1, (MIT Press), 272-273
24. Bredeche, N., Haasdijk, E., and Eiben, A. E. (2010). "On-line, on-board evolution of robot controllers," in Proceedings of the 9th International Conference on Artificial Evolution (Evolution Artificielle-EA'09), Strasbourg: LNCS, Oct 2009 (Springer), 5975, 110-121
25. Bredeche, N., and Montanier, J.-M. (2010). "Environment-driven embodied evolution in a population of autonomous agents," in Parallel Problem Solving from Nature (PPSN), (LNCS, Springer), 290-299
26. Bredeche, N., Montanier, J.-M., Liu, W., and Winfield, A. F. (2012). Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Math. Comput. Model. Dyn. Syst. 18, 101-129. doi:10.1080/13873954.2011.601425
27. Brown, E., Rodenberg, N., Amend, J., Mozeika, A., Steltz, E., Zakin, M. R., et al. (2010). Universal robotic gripper based on the jamming of granular material. Proc. Natl. Acad. Sci. U.S.A. 107, 18809-18814. doi:10.1073/pnas.1003250107
28. Bryson, D. M., and Ofria, C. (2013). Understanding evolutionary potential in virtual CPU instruction set architectures. PLoS ONE 8:e83242. doi:10.1371/journal.pone.0083242
29. Bullmore, E., and Sporns, O. (2009). Complex brain networks: graph theoretical analysis of structural and functional systems. Nat. Rev. Neurosci. 10, 186-198. doi:10.1038/nrn2618
30. Calandra, R., Peters, J., and Deisenroth, M. P. (2014). "An experimental comparison of Bayesian optimization for bipedal locomotion," in Proc. of 2014 IEEE International Conference on Robotics and Automation (ICRA), 1-8
31. Changeux, J. P., Courrège, P., and Danchin, A. (1973). A theory of the epigenesis of neuronal networks by selective stabilization of synapses. Proc. Natl. Acad. Sci. U.S.A. 70, 2974-2978. doi:10.1073/pnas.70.10.2974
32. Cheney, N., Clune, J., and Lipson, H. (2014). "Evolved electrophysiological soft robots," in Proc. of Artificial Life, 222-229
33. Cheney, N., MacCurdy, R., Clune, J., and Lipson, H. (2013). "Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding," in Proc. of the International Conference on Genetic and Evolutionary Computation Conference (GECCO'13), (ACM), 167-174
34. Clune, J., Beckmann, B. E., McKinley, P. K., and Ofria, C. (2010). "Investigating whether hyperneat produces modular neural networks," in Proc. of the International Conference on Genetic and Evolutionary Computation Conference (GECCO'10), (ACM), 635-642
35. Clune, J., Mouret, J.-B., and Lipson, H. (2013). The evolutionary origins of modularity. Proc. Biol. Soc. 280, 20122863. doi:10.1098/rspb.2012.2863
36. Coleman, O., Blair, A., and Clune, J. (2014). "Automated generation of environments to test the general learning capabilities of AI agents," in Proc. of the International Conference on Genetic and Evolutionary Computation Conference (GECCO'14), (ACM), 161-168
37. Collins, S., Ruina, A., Tedrake, R., and Wisse, M. (2005). Efficient bipedal robots based on passive-dynamic walkers. Science 307, 1082-1085. doi:10.1126/science.1107799
38. Cuccu, G., and Gomez, F. (2011). "When novelty is not enough," in Applications of Evolutionary Computation, (Heidelberg: Springer), 234-243
39. Cussat-Blanc, S., and Pollack, J. (2012). "A cell-based developmental model to generate robot morphologies," in Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference (GECCO'12), (ACM), 537-544
40. Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst. 2, 303-314. doi:10.1007/BF02551274
41. Darwin, C. (1859). On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life. London: John Murray
42. de Margerie, E., Mouret, J.-B., Doncieux, S., and Meyer, J.-A. (2007). Artificial evolution of the morphology and kinematics in a flapping-wing mini-UAV. Bioinspir. Biomim. 2, 65-82. doi:10.1088/1748-3182/2/4/002
43. Dennett, D. C. (1995). Darwin's Dangerous Idea: Evolution and the Meanings of Life. Simon & Schuster
44. Dennett, D. C. (2003). "The Baldwin effect: a crane, not a skyhook," in Evolution and Learning: The Baldwin Effect Reconsidered, eds B. H. Weber and D. J. Depew (MIT Press), 60-79
45. Dial, K. P. (2003). Wing-assisted incline running and the evolution of flight. Science 299, 402-404. doi:10.1126/science.1078237
46. Doncieux, S., Bredeche, N., and Mouret, J.-B. (eds) (2011). New Horizons in Evolutionary Robotics, Volume 341 of Studies in Computational Intelligence. Springer
47. Doncieux, S., and Meyer, J.-A. (2004). "Evolving modular neural networks to solve challenging control problems," in Proceedings of the Fourth International ICSC Symposium on Engineering of Intelligent Systems (EIS 2004), 1-7
48. Doncieux, S., and Meyer, J.-A. (2005). Evolving PID-like neurocontrollers for nonlinear control problems. Control Intell. Syst. 33, 55-62. doi:10.2316/Journal.201.2005.1.201-1555
49. Doncieux, S., and Mouret, J.-B. (2014). Beyond black-box optimization: a review of selective pressures for evolutionary robotics. Evol. Intell. 7, 71-93. doi:10.1007/s12065-014-0110-x
50. Edelman, G. (1987). Neural Darwinism: The Theory of Neuronal Group Selection. Oxford Paperbacks
51. Eiben, A. E., Haasdijk, E., and Bredeche, N. (2010). "Embodied, on-line, on-board evolution for autonomous robotics," in Symbiotic Multi-Robot Organisms: Reliability, Adaptability, Evolution, Chap. 5.2, eds P. Levi and S. Kernbach (Springer), 361-382
52. Eiben, A. E., and Smith, J. E. (2003). Introduction to Evolutionary Computing. Springer
53. Espinosa-Soto, C., and Wagner, A. (2010). Specialization can drive the evolution of modularity. PLoS Comput. Biol. 6:e1000719. doi:10.1371/journal.pcbi.1000719
54. Fernando, C., Szathmary, E., and Husbands, P. (2012). Selectionist and evolutionary approaches to brain function: a critical appraisal. Front. Comput. Neurosci. 6:24. doi:10.3389/fncom.2012.00024
55. Ficici, S., Watson, R., and Pollack, J. (1999). "Embodied evolution: a response to challenges in evolutionary robotics," in Proceedings of the Eighth European Workshop on Learning Robots, 14-22
56. Floreano, D., Dürr, P., and Mattiussi, C. (2008). Neuroevolution: from architectures to learning. Evol. Intell. 1, 47-62. doi:10.1007/s12064-008-0029-9
57. Floreano, D., and Keller, L. (2010). Evolution of adaptive behaviour in robots by means of Darwinian selection. PLoS Biol. 8:e1000292. doi:10.1371/journal.pbio.1000292
58. Floreano, D., and Mattiussi, C. (2008). Bio-Inspired Artificial Intelligence: Theories, Methods, and Technologies. MIT Press
59. Floreano, D., Mitri, S., Magnenat, S., and Keller, L. (2007). Evolutionary conditions for the emergence of communication in robots. Curr. Biol. 17, 514-519. doi:10.1016/j.cub.2007.01.058
60. Fogel, D. (1998). Evolutionary Computation: The Fossil Record. IEEE Press
61. Fogel, L. J., Owens, A. J., and Walsh, M. J. (1966). Artificial Intelligence Through Simulated Evolution. New York, NY: John Wiley & Sons Inc
62. Funahashi, K., and Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Netw. 6, 801-806. doi:10.1016/S0893-6080(05)80125-X
63. Geijtenbeek, T., van de Panne, M., and van der Stappen, A. F. (2013). Flexible muscle-based locomotion for bipedal creatures. ACM Trans. Graph. 32, 1-11. doi:10.1145/2508363.2508399
64. Gerhart, J., and Kirschner, M. (2007). The theory of facilitated variation. Proc. Natl. Acad. Sci. U.S.A. 104, 8582-8589. doi:10.1073/pnas.0701035104
65. Glette, K., Klaus, G., Zagal, J., and Torresen, J. (2012). "Evolution of locomotion in a simulated quadruped robot and transferral to reality," in Proceedings of the Seventeenth International Symposium on Artificial Life and Robotics, 1-4
66. Goldman, C. V., and Zilberstein, S. (2004). Decentralized control of cooperative systems: categorization and complexity analysis. J. Artif. Intell. Res. 22, 143-174. doi:10.1613/jair.1427
67. Gomes, J., and Christensen, A. L. (2013). "Generic behaviour similarity measures for evolutionary swarm robotics," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'13), (ACM), 199-206
68. Gomes, J., Urbano, P., and Christensen, A. (2013). Evolution of swarm robotics systems with novelty search. Swarm Intell. 7, 115-144. doi:10.1007/s11721-013-0081-z
69. Gomes, J., Urbano, P., and Christensen, A. L. (2012). "Introducing novelty search in evolutionary swarm robotics," in Proc. of the International Conference on Swarm Intelligence (ANTS'2012), 85-96
70. Gould, S. J., and Vrba, E. S. (1982). Exaptation; a missing term in the science of form. Paleobiology 8, 4-15
71. Gruau, F. (1995). Automatic definition of modular neural networks. Adapt. Behav. 3, 151-183. doi:10.1177/105971239400300202
72. Haasdijk, E., Bredeche, N., and Eiben, A. E. (2014). Combining environment-driven adaptation and task-driven optimisation in evolutionary robotics. PLoS ONE 9:e98466. doi:10.1371/journal.pone.0098466
73. Hamilton, W. D. (1964). The genetical evolution of social behaviour. J. Theor. Biol. 7, 1-16. doi:10.1016/0022-5193(64)90039-6
74. Hansen, N., and Ostermeier, A. (2001). Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9, 159-195. doi:10.1162/106365601750190398
75. Hara, F., and Pfeifer, R. (2003). Morpho-Functional Machines: The New Species: Designing Embodied Intelligence. Springer-Verlag Tokyo Inc
76. Harvey, I., Husbands, P., and Cliff, D. (1994). "Seeing the light: artificial evolution; real vision," in From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (SAB), (MIT Press), 392-401
77. Hauert, S., Zufferey, J.-C., and Floreano, D. (2008). Evolved swarming without positioning information: an application in aerial communication relay. Auton. Robots 26, 21-32. doi:10.1007/s10514-008-9104-9
78. Haykin, S. (1998). Neural Networks: A Comprehensive Foundation, 2nd Edn. Prentice Hall
79. Heidrich-Meisner, V. (2008). "Variable metric reinforcement learning methods applied to the noisy mountain car problem," in Recent Advances in Reinforcement Learning, (Berlin: Springer), 136-150
80. Heidrich-Meisner, V., and Igel, C. (2009). Neuroevolution strategies for episodic reinforcement learning. J. Algorithms 64, 152-168. doi:10.1016/j.jalgor.2009.04.002
81. Hinton, G., and Nowlan, S. (1987). How learning can guide evolution. Complex Syst. 1, 495-502
82. Holland, J. (1962a). Outline for a logical theory of adaptive systems. J. Assoc. Comput. Mach. 3, 297-314. doi:10.1145/321127.321128
83. Holland, J. (1962b). Outline for a logical theory of adaptive systems. J. Assoc. Comput. Mach. 3, 297-314. doi:10.1145/321127.321128
84. Holland, J., and Reitman, J. (1977). Cognitive systems based on adaptive algorithms. ACM SIGART Bull. 63, 1977
85. Hornby, G. S., and Pollack, J. B. (2002). Creating high-level components with a generative representation for body-brain evolution. Artif. Life 8, 223-246. doi:10.1162/106454602320991837
86. Hornby, G. S., Takamura, S., Yamamoto, T., and Fujita, M. (2005). Autonomous evolution of dynamic gaits with two quadruped robots. IEEE Trans. Robot. 21, 402-410. doi:10.1109/TRO.2004.839222
87. Hu, T., and Banzhaf, W. (2010). Evolvability and speed of evolutionary algorithms in light of recent developments in biology. J. Artif. Evol. Appl. 28. doi:10.1155/2010/568375
88. Hughes, R. I. G. (1997). "Models and representation," in Philosophy of science, 64 (Supplement Proceedings of the 1996 Biennial Meetings of the Philosophy of Science Association. Part II: Symposia Papers), (The University of Chicago Press), 325-336
89. Huizinga, J., Mouret, J.-B., and Clune, J. (2014). "Evolving neural networks that are both modular and regular: HyperNeat plus the connection cost technique," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'14), (ACM)
90. Jakobi, N. (1998). "Running across the reality gap: octopod locomotion evolved in a minimal simulation," in Evolutionary Robotics, (Berlin: Springer), 39-58
91. Jakobi, N., Husbands, P., and Harvey, I. (1995). Noise and the reality gap: the use of simulation in evolutionary robotics. Lect. Notes Comput. Sci. 929, 704-720. doi:10.1007/3-540-59496-5_337
92. Jin, Y. (2011). Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol. Comput. 1, 61-70. doi:10.1016/j.swevo.2011.05.001
93. Kashtan, N., and Alon, U. (2005). Spontaneous evolution of modularity and network motifs. Proc. Natl. Acad. Sci. U.S.A. 102, 13773. doi:10.1073/pnas.0503610102
94. Kauffman, S. A. (1993). The Origins of Order: Self-Organization and Selection in Evolution. Oxford University Press
95. Kirschner, M. W., and Gerhart, J. C. (2006). The Plausibility of Life: Resolving Darwin's Dilemma. Yale University Press
96. Kober, J., Bagnell, J. A., and Peters, J. (2013). Reinforcement learning in robotics: a survey. Int. J. Rob. Res. 32, 1238-1274. doi:10.1177/0278364913495721
97. Kodjabachian, J., and Meyer, J.-A. (1997). Evolution and development of neural networks controlling locomotion, gradient-following, and obstacle-avoidance in artificial insects. IEEE Trans. Neural Netw. 9, 796-812. doi:10.1109/72.712153
98. Komosinski, M. (2003). The framsticks system: versatile simulator of 3D agents and their evolution. Kybernetes 32, 156-173. doi:10.1108/03684920310452382
99. Koos, S., Cully, A., and Mouret, J.-B. (2013a). Fast damage recovery in robotics with the T-resilience algorithm. Int. J. Rob. Res. 32, 1700-1723. doi:10.1177/0278364913499192
100. Koos, S., Mouret, J.-B., and Doncieux, S. (2013b). The transferability approach: crossing the reality gap in evolutionary robotics. IEEE Trans. Evol. Comput. 17, 122-145. doi:10.1109/TEVC.2012.2185849
101. Kowaliw, T., Bredeche, N., and Doursat, R. (eds) (2014). Growing Adaptive Machines, Volume 557 of Studies in Computational Intelligence. Berlin: Springer
102. Krcah, P. (2010). "Solving deceptive tasks in robot body-brain co-evolution by searching for behavioral novelty," in Proc. of the International Conference on Intelligent Systems Design and Applications (ISDA'2010), (IEEE), 284-289
103. Lehman, J., Risi, S., Ambrosio, D. D., and Stanley, K. O. (2013a). Encouraging reactivity to create robust machines. Adapt. Behav. 21, 484-500. doi:10.1177/1059712313487390
104. Lehman, J., Stanley, K. O., and Miikkulainen, R. (2013b). "Effective diversity maintenance in deceptive domains," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'13), (New York, NY: ACM Press), 215-222. doi:10.1145/2463372.2463393
105. Lehman, J., and Stanley, K. O. (2008). "Exploiting open-endedness to solve problems through the search for novelty," in Proc. of Artificial Life Conference (ALife XI), (MIT Press), 329-336
106. Lehman, J., and Stanley, K. O. (2011). Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19, 189-223. doi:10.1162/EVCO_a_00025
107. Lehman, J., and Stanley, K. O. (2013). Evolvability is inevitable: increasing evolvability without the pressure to adapt. PLoS ONE 8:e62186. doi:10.1371/journal.pone.0062186
108. Lenski, R. E., Ofria, C., Collier, T. C., and Adami, C. (1999). Genome complexity, robustness and genetic interactions in digital organisms. Nature 400, 661-664. doi:10.1038/23245
109. Liapis, A., Yannakakis, G., and Togelius, J. (2013). "Enhancements to constrained novelty search: two-population novelty search for generating game content," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'13), (ACM), 343-350
110. Lipson, H. (2007). Principles of modularity, regularity, and hierarchy for scalable systems. J. Biol. Phys. Chem. 7, 125-128. doi:10.4024/40701.jbpc.07.04
111. Lipson, H., and Pollack, J. B. (2000). Automatic design and manufacture of robotic lifeforms. Nature 406, 974-978. doi:10.1038/35023115
112. Lizotte, D., Wang, T., Bowling, M., and Schuurmans, D. (2007). "Automatic gait optimization with Gaussian process regression," in Proc. of International Joint Conference on Artificial Intelligence (IJCAI), (Morgan Kaufman Publishers), 944-949
113. Long, J. (2012). Darwin's Devices: What Evolving Robots Can Teach us about the History of Life and the Future of Technology. Basic Books
114. Macalpine, P., Depinet, M., and Stone, P. (2015). "UT Austin Villa 2014: RoboCup 3D simulation league champion via overlapping layered learning," in Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15), (Austin, TX), 1-7
115. Maesani, A., Fernando, P. R., and Floreano, D. (2014). Artificial evolution by viability rather than competition. PLoS ONE 9:e86831. doi:10.1371/journal.pone.0086831
116. Mataric, M., and Michaud, F. (2008). "Behavior-based robotics," in Handbook of Robotics, (Springer), 891-909
117. Maynard-Smith, J. (1992). Byte-sized evolution. Nature 355, 772-773. doi:10.1038/355772a0
118. Mayr, E. (2001). What Evolution Is?. Basic Books
119. McCulloch, W. S., and Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115-133. doi:10.1007/BF02478259
120. Miglino, O., Lund, H. H., and Nolfi, S. (1995). Evolving mobile robots in simulated and real environments. Artif. Life 2, 417-434. doi:10.1162/artl.1995.2.4.417
121. Miller, W. T., Werbos, P. J., and Sutton, R. S. (1995). Neural Networks for Control. MIT Press
122. Mitri, S., Floreano, D., and Keller, L. (2009). The evolution of information suppression in communicating robots with conflicting interests. Proc. Natl. Acad. Sci. U.S.A. 106, 15786-15790. doi:10.1073/pnas.0903152106
123. Mitri, S., Wischmann, S., Floreano, D., and Keller, L. (2013). Using robots to understand social behaviour. Biol. Rev. Camb. Philos. Soc. 88, 31-39. doi:10.1111/j.1469-185X.2012.00236.x
124. Mockus, J., Tiesis, V., and Zilinskas, A. (1978). "The application of Bayesian methods for seeking the extremum," in Towards Global Optimization, Vol. 2, eds L. Dixon and G. Szego (Amsterdam: Elsevier), 117-129
125. Moeckel, R., Perov, Y. N., Nguyen, A. T., Vespignani, M., Bonardi, S., Pouya, S., et al. (2013). "Gait optimization for roombots modular robots. Matching simulation and reality," in Proceedings of IEEE International Conference on Intelligent Robots and Systems (IROS), (IEEE), 3265-3272
126. Montanier, J.-M., and Bredeche, N. (2011). "Surviving the tragedy of commons: emergence of altruism in a population of evolving autonomous agents," in Proc. of the European Conference on Artificial Life (ECAL 2011), (MIT Press), 550-557
127. Montanier, J.-M., and Bredeche, N. (2013). "Evolution of altruism and spatial dispersion: an artificial evolutionary ecology approach," in Proc. of the European Conference on Artificial Life (ECAL 2013), (MIT Press), 260-267
128. Mouret, J.-B. (2011). "Novelty-based multiobjectivization," in New Horizons in Evolutionary Robotics: Extended contributions of the 2009 EvoDeRob Workshop, (Springer), 139-154
129. Mouret, J.-B., and Doncieux, S. (2008). MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evol. Intell. 1, 187-207. doi:10.1007/s12065-008-0015-7
130. Mouret, J.-B., and Doncieux, S. (2009). "Evolving modular neural-networks through exaptation," in Proc. of the IEEE Congress on Evolutionary Computation (CEC'09), (IEEE), 1570-1577
131. Mouret, J.-B., and Doncieux, S. (2010). "Sferes v2: evolvin-in the multicore world," in Proc. of the IEEE World Congress on Computational Intelligence (CEC'10), (IEEE), 4079-4086
132. Mouret, J.-B., and Doncieux, S. (2012). Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol. Comput. 20, 91-133. doi:10.1162/EVCO_a_00048
133. Mouret, J.-B., Koos, S., and Doncieux, S. (2012). "Crossing the reality gap: a short introduction to the transferability approach," in ALIFE workshop "Evolution on Physical Systems, 1-7
134. Mouret, J.-B., and Tonelli, P. (2014). "Artificial evolution of plastic neural networks: a few key concepts," in Growing Adaptive Machines, (Springer), 251-261
135. Müller, G. (2007). Evodevo: extending the evolutionary synthesis. Nat. Rev. Genet. 8, 943-949. doi:10.1038/nrg2219
136. Nolfi, S., and Floreano, D. (2000). Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. MIT Press
137. Oliveira, M. A. C., Doncieux, S., Mouret, J.-B., Peixoto, and dos Santos, C. M. (2013). "Optimization of humanoid walking controller: crossing the reality gap," in Proc. of the IEEE-RAS International Conference on Humanoid Robots (Humanoids'2013), (IEEE), 1-7
138. Olson, R. S., Hintze, A., Dyer, F. C., Knoester, D. B., and Adami, C. (2013). Predator confusion is sufficient to evolve swarming behaviour. J. R. Soc. Interface 10, 20130305. doi:10.1098/rsif.2013.0305
139. Papadimitriou, C. H. (1994). Computational Complexity. Addison-Wesley
140. Pfeifer, R., and Bongard, J. (2007). How the Body Shapes the Way We Think: A New View of Intelligence. MIT Press
141. Pigliucci, M. (2008). Is evolvability evolvable? Nat. Rev. Genet. 9, 75-82. doi:10.1038/nrg2278
142. Radcliffe, N. J. (1993). Genetic set recombination and its application to neural network topology optimisation. Neural Comput. Appl. 1, 67-90. doi:10.1007/BF01411376
143. Ray, T. S. (1991). "An approach to the synthesis of life," in Artificial Life II, 371-406
144. Risi, S., Hughes, C. E., and Stanley, K. O. (2010). Evolving plastic neural networks with novelty search. Adapt. Behav. 18, 470-491. doi:10.1177/1059712310379923
145. Risi, S., Vanderbleek, S. D., Hughes, C. E., and Stanley, K. O. (2009). "How novelty search escapes the deceptive trap of learning to learn," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'09), (ACM), 153-160
146. Rothlauf, F. (2006). Representations for Genetic and Evolutionary Algorithms. Berlin: Springer-Verlag
147. Rubinstein, R. Y., and Kroese, D. P. (2004). The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer
148. Russell, S., and Norvig, P. (2009). Artificial Intelligence: A Modern Approach, 3rd Edn. Prentice Hall Press
149. Saffiotti, A. (1997). The uses of fuzzy logic in autonomous robot navigation. Soft Comput. 1, 180-197. doi:10.1007/s005000050020
150. Sahin, E., and Spears, W. M. (eds) (2005). Swarm Robotics, Volume 3342 of Lecture Notes in Computer Science. Berlin: Springer
151. Saxena, A., Sun, M., and Ng, A. Y. (2009). Make3D: learning 3D scene structure from a single still image. IEEE Trans. Pattern Anal. Mach. Intell. 31, 824-840. doi:10.1109/TPAMI.2008.132
152. Shadow. (2013). Shadow Dexterous Hand Technical Specification. Technical Report. Shadow Robot Company
153. Shubin, N. (2008). Your Inner Fish: A Journey into the 3.5-Billion-Year History of the Human Body. Random House LLC
154. Siciliano, B., and Khatib, O. (2008). Springer Handbook of Robotics. Springer
155. Siegwart, R., Nourbakhsh, I., and Scaramuzza, D. (2011). Introduction to Autonomous Mobile Robots. MIT Press
156. Sims, K. (1994). "Evolving virtual creatures," in Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'94), (ACM), 15-22
157. Skyrms, B. (2003). The Stag Hunt and the Evolution of Social Structure. Cambridge University Press
158. Solomon, M., Soule, T., and Heckendorn, R. B. (2012). "A comparison of a communication strategies in cooperative learning," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'12), (ACM), 153
159. Soltoggio, A., Bullinaria, J. A., Mattiussi, C., Dürr, P., and Floreano, D. (2008). "Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios," in Proc. of Artificial Life Conference (ALife XI), (MIT Press), 11
160. Soltoggio, A., Durr, P., Mattiussi, C., and Floreano, D. (2007). "Evolving neuromodulatory topologies for reinforcement learning-like problems," in Proc. of the IEEE Congress on Evolutionary Computation (CEC'07), (IEEE), 2471-2478
161. Soros, L., and Stanley, K. (2014). "Identifying necessary conditions for open-ended evolution through the artificial life world of chromaria," in Proc. of Artificial Life Conference (ALife 14), (MIT Press), 793-800
162. Stanley, K., and Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies. Evol. Comput. 10, 99-127. doi:10.1162/106365602320169811
163. Stanley, K. O., D'Ambrosio, D. B., and Gauci, J. (2009). A hypercube-based indirect encoding for evolving large-scale neural networks. Artif. Life 15, 185-212. doi:10.1162/artl.2009.15.2.15202
164. Stanley, K. O., and Miikkulainen, R. (2003). A taxonomy for artificial embryogeny. Artif. Life 9, 93-130. doi:10.1162/106454603322221487
165. Stulp, F., and Sigaud, O. (2012). "Path integral policy improvement with covariance matrix adaptation," in Proc. of the 29th International Conference on Machine Learning (ICML-12), 281-288
166. Stulp, F., and Sigaud, O. (2013). Robot skill learning: from reinforcement learning to evolution strategies. Paladyn 4, 49-61. doi:10.2478/pjbr-2013-0003
167. Suh, N. P. (1990). The Principles of Design, Vol. 226. Oxford University Press
168. Sutton, R. S., and Barto, A. G. (1998). Introduction to Reinforcement Learning. MIT Press
169. Tarapore, D., and Mouret, J.-B. (2014a). "Comparing the evolvability of generative encoding schemes," in Proc. of Artificial Life Conference (ALife 14), (MIT Press), 55-62
170. Tarapore, D., and Mouret, J.-B. (2014b). Evolvability signatures of generative encodings: beyond standard performance benchmarks. 1-20. arXiv:1410.4985
171. Taylor, M., Whiteson, S., and Stone, P. (2007). "Temporal difference and policy search methods for reinforcement learning: an empirical comparison," in Proc. of AAAI Conference, (AAAI Press), 1675-1678
172. Taylor, M. E., Whiteson, S., and Stone, P. (2006). "Comparing evolutionary and temporal difference methods in a reinforcement learning domain," in Proc. of the International Conference on Genetic and Evolutionary Computation Conference (GECCO'06), (New York, NY: ACM Press), 1321
173. Thrun, S., Burgard, W., and Fox, D. (2005). Probabilistic Robotics. MIT Press
174. Togelius, J., Schaul, T., Wierstra, D., Igel, C., and Schmidhuber, J. (2009). Ontogenetic and phylogenetic reinforcement learning. Kuenstliche Intelligenz 9, 30-33
175. Tomassini, M. (2005). Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series). New York, NY: Springer-Verlag
176. Tonelli, P., and Mouret, J.-B. (2011). "On the relationships between synaptic plasticity and generative systems," in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, (New York, NY: ACM Press), 1531-1538
177. Tonelli, P., and Mouret, J.-B. (2013). On the relationships between generative encodings, regularity, and learning abilities when evolving plastic artificial neural networks. PLoS ONE 8:e79138. doi:10.1371/journal.pone.0079138
178. Trianni, V. (2014). Evolutionary robotics: model or design? Front. Robot. AI 1:13. doi:10.3389/frobt.2014.00013
179. Trianni, V., Nolfi, S., and Dorigo, M. (2008). "Evolution, self-organisation and swarm robotics," in Swarm Intelligence, (Springer), 163-191
180. Trivedi, D., Rahn, C. D., Kier, W. M., and Walker, I. D. (2008). Soft robotics: biological inspiration, state of the art, and future research. Appl. Bionics Biomech. 5, 99-117. doi:10.1080/11762320802557865
181. Turkewitz, G., and Kenny, P. A. (1985). The role of developmental limitations of sensory input on sensory/perceptual organization. J. Dev. Behav. Pediatr. 6, 302-306. doi:10.1097/00004703-198510000-00014
182. Urzelai, J., and Floreano, D. (2001). Evolution of adaptive synapses: robots with fast adaptive behavior in new environments. Evol. Comput. 9, 495-524. doi:10.1162/10636560152642887
183. Verbancsics, P., and Stanley, K. O. (2011). "Constraining connectivity to encourage modularity in HyperNEAT," in Proc. of the International Conference on Genetic and Evolutionary Computation (GECCO'11), (ACM), 1483-1490
184. Wagner, G., and Altenberg, L. (1996). Complex adaptations and the evolution of evolvability. Evolution 50, 967-976. doi:10.2307/2410639
185. Wagner, G. P. P., Pavlicev, M., and Cheverud, J. M. (2007). The road to modularity. Nat. Rev. Genet. 8, 921-931. doi:10.1038/nrg2267
186. Waibel, M., Floreano, D., and Keller, L. (2011). A quantitative test of Hamilton's rule for the evolution of altruism. PLoS Biol. 9:e1000615. doi:10.1371/journal.pbio.1000615
187. Waibel, M., Keller, L., and Floreano, D. (2009). Genetic team composition and level of selection in the evolution of cooperation. IEEE Trans. Evol. Comput. 13, 648-660. doi:10.1109/TEVC.2008.2011741
188. Watson, R. A., Ficici, S. G., and Pollack, J. B. (2002). Embodied evolution: distributing an evolutionary algorithm in a population of robots. Rob. Auton. Syst. 39, 1-18. doi:10.1016/S0921-8890(02)00170-7
189. Whiteson, S. (2012). "Evolutionary computation for reinforcement learning," in Reinforcement Learning: State of the Art, (Berlin: Springer), 1-30
190. Wischmann, S., Floreano, D., and Keller, L. (2012). Historical contingency affects signaling strategies and competitive abilities in evolving populations of simulated robots. Proc. Natl. Acad. Sci. U.S.A. 109, 864-868. doi:10.1073/pnas.1104267109
191. Wiser, M. J., Ribeck, N., and Lenski, R. E. (2013). Long-term dynamics of adaptation in asexual populations. Science 342, 1364-1367. doi:10.1126/science.1243357
192. Woods, R. J., Barrick, J. E., Cooper, T. F., Shrestha, U., Kauth, M. R., and Lenski, R. E. (2011). Second-order selection for evolvability in a large Escherichia coli population. Science 331, 1433-1436. doi:10.1126/science.1198914
193. Yao, X. (1999). Evolving artificial neural networks. Proc. IEEE 87, 1423-1447. doi:10.1109/5.784219
194. Yosinski, J., Clune, J., Hidalgo, D., Nguyen, S., Zagal, J., and Lipson, H. (2011). "Evolving robot gaits in hardware: the HyperNEAT generative encoding vs. parameter optimization," in Proc. of the European Conference on Artificial Life (ECAL), (MIT Press), 1-8
195. Zagal, J. C., and Ruiz-del Solar, J. (2007). Combining simulation and reality in evolutionary robotics. J. Intell. Rob. Syst. 50, 19-39. doi:10.1007/s10846-007-9149-6
196. Ziemke, T., and Thieme, M. (2002). Neuromodulation of reactive sensorimotor mappings as a short-term memory mechanism in delayed response tasks. Adapt. Behav. 10, 185-199. doi:10.1177/1059712302010003003
197. Zykov, V., Chan, A., and Lipson, H. (2007). "Molecubes: an open-source modular robotics kit," in IROS-2007 Self-Reconfigurable Robotics Workshop, 3-6