Learning to move in modular robots using central pattern generators and online optimization

International Journal of Robotics Research

Volumn 27, Issue 3-4, 2008, Pages 423-443

  Sproewitz, Alexander ^a   Moeckel, Rico ^a   Maye, Jérôme ^a   Ijspeert, Auke Jan ^a  

^a EPFL   (Switzerland)

Author keywords

Bluetooth; Central pattern generators; Locomotion; Modular robotics; Online learning; Powell's method

Indexed keywords

ALGORITHMS; BLUETOOTH; DISTRIBUTED COMPUTER SYSTEMS; LEARNING SYSTEMS; OPTIMIZATION;

CENTRAL PATTERN GENERATORS; ONLINE LEARNING; POWELL'S METHOD;

MODULAR ROBOTS;

EID: 40049094998     PISSN: 02783649     EISSN: 17413176     Source Type: Journal    
DOI: 10.1177/0278364907088401     Document Type: Article

References (55)

1. Aoi, S. and Tsuchiya, K. (2006). Stability analysis of a simple walking model driven by an oscillator with a phase reset using sensory feedback. IEEE Transactions on Robotics, 22 (2). 391 - 397.
2. Arena, P., Fortuna, L., Frasca, M., and Sicurella, G. (2004). An adaptive, self-organizing dynamical system for hierarchical control of bio-inspired locomotion. IEEE Transactions on Systems, Man and Cybernetics, Part B, 34 (4). 1823 - 1837.
3. Beer, R.D. and Gallagher, J.C. (1992). Evolving dynamical neural networks for adaptive behavior. Adaptive Behavior, 1 (1). 91 - 122.
4. Bongard, J., Zykov, V., and Lipson, H. (2006). Resilient machines through continuous self-modeling. Science, 314 (5802). 1118 - 1121.
5. Brent, R. (1973). Algorithms for Minimization without Derivatives. NJ Prentice-Hall.
6. Buchli, J., Iida, F., and Ijspeert, A. (2006 a). Finding resonance: Adaptive frequency oscillators for dynamic legged locomotion. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2006), pp. 3903-3909. IEEE.
7. Buchli, J., Righetti, L., and Ijspeert, A. (2006 b). Engineering entrainment and adaptation in limit cycle systems - from biological inspiration to applications in robotics. Biological Cybernetics, 95 (6). 645 - 664.
8. Cohen, A.H., Holmes, P.J., and Rand, R. (1982). The nature of coupling between segmented oscillations and the lamprey spinal generator for locomotion: a mathematical model. Journal of Mathematical Biology, 13: 345 - 369.
9. Collins, S., Ruina, A., Tedrake, R., and Wisse, M. (2005). Efficient Bipedal Robots Based on Passive-Dynamic Walkers. Science, 307 (5712). 1082 - 1085.
10. Conradt, J. and Varshavskaya, P. (2003). Distributed central pattern generator control for a serpentine robot. In International Conference on Artificial Neural Networks (ICANN 2003).
11. Delcomyn, F. (1980). Neural basis for rhythmic behaviour in animals. Science, 210: 492 - 498.
12. Duff, D., Yim, M., and Roufas, K. (2001). Evolution of polybot: A modular reconfigurable robot. In Proceedings of the Harmonic Drive International Symposium, Nagano, Japan, and Proceedings of COE/Super-Mechano-Systems Workshop, Tokyo, Japan, November 2001.
13. Endo, G., Nakanishi, J., Morimoto, J., and Cheng, G. (2005). Experimental studies of a neural oscillator for biped locomotion with Q RIO. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA2005), Barcelona, Spain, pp. 598-604.
14. Geng, T., Porr, B., and Wrgtter, F. (2006). Fast biped walking with a reflexive neuronal controller and real-time online learning. International Journal of Robotics Research, 3: 243 - 261.
15. Grillner, S. (1985). Neural control of vertebrate locomotion - central mechanisms and reflex interaction with special reference to the cat. In W. J. P. Barnes and M. H. Gladden (Eds.), Feedback and motor control in invertebrates and vertebrates, pp. 35 - 56. Croom Helm.
16. Gruau, F. and Quatramaran, K. (1997). Cellular encoding for interactive evolutionary robotics. In P. Husbands and I. Harvey (Eds.), Proceedings of the Fourth European Conference on Artificial Life, ECAL97, pp. 368-377. MIT Press.
17. Ijspeert, A. (2001). A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84 (5). 331 - 348.
18. Ijspeert, A. (2008). Central pattern generators for locomotion control in animals and robots: a review Neural Networks, (In press).
19. Ijspeert, A. and Kodjabachian, J. (1999). Evolution and development of a central pattern generator for the swimming of a lamprey. Artificial Life, 5 (3). 247 - 269.
20. Ijspeert, A.J. and Crespi, A. (2007). Online trajectory generation in an amphibious snake robot using a lamprey-like central pattern generator model. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2007).
21. Ijspeert, A.J., Crespi, A., Ryczko, D., and Cabelguen, J.M. (2007). From swimming to walking with a salamander robot driven by a spinal cord model. Science, 315 (5817). 1416 - 1420.
22. Kamimura, A., Kurokawa, H., Toshida, E., Tomita, K., Murata, S., and Kokaji, S. (2003). Automatic locomotion pattern generation for modular robots. In IEEE International Conference on Robotics and Automation (ICRA2003).
23. Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., and Kokaji, S. (2004). Distributed adaptive locomotion by a modular robotic system, M-TRAN II. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2004), pp. 2370-2377.
24. Kimura, H., Akiyama, S., and Sakurama, K. (1999). Realization of dynamic walking and running of the quadruped using neural oscillators. Autonomous Robots, 7 (3). 247 - 258.
25. Kohl, N. and Stone, P. (2004). Policy gradient reinforcement learning for fast quadrupedal locomotion. In Proceedings of the 2004 IEEE International Conference on Robotics & Automation, pp. 2619-2624.
26. Kopell, N., Ermentrout, G.B., and Williams, T.L. (1991). On chains of oscillators forced at one end. SIAM, Journal of Applied Mathematics, 51 (5). 1397 - 1417.
27. Lachat, D., Crespi, A., and Ijspeert, A.J. (2006). Boxybot: a swimming and crawling fish robot controlled by a central pattern generator. In Proceedings of the first IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2006).
28. Lewis, M.A., Fagg, A.H., and Bekey, G.A. (1993). Genetic algorithms for gait synthesis in a hexapod robot. In Y. F. Zheng (Ed.), Recent trends in mobile robots. World Scientific.
29. Marbach, D. and Ijspeert, A.J. (2005). Online optimization of modular robot locomotion. In Proceedings of the IEEE International Conference on Mechatronics and Automation (ICMA 2005), pp. 248-253.
30. Matsubara, T., Morimoto, J., Nakanishi, J., Sato, M., and Doya, K. (2006). Learning CPG-based biped locomotion with a policy gradient method. Robotics and Autonomous Systems, 54: 911 - 920.
31. Moeckel, R., Jaquier, C., Drapel, K., Dittrich, E., Upegui, A., and Ijspeert, A.J. (2006). Exploring adaptive locomotion with YaMoR, a novel autonomous modular robot with Bluetooth interface. Industrial Robot, 33 (4). 285 - 290.
32. Moeckel, R., Sproewitz, A., Maye, J., and Ijspeert, A.J. (2007). An easy to use bluetooth scatternet protocol for fast data exchange in wireless sensor networks and autonomous robots. In Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2801- 2806.
33. Murata, S., Yoshida, E., Kamimura, A., Kurokawa, H., Tomita, K., and Kokaji, S. (2002). M-tran: Selfreconfigurable modular robotic system. IEEE/ASME Transactions on Mechatronics, 7 (4). 431 - 441.
34. Nakamura, Y., Mori, T., Sato, M., and Ishii, S. (2007). Reinforcement learning for a biped robot based on a cpg-actor-critic method. Neural Networks, 20 (6). 723 - 735.
35. Nishii, J., Uno, Y., and Suzuki, R. (1994). Mathematical models for the swimming pattern of a lamprey, i. analysis of collective oscillators with time-delayed interaction and multiple coupling. Biological Cybernetics, 72: 1 - 9.
36. Ostergaard, E.H. and Lund H.H. (2003). Evolving control for modular robotic unit. In Proceedings of CIRA'03, IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 886-892.
37. Paul, C. and Bongard, J.C. (2001). The road less travelled: Morphology in the optimization of biped robot locomotion. In Proceedings of The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2001).
38. Press, W., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. (1994). Numerical recipes in C: the art of scientific computing, 2 nd edition. Cambridge University Press.
39. Righetti, L. and Ijspeert, A. (2006). Programmable central pattern generators: an application to biped locomotion control. In Proceedings of the 2006 IEEE International Conference on Robotics and Automation (ICRA2006), pp. 1585-1590.
40. Righetti, L. and Ijspeert, A. (2008). Pattern generators with sensory feedback for the control of quadruped locomotion. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation. Accepted for publication.
41. Salemi, B. and Shen, W. (2004). Distributed behavior collaboration for self-reconfigurable robots. In Proceedings of the IEEE Conference on Robotics and Automation (ICRA 2004).
42. Shen, W., Salemi, B., and Will, P. (2002). Hormone-inspired adaptive communication and distributed control for selfreconfigurable robots. IEEE Transactions on Robotics and Automation, 18 (5). 1 - 12.
43. Shen, W., Will, P., Galstyan, A., and Chuong, C. (2004). Hormone-inspired self-organization and distributed control of robotic swarms. Autonomous Robots, 17 (4). 93 - 105.
44. Sigvardt, K.A. and Williams, T.L. (1996). Effects of local oscillator frequency on intersegmental coordination in the lamprey locomotor CPG: theory and experiment. Journal of Neurophysiology, 76 (6). 4094 - 4103.
45. Sims, K. (1994). Evolving 3d morphology and behavior by competition. In Proceedings, Artificial Life IV, pp. 28-39. MIT Press.
46. Stoy, K., Shen, W., and Will, P. (2003). Implementing configuration dependent gaits in a self-reconfigurable robot. In Proceedings of the IEEE International conference on Robotics and Automation (ICRA2003).
47. Sutton, R. and Barto, A.G. (1998). Reinforcement learning: an introduction. MIT Press.
48. Upegui, A., Moeckel, R., Dittrich, E., Ijspeert, A.J., and Sanchez, E. (2005). An fpga dynamically reconfigurable framework for modular robotics. In U. Brinkschulte (Ed.), Workshop Procedings of the 18th International Conference on Architecture of Computing Systems 2005 (ARCS'05). VDE Verlag, Berlin.
49. Vassilvitskii, S., Kubica, J., Rieffel, E., Suh, J., and Yim.M. (2002). On the general reconfiguration problem for expanding cube style modular robots. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA2002).
50. Vona, M. and Rus, D. (2000). A physical implementation of the self-reconfigurable crystalline robot. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA2000), pp. 1726-1733.
51. Weingarten, J.D., Lopes, G., Buehler, M., Groff, R.E., and Koditschek, D.E. (2004, April). Automated gait adaptation for legged robots. In Proceedings of the 2004 IEEE International Conference on Robotics & Automation, pp. 2153- 2158.
52. Williams, T.L., Sigvardt, K.A., Kopell, N., Ermentrout, G.B., and Rempler, M.P. (1990). Forcing of coupled nonlinear oscillators: studies of intersegmental coordination in the lamprey locomotor central pattern generator. Journal of Neurophysiology, 64: 862 - 871.
53. Yerly, M. (2007). Yamor lifelong learning. Master's thesis, EPFL.
54. Yim, M. (1994). Locomotion with a Unit Modular Reconfigurable Robot. Ph. D. thesis, Stanford University Mechanical Engineering Dept.
55. Zhang, Y., Fromherz, M., Crawford, L., and Shang, Y. (2002). A general constraint-based control framework with examples in modular self-reconfigurable robots. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2002).