A reinforcement learning model of reaching integrating kinematic and dynamic control in a simulated arm robot

2010 IEEE 9th International Conference on Development and Learning, ICDL-2010 - Conference Program

Volumn , Issue , 2010, Pages 211-218

  Caligiore, Daniele ^a,b   Guglielmelli, Eugenio ^b   Borghi, Anna M ^a   Parisi, Domenico ^a   Baldassarre, Gianluca ^a  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

^b UNIVERSITY CAMPUS BIO MEDICO   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ACTOR CRITIC; BIOINSPIRED MODELS; CONTROL MODEL; CONTROLLED VARIABLES; DYNAMIC CONTROLS; EQUILIBRIUM POINT; FINE TUNING; KINEMATIC CONTROL; MOTOR CONTROL; ON DYNAMICS; PHYSICAL SIMULATOR; REACHING TASK; REINFORCEMENT LEARNING MODELS;

KINEMATICS; LEARNING ALGORITHMS;

THREE DIMENSIONAL;

EID: 78149253533     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/DEVLRN.2010.5578840     Document Type: Conference Paper

References (34)

1. R. Shadmehr and S. P. Wise, Eds., The Computational Neurobiology of Reaching and Pointing. Cambridge, MA: The MIT Press, 2005.
2. M. Kawato, "Internal models for motor control and trajectory planning," Curr Opin Neurobiol, vol. 9, pp. 718-727, 1999.
3. N. Hogan, "Mechanical impedance of single- and multi-articular systems," in Multiple muscle systems. Biomechanics and movement organization, W. S.-Y. Winters, J. M., Ed. Springer, New York, 1990, pp. 149-164.
4. F. Nori, G. Sandini, and J. Konczak, "Can imprecise internal motor models explain the ataxic hand trajectories during reaching in young infants?" in Proceedings of the Ninth International Conference on Epigenetic Robotics (EpiRob2009), ser. Lund University Cognitive Studies, L. Canamero, P.-Y. Oudeyer, and C. Balkenius, Eds. Lund: Lund University, 2009, no. 146, pp. 215-216.
5. M. I. Jordan and D. E. Rumelhart, "Forward models: supervised learning with a distal teacher," Cognitive Sci, vol. 16, pp. 307-354, 1992.
6. N. Bhushan and R. Shadmehr, "Computational nature of human adaptive control during learning of reaching movements in force fields," Biol Cybern, vol. 81, pp. 39-60, 1999.
7. A. G. Feldman, "Once more on the equilibrium-point hypothesis (lambda model) for motor control." J Motor Behav, vol. 18, pp. 17-54, 1986.
8. E. Bizzi, N. Hogan, F. A.Mussa-Ivaldi, and S. Giszter, "Does the nervous system use equilibrium-point control to guide single and multiple joint movements," Behav Brain Sci, vol. 15, pp. 603-613, 1992.
9. J. Won and N. Hogan, "Stability properties of human reaching movements," Exp Brain Res, vol. 107, pp. 125-136, 1995.
10. J. F. Pilon, S. J. De Serres, and A. G. Feldman, "Threshold position control of arm movement with anticipatory increase in grip force," Exp Brain Res, vol. 181, pp. 49-67, 2007.
11. R. Balasubramaniam and A. G. Feldman, "Guiding movements without redundancy problems," in Coordination Dynamics, V. K. Jirsa and J. A. S. Kelso, Eds. New York: Springer, 2004.
12. H. Gomi and M. Kawato, "Equilibrium-point control hypothesis examined by measured arm stiffness during multijoint movement," Science, vol. 272, pp. 117-120, 1996. (Pubitemid 26110765)
13. D. Caligiore, T. Ferrauto, D. Parisi, N. Accornero, M. Capozza, and G. Baldassarre, "Using motor babbling and hebb rules for modeling the development of reaching with obstacles and grasping," in Proc. of COGSYS 2008, R. Dillmann, C. Maloney, G. Sandini, T. Asfour, G. Cheng, G. Metta, and A. Ude, Eds. Karlsruhe, Germany: Springer, 2008.
14. J. Peters and S. Schaal, "Reinforcement learning for parameterized motor primitives," in IJCNN. IEEE, 2006, pp. 73-80.
15. N. E. Berthier, M. T. Rosenstein, and A. G. Barto, "Approximate optimal control as a model for motor learning," Psychol Rev, vol. 112, pp. 329- 346, 2005.
16. P. Cesari, T. Shiratori, P. Olivato, and M. Duarte, "Analysis of kinematically redundant reaching movements using the equilibrium-point hypothesis," Biol Cybern, vol. 84, pp. 217-226, 2001.
17. D. Caligiore, D. Parisi, and G. Baldassarre, "Toward an integrated biomimetic model of reaching," in Proc. of ICDL 2007, Y. Demiris, B. Scassellati, and D. Mareschal, Eds. London: Imperial College, 2007, pp. E1-6.
18. T. Aflalo and M. Graziano, "Partial tuning of motor cortex neurons to final posture in a free moving paradigm," PNAS, vol. 103, pp. 2909-2914, 2006.
19. J. Flanagan, D. Ostry, and A. Feldman, "Control of trajectory modifications in target-directed reaching." J Motor Behav, vol. 25, pp. 140-152, 1993.
20. A. Pouget and P. E. Latham, "Population codes," in The Handbook of Brain Theory and Neural Networks, 2nd ed., M. A. Arbib, Ed. Cambridge, MA, USA: The MIT Press, 2003.
21. A. G. Barto, "Adaptive critics and the basal ganglia," in Models of Information Processing in the Basal Ganglia, J. Houk, J. Davis, and D. Beiser, Eds. Cambridge MA, USA: The MIT Press, 1995, pp. 215- 232.
22. M. Butz, O. Sigaud, G. Pezzulo, and G. Baldassarre, Eds., Anticipatory behavior in adaptive learning systems: from brains to individual and social behavior, ser. LNAI4520. Berlin: Springer-Verlag, 2007.
23. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge MA, USA: The MIT Press, 1998.
24. L. Kaelbling, M. Littman, and A. Cassandra, "Planning and acting in partially observable stochastic domains," Journal of Artificial Intelligence, vol. 101, pp. 99-134, 1998.
25. N. E. Berthier, R. K. Clifton, D. D. McCall, and D. J. Robin, "Proximodistal structure of early reaching in human infants." Exp Brain Res, vol. 127, pp. 259-269, 1999.
26. D. Campolo, D. Accoto, F. Taffoni, and E. Guglielmelli, "On the kinematics of human wrist during pointing tasks with application to motor rehabilitation," in IEEE International Conference on Robotics and Automation (ICRA 2008). IEEE, 2008, pp. 1318-1323, pasadena, California, May 19-23.
27. D. Bullock and S. Grossberg, "Vite and flete: neural modules for trajectory formation and postural control." in Volitional Action, W. Hershberger, Ed. Amsterdam: Elsevier., 1989, pp. 253-298.
28. G. Bosco, R. E. Poppele, and J. Eian, "Reference frames for spinal proprioception limb endpoint based or joint-level based," J Neurophysiol, vol. 83, pp. 2931-2945, 2000. (Pubitemid 30321482)
29. K. Doya, "Reinforcement learning in continuous time and space." Neural Comput, vol. 12, no. 1, pp. 219-245, Jan 2000.
30. J. Peters, S. Vijayakumar, and S. Schaal, "Reinforcement learning for humanoid robotics," in Third IEEE-RAS International Conference on Humanoid Robots (Humanoids2003), 2003, karlsruhe, Germany, Sept.29- 30.
31. J. Konczak and J. Dichgans, "The development toward stereotypic arm kinematics during reaching in the first 3 years of life," Experimental Brain Research, vol. 117, no. 2, pp. 346-354, 1997.
32. J. Kuhtz-Buschbeck, H. Stolze, K. J"ohnk, A. Boczek-Funcke, and M. Illert, "Development of prehension movements in children: a kinematic study," Experimental Brain Research, vol. 122, no. 4, pp. 424-432, 1998. (Pubitemid 28457139)
33. E. Burdet, R. Osu, D. W. Franklin, T. E. Milner, and M. Kawato, "The central nervous system stabilizes unstable dynamics by learning optimal impedance." Nature, vol. 414, pp. 446-449, 2001.
34. R. Osu, D. W. Franklin, H. Kato, H. Gomi, K. Domen, T. Yoshioka, and M. Kawato, "Short- and long-term changes in joint co-contraction associated with motor learning as revealed from surface emg." J Neurophysiol, vol. 88, no. 2, pp. 991-1004, Aug 2002.