Fast reinforcement learning for three-dimensional kinetic human-robot cooperation with an EMG-to-activation model

Advanced Robotics

Volumn 25, Issue 5, 2011, Pages 563-580

  Tamei, Tomoya ^a   Shibata, Tomohiro ^a  

^a NARA INSTITUTE OF SCIENCE AND TECHNOLOGY   (Japan)

Author keywords

electromyogram; Human machine interaction; motion capturing; reinforcement learning

Indexed keywords

ACTIVATION MODELS; CLOSED-LOOP FEEDBACK SYSTEM; COOPERATIVE TASKS; ELECTROMYOGRAM; ELECTROMYOGRAPHIC SIGNAL; EMG SIGNAL; FAST LEARNING; FORCE SENSOR; GOAL-ORIENTED; HUMAN MACHINE INTERACTION; HUMAN-MACHINE COOPERATION; HUMAN-ROBOT COOPERATION; KINETIC INTERACTIONS; MOTION CAPTURING; MUSCLE COORDINATION; NATURAL INTERACTIONS; POLICY GRADIENT; POWER ASSIST; RESEARCH FIELDS; TIME VARYING;

MAN MACHINE SYSTEMS; REINFORCEMENT LEARNING; THREE DIMENSIONAL; TIME VARYING SYSTEMS;

HUMAN ROBOT INTERACTION;

EID: 79953091002     PISSN: 01691864     EISSN: 15685535     Source Type: Journal    
DOI: 10.1163/016918611X558252     Document Type: Article

References (27)

1. H. Kawamoto and Y. Sankai, Power assist system HAL-3 for gait disorder person, Lecture Notes Comput. Sci. 2398, 196-203 (2002).
2. J. E. Pratt, B. T. Krupp, C. J. Morse and S. H. Collins, The RoboKnee: An exoskeleton for enhancing strength and endurance during walking, in: Proc. IEEE Int. Conf. on Robotics and Automation, New Orleans, LA, vol. 3, pp. 2430-2435 (2004).
3. W. S. Harwin, T. Rahman and R. A. Foulds, A review of design issues in rehabilitation robotics with reference to North American research, IEEE Trans. Rehabil. Eng. 3, 3-13 (1995).
4. R. Riener, Model-based development of neuroprosthesis for paraplegic patients, Philos. Trans. Roy. Soc. B 354, 877 (1999).
5. T. Miyashita, T. Tajika, H. Ishiguro, K. Kogure and N. Hagita, Haptic communication between humans and robots, in: Proc. 12th Int. Symp. on Robotics Research, San Francisco, CA, CD-ROM (2005).
6. T. Hoshi and H. Shinoda, A tactile sensing element for a whole body robot skin, in: Proc. 12th Int. Symp. on Robotics Research, San Francisco, CA, CD-ROM (2005).
7. E. E. Cavallaro, J. Rosen, J. C. Perry and S. Burns, Real-time myoprocessors for a neural controlled powered exoskeleton arm, IEEE Trans. Biomed. Eng. 53, 2387-2396 (2006). (Pubitemid 44629878)
8. H. Kawamoto and Y. Sankai, Quantitative motion control analysis method for power assist system based on human motion property, Trans. JSME C 70, 1115-1123 (2004).
9. O. Fukuda, T. Tsuji, M. Kaneko and A. Otsuka, A human-assisting manipulator teleoperated by EMG signals and arm motions, IEEE Trans. Robotics Automat. 19, 210-222 (2003).
10. H. Ogino, J. Arita and T. Tsuji, A wearable pointing device using EMG signals, J. RoboticsMechatron. 17, 173-180 (2005). (Pubitemid 40706375)
11. T. Iberall, G. Sukhatme, D. Beattie and G. Bekey, On the development of EMG control for a prosthesis using a robotic hand, in: Proc. IEEE Int. Conf. on Robotics and Automation, San Diego, CA, pp. 1753-1758 (1994).
12. P. K. Artemiadis and K. J. Kyriakopoulos, EMG-based position and force control of a robot arm: Application to teleoperation and orthosis, in: Proc. IEEE/ASME Int. Conf. on Advanced Intelligence and Mechatronics, Zürich, pp. 1-6 (2007).
13. T. Tamei, S. Ishii and T. Shibata, Virtual force/tactile sensors for interactive machines using the user's biological signals, Adv. Robotics 22, 893-911 (2008).
14. K. Manal, R. V. Gonzalez, D. G. Lloyd and T. S. Buchanan, A real-time EMG-driven virtual arm, Comp. Biol. Med. 32, 25-36 (2002). (Pubitemid 34001640)
15. D. G. Lloyd and T. F. Besier, An EMG-driven musculoskeletal model to estimate muscle forces and knee joint moments in vivo, J. Biomech. 36, 765-776 (2003). (Pubitemid 36555374)
16. T. S. Buchanan, D. G. Lloyd, K. Manal and T. F. Besier, Neuromusculoskeletal modeling: Estimation of muscle forces and joint moments and movements from measurements of neural command, J. Appl. Biomech. 20, 367 (2004).
17. T. Tamei, S. Ishii and T. Shibata, Dynamic and cooperative interaction with a robot that possesses no force/tactile sensors, in: Proc. IEEE Int. Conf. on Advanced Robotics, Jeju, pp. 647-652 (2007).
18. P. R. Cavanagh and P. V. Komi, Electromechanical delay in human skeletal muscle under concentric and eccentric contractions, Eur. J. Appl. Physiol. 42, 159-163 (1979). (Pubitemid 10207312)
19. D. M. Corcos, G. L. Gottlieb, M. L. Latash, G. L. Almeida and G. C. Agarwal, Electromechanical delay: An experimental artifact, J. Electromyogr. Kinesiol. 2, 59-68 (1992).
20. D. G. Thelen, A. B. Schultz, S. D. Fassois and J. A. Ashton-Miller, Identification of dynamic myoelectric signal-to-force models during isometric lumbar muscle contractions, J. Biomech. 27, 907-919 (1994). (Pubitemid 24169714)
21. F. E. Zajac, Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control, Crit. Rev. Biomed. Eng. 17, 359-411 (1989).
22. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA (1998).
23. A. Tapus, C. Tapus and M. J. Mataric, Hands-off therapist robot behavior adaptation to user personality for post-stroke rehabilitation therapy, in: Proc. IEEE Int. Conf. on Robotics and Automation, Rome, pp. 1547-1553 (2007). (Pubitemid 350140402)
24. N. Mitsunaga, C. Smith, T. Kanda, H. Ishiguro and N. Hagita, Adapting robot behavior for human- robot interaction, IEEE Trans. Robotics 24, 911-916 (2008).
25. R. E. Bellman, A Markov decision process, J. Math. Mech. 6, 679-684 (1957).
26. L.Weaver and N. Tao, The optimal reward baseline for gradient-based reinforcement learning, in: Proc. 17th Conf. on Uncertainty in Artificial Intelligence, Seattle, WA, pp. 538-545 (2001).
27. J. Baxter and P. L. Bartlett, Infinite-horizon policy-gradient estimation, J. Artif. Intell. Res. 15, 319-350 (2001).