Humans Can Integrate Augmented Reality Feedback in Their Sensorimotor Control of a Robotic Hand

IEEE Transactions on Human-Machine Systems

Volumn 47, Issue 4, 2017, Pages 583-589

  Clemente, Francesco ^a   Dosen, Strahinja ^b   Lonini, Luca ^c   Markovic, Marko ^b   Farina, Dario ^d   Cipriani, Christian ^a  

^a SCUOLA SUPERIORE SANT ANNA   (Italy)

^b UNIVERSITY MEDICAL CENTER GÖTTINGEN   (Germany)

^c REHABILITATION INSTITUTE OF CHICAGO   (United States)

^d IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Augmented reality (AR); motor learning; sensorimotor control; sensory substitution; visual system; wearable systems

Indexed keywords

AUGMENTED REALITY; PROSTHETICS; ROBOTIC ARMS; ROBOTICS; SENSORY FEEDBACK; VISUAL COMMUNICATION; VISUAL SERVOING; WEARABLE COMPUTERS; WEARABLE TECHNOLOGY;

AMOUNT OF INFORMATION; ELECTROTACTILE STIMULATION; GRASPING AND MANIPULATION; SENSORIMOTOR CONTROL; TACTILE FEEDBACK; TACTILE SENSITIVITIES; TECHNOLOGICAL LIMITATIONS; VISUAL INFORMATION;

FEEDBACK;

EID: 84991071481     PISSN: 21682291     EISSN: None     Source Type: Journal    
DOI: 10.1109/THMS.2016.2611998     Document Type: Article

References (37)

1. R. F. FF. Weir and J.W. Sensinger, "Design of artificial arms and hands for prosthetic applications," in Standard Handbook of Biomedical Engineering and Design. New York, NY, USA: McGraw-Hill, 2003, pp. 537-598.
2. C. Antfolk, M. D'Alonzo, B. Rosén, G. Lundborg, F. Sebelius, and C. Cipriani, "Sensory feedback in upper limb prosthetics," Expert Rev. Med. Devices, vol. 10, no. 1, pp. 45-54, 2013.
3. D. S. Childress, "Closed-loop control in prosthetic systems: Historical perspective," Ann. Biomed. Eng., vol. 8, nos. 4-6, pp. 293-303, 1980.
4. G. S. Dhillon and K. W. Horch, "Direct neural sensory feedback and control of a prosthetic arm," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 4, pp. 468-472, Dec. 2005.
5. M. J. Ortiz-Catalan, B. Hakansson, and R. Branemark, "An osseointegrated human-machine gateway for long-term sensory feedback and motor control of artificial limbs," Sci. Transl. Med., vol. 6, no. 257, p. 257re6, 2014.
6. S.Raspopovic et al., "Restoring natural sensory feedback in real-time bidirectional hand prostheses," Sci. Transl. Med., vol. 6, no. 222, p. 222ra19, 2014.
7. D. W. Tan, M. A. Schiefer, M. W. Keith, J. R. Anderson, J. Tyler, and D. J. Tyler, "A neural interface provides long-term stable natural touch perception," Sci. Transl. Med., vol. 6, no. 257, p. 257ra138, Oct. 2014.
8. R. W. Mann and S. D. Reimers, "Kinesthetic sensing for the EMG controlled 'Boston Arm,"' IEEE Trans. Man-Mach. Syst., vol. 11, no. 1, pp. 110-115, Mar. 1970.
9. A. Chatterjee, P. Chaubey, J. Martin, and N. Thakor, "Testing a prosthetic haptic feedback simulator with an interactive force matching task," J. Prosthetics Orthotics, vol. 20, no. 2, pp. 27-34, 2008.
10. I. Saunders and S. Vijayakumar, "The role of feed-forward and feedback processes for closed-loop prosthesis control," J. Neuroeng. Rehabil., vol. 8, no. 1, p. 60, 2011.
11. S. Dosen, A. Ninu, T. Yakimovich, H. Dietl, and D. Farina, "A novel method to generate amplitude-frequency modulated vibrotactile stimulation," IEEE Trans. Haptics, vol. 9, no. 1, pp. 3-12, Jan.-Mar. 2015.
12. C. Antfolk, A. Björkman, S. O. Frank, F. Sebelius, G. Lundborg, and B. Rosen, "Sensory feedback from a prosthetic hand based on air-mediated pressure from the hand to the forearm skin," J. Rehabil. Med., vol. 44, no. 8, pp. 702-707, 2012.
13. F. Clemente, M. D'Alonzo, M. Controzzi, B. B. Edin, and C. Cipriani, "Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses," IEEE Trans. Neural Syst. Rehabil. Eng., to be published, doi: 10.1109/TNSRE.2015.2500586.
14. T. A. Kuiken, P. D. Marasco, B. A. Lock, R. N. Harden, and J. P. A. Dewald, "Redirection of cutaneous sensation from the hand to the chest skin of human amputees with targeted reinnervation," Proc. Nat. Acad. Sci. USA, vol. 104, no. 50, pp. 20061-20066, 2007.
15. C. Antfolk et al., "Artificial redirection of sensation from prosthetic fingers to the phantom hand map on transradial amputees: Vibrotactile versus mechanotactile sensory feedback," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 21, no. 1, pp. 112-120, Jan. 2013.
16. J. S. Schofield, K. R. Evans, J. P. Carey, and J. S. Hebert, "Applications of sensory feedback in motorized upper extremity prosthesis: A review," Expert Rev. Med. Devices, vol. 11, no. 5, pp. 1-13, 2014.
17. G. Lundborg, "Nerve injury and repair - A challenge to the plastic brain," J. Peripher. Nerv. Syst., vol. 8, no. 4, pp. 209-226, 2003.
18. B. Rosen, G. Lundborg, L. B. Dahlin, J. Holmberg, and B. Karlson, "Nerve repair: Correlation of restitution of functional sensibility with specific cognitive capacities," J. Hand Surgery Brit. Eur. Vol., vol. 19, no. 4, pp. 452-458, 1994.
19. C. Cipriani, J. L. Segil, F. Clemente, R. F.Weir, and B. B. Edin, "Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand," Exp. Brain Res., vol. 232, no. 11, pp. 3421-3429, 2014.
20. M. D'Alonzo, S. Dosen, C. Cipriani, and D. Farina, "HyVE-hybrid vibro-electrotactile stimulation-is an efficient approach to multi-channel sensory feedback," IEEE Trans. Haptics, vol. 7, no. 2, pp. 181-90, Apr.- Jun. 2014.
21. R. Sigrist, G. Rauter, R. Riener, and P.Wolf, "Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review," Psychon. Bull. Rev., vol. 20, no. 1, pp. 21-53, 2013.
22. E. D. Engeberg and S. Meek, "Enhanced visual feedback for slip prevention with a prosthetic hand," Prosthetics Orthotics Int., vol. 36, no. 4, pp. 423-429, 2012.
23. M. Markovic, S. Dosen, C. Cipriani, D. Popovic, and D. Farina, "Stereovision and augmented reality for closed-loop control of grasping in hand prostheses," J. Neural Eng., vol. 11, no. 4, 2014, Art. no. 046001.
24. R. Shadmehr,M. A Smith, and J.W. Krakauer, "Error correction, sensory prediction, and adaptation inmotor control," Annu. Rev. Neurosci., vol. 33, pp. 89-108, 2010.
25. A. Panarese, B. B. Edin, F. Vecchi, M. C. Carrozza, and R. S. Johansson, "Humans can integrate force feedback to toes in their sensorimotor control of a robotic hand," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 17, no. 6, pp. 560-567, Dec. 2009.
26. R. S. Johansson and B. B. Edin, "Predictive feed-forward sensory control during grasping and manipulation in man," Biomed. Res., vol. 14, pp. 95- 95, 1993.
27. E. L. Lehmann and H. J. D'Abrera, Nonparametrics: Statistical Methods Based on Ranks. San Francisco, CA, USA: Holden-Day, 1975.
28. D. Nowak, S. Glasauer, and J. Hermsdörfer, "How predictive is grip force control in the complete absence of somatosensory feedback?" Brain, vol. 127, no. 1, pp. 182-192, 2004.
29. C. E. Stepp and Y. Matsuoka, "Relative to direct haptic feedback, remote vibrotactile feedback improves but slows object manipulation," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biology Soc., 2010, pp. 2089-2092.
30. H. J. B. Witteveen, F. Luft, J. S. Rietman, and P. H. Veltink, "Stiffness feedback for myoelectric forearm prostheses using vibrotactile stimulation," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 22, no. 1, pp. 53-61, Jan. 2014.
31. R. S. Johansson and J. R. Flanagan, "Coding and use of tactile signals from the fingertips in object manipulation tasks," Nat. Rev. Neurosci., vol. 10, no. 5, pp. 345-359, 2009.
32. J. R. Flanagan, P. Vetter, R. S. Johansson, and D. M.Wolpert, "Prediction precedes control in motor learning," Curr. Biol., vol. 13, no. 2, pp. 146-50, 2003.
33. J. R. Flanagan,M. C. Bowman, and R. S. Johansson, "Control strategies in object manipulation tasks," Curr. Opin. Neurobiol., vol. 16, no. 6, pp. 650- 659, 2006.
34. N. Hogan, E. Bizzi, F. A. Mussa-Ivaldi, and T. Flash, "Controlling multijoint motor behavior," Exercise Sport Sci. Rev., vol. 15, pp. 153-190, 1987.
35. M. Kawato, "Internal models for motor control and trajectory planning," Curr. Opin. Neurobiol., vol. 9, no. 6, pp. 718-727, 1999.
36. S. Dosen, M. Markovic, K. Somer, B. Graimann, and D. Farina, "EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis," J. Neuroeng. Rehabil., vol. 12, no. 1, p. 55, 2015.
37. C. Cipriani, J. L. Segil, J. A. Birdwell, and R. F. FfWeir, "Dexterous control of a prosthetic hand using fine-wire intramuscular electrodes in targeted extrinsic muscles," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 22, no. 4, pp. 828-836, Jul. 2014.