Stereovision and augmented reality for closed-loop control of grasping in hand prostheses

Journal of Neural Engineering

Volumn 11, Issue 4, 2014, Pages

  Markovic, Marko ^a   Dosen, Strahinja ^a   Cipriani, Christian ^b   Popovic, Dejan ^c   Farina, Dario ^a  

^a UNIVERSITY MEDICAL CENTER GÖTTINGEN   (Germany)

^b SCUOLA SUPERIORE SANT ANNA   (Italy)

^c University of Belgrade   (Serbia)

Author keywords

artificial proprioception; augmented reality; closed loop control; control of grasping; stereovision

Indexed keywords

PROSTHETICS; SENSORY PERCEPTION; STEREO VISION; VISUAL COMMUNICATION;

ASSISTIVE DEVICES; AUTOMATIC SYSTEMS; CLOSED-LOOP CONTROL; CLUTTERED ENVIRONMENTS; REHABILITATION DEVICES; SEMI-AUTONOMOUS CONTROL; SENSORY INFORMATION; USER FRIENDLY INTERFACE;

AUGMENTED REALITY;

ALGORITHM; ARTICLE; ASSISTIVE TECHNOLOGY DEVICE; BRAIN COMPUTER INTERFACE; GRIP STRENGTH; HAND MOVEMENT; HAND PROSTHESIS; HUMAN; HUMAN EXPERIMENT; MYOELECTRIC CONTROL; NORMAL HUMAN; PRIORITY JOURNAL; PROPRIOCEPTIVE FEEDBACK; ROBOTICS; STEREOVISION CAMERA; VIRTUAL REALITY; BINOCULAR VISION; ELECTROMYOGRAPHY; HAND; HAND STRENGTH; PHYSIOLOGY; PROSTHESES AND ORTHOSES; PROSTHESIS; SENSORY FEEDBACK; STATISTICAL ANALYSIS;

ALGORITHMS; DATA INTERPRETATION, STATISTICAL; ELECTROMYOGRAPHY; FEEDBACK, SENSORY; HAND; HAND STRENGTH; HUMANS; PROSTHESES AND IMPLANTS; PROSTHESIS DESIGN; ROBOTICS; VISION, BINOCULAR;

EID: 84903777636     PISSN: 17412560     EISSN: 17412552     Source Type: Journal    
DOI: 10.1088/1741-2560/11/4/046001     Document Type: Article

References (61)

1. MacKenzie C and Iberall T 2010 The Grasping Hand (Amsterdam: Elsevier)
2. Bootsma R J and van Wieringen P C W 1992 Spatio-temporal organisation of natural prehension Hum. Mov. Sci. 11 205-15
3. Connolly J D and Goodale M a 1999 The role of visual feedback of hand position in the control of manual prehension Exp. Brain Res. 125 281-6
4. Popović M B 2003 Control of neural prostheses for grasping and reaching Med. Eng. Phys. 25 41-50 (Pubitemid 35453965)
5. Balasubramanian S, Klein J and Burdet E 2010 Robot-assisted rehabilitation of hand function Curr. Opin. Neurol. 23 661-70
6. Cipriani C, Controzzi M and Carrozza M C 2011 The SmartHand transradial prosthesis J. Neuroeng. Rehabil. 8 29
7. Heo P, Gu G M, Lee S, Rhee K and Kim J 2012 Current hand exoskeleton technologies for rehabilitation and assistive engineering Int. J. Precis. Eng. Manuf. 13 807-24
8. Touch Bionics iLimb 2014 www.touchbionics.com/products/active-prostheses/
9. Bebionic Hand 2014 http://bebionic.com/the-hand
10. Parker P, Englehart K and Hudgins B 2006 Myoelectric signal processing for control of powered limb prostheses J. Electromyogr. Kinesiol. 16 541-8 (Pubitemid 44666457)
11. Xiao R and Ding L 2013 Evaluation of EEG features in decoding individual finger movements from one hand Comput. Math. Methods Med. 2013 243257
12. Pistohl T, Schulze-Bonhage A, Aertsen A, Mehring C and Ball T 2012 Decoding natural grasp types from human ECoG NeuroImage 59 248-60
13. Carrozza M C, Persichetti A, Laschi C, Vecchi F, Lazzarini R, Vacalebri P and Dario P 2007 A wearable biomechatronic interface for controlling robots with voluntary foot movements IEEE /ASME Trans. Mechatronics 12 1-11 (Pubitemid 46444213)
14. Johansen D, Popović D B, Struijk L N S A, Sebelius F and Jensen S 2012 A novel hand prosthesis control scheme implementing a tongue control system Int. J. Eng. Manuf. 2 14
15. Controzzi M, Cipriani C, Popovic D B and Carrozza M C 2013 Controlling hand-assistive devices: utilizing electrooculography as a substitute for vision IEEE Robot. Autom. Mag. 20 40-52
16. Asghari Oskoei M and Hu H 2007 Myoelectric control systems - a survey Biomed. Signal Process. Control 2 275-94
17. Peerdeman B et al 2011 Myoelectric forearm prostheses: state of the art from a user-centered perspective J. Rehab. Res. Develop. 48 719-37
18. Tenore F V G, Ramos A, Fahmy A, Acharya S, Etienne-Cummings R and Thakor N V 2009 Decoding of individuated finger movements using surface electromyography IEEE Trans. Biomed. Eng. 56 1427-34
19. Cipriani C, Antfolk C, Controzzi M, Lundborg G, Rosen B, Carrozza M C and Sebelius F 2011 Online myoelectric control of a dexterous hand prosthesis by transradial amputees IEEE Trans. Neural Syst. Rehabil. Eng. 19 260-70
20. Micera S et al 2011 Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces J. Neuroeng. Rehabil. 8 53
21. Baker J J, Scheme E, Englehart K, Hutchinson D T and Greger B 2010 Continuous detection and decoding of dexterous finger flexions with implantable myoelectric sensors IEEE Trans. Neural Syst. Rehabil. Eng. 18 424-32
22. Szeto A Y and Saunders F A 1982 Electrocutaneous stimulation for sensory communication in rehabilitation engineering IEEE Trans. Biomed. Eng. 29 300-8
23. Kaczmarek K A, Webster J G, Bach-y-Rita P and Tompkins W J 1991 Electrotactile and vibrotactile displays for sensory substitution systems IEEE Trans. Biomed. Eng. 38 1-16
24. Patterson P E and Katz J A 1992 Design and evaluation of a sensory feedback system that provides grasping pressure in a myoelectric hand J. Rehabil. Res. Dev. 29 1-8
25. Meek S G, Jacobsen S C and Goulding P P 1989 Extended physiologic taction: design and evaluation of a proportional force feedback system J. Rehabil. Res. Dev. 26 53-62 (Pubitemid 19189051)
26. Scott R N, Brittain R H, Caldwell R R, Cameron A B and Dunfield V A 1980 Sensory-feedback system compatible with myoelectric control Med. Biol. Eng. Comput. 18 65-69 (Pubitemid 10210706)
27. Pylatiuk C, Kargov A and Schulz S 2006 Design and evaluation of a low-cost force feedback system for myoelectric prosthetic hands J. Prosthet. Orthot. 18 57-61 (Pubitemid 44510417)
28. Cipriani C, D'Alonzo M and Carrozza M C 2012 A miniature vibrotactile sensory substitution device for multifingered hand prosthetics IEEE Trans. Biomed. Eng. 59 400-8
29. D'Alonzo M, Dosen S, Cipriani C and Farina D 2013 HyVE: hybrid vibro-electrotactile stimulation for sensory feedback and substitution in rehabilitation IEEE Trans. Neural Syst. Rehabil. Eng. 22 290-301
30. Dhillon G S and Horch K W 2005 Direct neural sensory feedback and control of a prosthetic arm IEEE Trans. Neural Syst. Rehabil. Eng. 13 468-72 (Pubitemid 43019184)
31. Karnati N, Kent B A and Engeberg E D 2013 Bioinspired sinusoidal finger joint synergies for a dexterous robotic hand to screw and unscrew objects with different diameters IEEE/ASME Trans. Mechatronics 18 612-23
32. Malhotra M, Rombokas E, Theodorou E, Todorov E and Matsuoka Y 2012 Reduced dimensionality control for the ACT hand IEEE Int. Conf. on Robotics and Automation pp 5117-22
33. Matrone G C, Cipriani C, Carrozza M C and Magenes G 2012 Real-time myoelectric control of a multi-fingered hand prosthesis using principal components analysis J. Neuroeng. Rehabil. 9 40
34. Dosen S, Muller K-R and Farina D 2012 Myoelectric control of artificial limbs - is there a need to change focus? IEEE Signal Process. Mag. 29 150-2
35. Dosen S, Cipriani C, Kostić M, Controzzi M, Carrozza M C and Popović D B 2010 Cognitive vision system for control of dexterous prosthetic hands: experimental evaluation J. Neuroeng. Rehabil. 7 42
36. Tomovic R and Boni G 1962 An adaptive artificial hand IRE Trans. Autom. Control 7 3-10
37. Tomovic R, Bekey G and Karplus W 1987 A strategy for grasp synthesis with multifingered robot hands Proc. IEEE Int. Conf. on Robotics and Automation vol 4 pp 83-89
38. Chappell P H, Nightingale J M, Kyberd P J and Barkhordar M 1987 Control of a single degree of freedom artificial hand J. Biomed. Eng. 9 273-7 (Pubitemid 17106500)
39. Otto Bock Sensor Hand Speed 2014 www.ottobock.de/cps/rde/xchg/ob-com-en/ hs.xsl/3652.html
40. Engeberg E D and Meek S G 2013 Adaptive sliding mode control for prosthetic hands to simultaneously prevent slip and minimize deformation of grasped objects IEEE/ASME Trans. Mechatronics 18 376-85
41. Wettels N, Parnandi A R, Loeb G E and Sukhatme G S 2009 Grip control using biomimetic tactile sensing systems IEEE/ASME Trans. Mechatronics 14 718-23
42. Light C M, Chappell P H, Hudgins B and Engelhart K 2002 Intelligent multifunction myoelectric control of hand prostheses J. Med. Eng. Technol. 26 139-46 (Pubitemid 35176669)
43. Cipriani C, Zaccone F, Micera S and Carrozza M C 2008 On the shared control of an EMG-controlled prosthetic hand: analysis of user-prosthesis interaction IEEE Trans. Robot. 24 170-84 (Pubitemid 351411206)
44. Corbett E A, Körding K P and Perreault E J 2013 Real-time evaluation of a noninvasive neuroprosthetic interface for control of reach IEEE Trans. Neural Syst. Rehabil. Eng. 21 674-83
45. Kragic D, Miller A T and Allen P K 2001 Real-time tracking meets online grasp planning ICRA'01: Proc. IEEE Int. Conf. on Robotics and Automation vol 3 pp 2460-5 (Cat. No.01CH37164)
46. Google Glass Project 2014 www.google.com/glass/start/
47. Geiger A, Roser M and Urtasun R 2010 Computer Vision-ACCV (Lecture Notes in Computer Sciences vol 6492) (Berlin: Springer)
48. Bouguet J-Y 2013 Camera Calibration Toolbox for Matlab www.vision.caltech.edu/bouguetj/calib-doc/
49. Fischler M A and Bolles R C 1981 Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Commun. ACM 24 381-95
50. Cutkosky M R 1989 On grasp choice, grasp models, and the design of hands for manufacturing tasks IEEE Trans. Robot. Autom. 5 269-79
51. Siebert J P and Cyganek B 2009 An Introduction to 3D Computer Vision Techniques and Algorithms (New York: Wiley) p 504
52. Hutchinson S, Hager G D and Corke P I 1996 A tutorial on visual servo control IEEE Trans. Robot. Autom. 12 651-70 (Pubitemid 126786243)
53. Childress D S 1980 Closed-loop control in prosthetic systems: historical perspective Ann. Biomed. Eng. 8 293-303 (Pubitemid 11064542)
54. The da Vinci® Surgical System 2014 www.davincisurgery.com/da-vinci- surgery/da-vinci-surgical-system/
55. Prime Sense Sensors 2014 www.primesense.com/
56. Creative Senz3D 2014 http://de.creative.com/p/web-cameras/creative-senz3d
57. PCL 2014 http://pointclouds.org/
58. Ciocarlie M T and Allen P K 2009 Hand posture subspaces for dexterous robotic grasping Int. J. Rob. Res. 28 851-67
59. Dosen S, Markovic M, Popović D B, Farina D and Graimann B Sensor fusion for control of upper limb protheses: integration of myoelectric control with stereovision, augmented reality and inertial sensing Patent number EP 13171671.42013
60. Subramanian S, Knaut L A, Beaudoin C, McFadyen B J, Feldman A G and Levin M F 2007 Virtual reality environments for post-stroke arm rehabilitation J. Neuroeng. Rehabil. 4 20
61. Murray C D, Pettifer S, Howard T, Patchick E L, Caillette F, Kulkarni J and Bamford C 2007 The treatment of phantom limb pain using immersive virtual reality: three case studies Disabil. Rehabil. 29 1465-9 (Pubitemid 47343376)