Steering and Control of Miniaturized Untethered Soft Magnetic Grippers with Haptic Assistance

IEEE Transactions on Automation Science and Engineering

Volumn 15, Issue 1, 2018, Pages 290-306

  Pacchierotti, Claudio ^a   Ongaro, Federico ^b   Van Den Brink, Frank ^b   Yoon, Changkyu ^c   Prattichizzo, Domenico ^d,e   Gracias, David H ^c   Misra, Sarthak ^b,f  

^a CAMPUS DE BEAULIEU   (France)

^b UNIVERSITY OF TWENTE   (Netherlands)

^c Whiting School of Engineering   (United States)

^d UNIVERSITY OF SIENA   (Italy)

^e ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^f UNIVERSITY MEDICAL CENTER GRONINGEN   (Netherlands)

Author keywords

Grasping; haptics; magnetic control; microrobotics; microteleoperation; soft robotics

Indexed keywords

GRIPPERS; HAPTIC INTERFACES; MAGNETISM; ROBOTS;

HAPTIC ASSISTANCE; HAPTIC SENSATION; MICRO MANIPULATION; MICRO-TELEOPERATION; REMOTE ENVIRONMENT; TARGETED DRUG DELIVERY; THERMAL CONTROL SYSTEMS; WEAK MAGNETIC FIELDS;

POSITION CONTROL;

EID: 85009944765     PISSN: 15455955     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASE.2016.2635106     Document Type: Article

References (75)

1. A. A. Solovev, Y. Mei, E. B. Ureña, G. Huang, and O. G. Schmidt, "Catalytic microtubular jet engines self-propelled by accumulated gas bubbles," Small, vol. 5, no. 14, pp. 1688-1692, 2009.
2. S. P. Woods and T. G. Constandinou, "Towards a micropositioning system for targeted drug delivery in wireless capsule endoscopy," in Proc. Int. Conf. IEEE Eng. Med. Biol. Soc., Sep. 2011, pp. 7372-7375.
3. H. Marino, C. Bergeles, and B. J. Nelson, "Robust electromagnetic control of microrobots under force and localization uncertainties," IEEE Trans. Autom. Sci. Eng., vol. 11, no. 1, pp. 310-316, Jan. 2014.
4. M. Rakotondrabe, C. Clévy, K. Rabenorosoa, and K. Ncir, "Presentation, force estimation and control of an instrumented platform dedicated to automated micromanipulation tasks," in Proc. IEEE Conf. Autom. Sci. Eng. (CASE), Aug. 2010, pp. 722-727.
5. S. Balasubramanian et al., "Micromachine-enabled capture and isolation of cancer cells in complex media," Angew. Chem. Int. Ed., vol. 50, no. 18, pp. 4161-4164, 2011.
6. J. Kim, H. Ladjal, D. Folio, A. Ferreira, and J. Kim, "Evaluation of telerobotic shared control strategy for efficient single-cell manipulation," IEEE Trans. Autom. Sci. Eng., vol. 9, no. 2, pp. 402-406, Apr. 2012.
7. L. Zhang, T. Petit, K. E. Peyer, and B. J. Nelson, "Targeted cargo delivery using a rotating nickel nanowire," Nanomed. Nanotechnol., Biol. Med., vol. 8, no. 7, pp. 1074-1080, 2012.
8. E. Gultepe et al., "Biopsy with thermally-responsive untethered microtools," Adv. Mater., vol. 25, no. 4, pp. 514-519, 2013.
9. M. Matteucci et al., "A compact and disposable transdermal drug delivery system," Microelectron. Eng., vol. 85, no. 5, pp. 1066-1073, 2008.
10. W. Xi, A. A. Solovev, A. N. Ananth, D. H. Gracias, S. Sanchez, and O. G. Schmidt, "Rolled-up magnetic microdrillers: Towards remotely controlled minimally invasive surgery," Nanoscale, vol. 5, no. 4, pp. 1294-1297, 2013.
11. M. Guix et al., "Superhydrophobic alkanethiol-coated microsubmarines for effective removal of oil," ACS Nano, vol. 6, no. 5, pp. 4445-4451, 2012.
12. L. Soler, V. Magdanz, V. M. Fomin, S. Sanchez, and O. G. Schmidt, "Self-propelled micromotors for cleaning polluted water," ACS Nano, vol. 7, no. 11, pp. 9611-9620, 2013.
13. F. Kuralay, S. Sattayasamitsathit, W. Gao, A. Uygun, A. Katzenberg, and J. Wang, "Self-propelled carbohydrate-sensitive microtransporters with built-in boronic acid recognition for isolating sugars and cells," J. Amer. Chem. Soc., vol. 134, no. 37, pp. 15217-15220, 2012.
14. J. Orozco et al., "Molecularly imprinted polymer-based catalytic micromotors for selective protein transport," J. Amer. Chem. Soc., vol. 135, no. 14, pp. 5336-5339, 2013.
15. H. Li, J. Tan, and M. Zhang, "Dynamics modeling and analysis of a swimming microrobot for controlled drug delivery," IEEE Trans. Autom. Sci. Eng., vol. 6, no. 2, pp. 220-227, Apr. 2009.
16. B. J. Nelson, I. K. Kaliakatsos, and J. J. Abbott, "Microrobots for minimally invasive medicine," Annu. Rev. Biomed. Eng., vol. 12, pp. 55-85, Aug. 2010.
17. K. Malachowski et al., "Stimuli-responsive theragrippers for chemomechanical controlled release," Angew. Chem., vol. 126, no. 31, pp. 8183-8187, 2014.
18. S. Sanchez, A. A. Solovev, S. Schulze, and O. G. Schmidt, "Controlled manipulation of multiple cells using catalytic microbots," Chem. Commun., vol. 47, no. 2, pp. 698-700, 2011.
19. S. Martel and M. Mohammadi, "Using a swarm of self-propelled natural microrobots in the form of flagellated bacteria to perform complex micro-assembly tasks," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2010, pp. 500-505.
20. W. Gao et al., "Artificial micromotors in the mouse's stomach: A step toward in vivo use of synthetic motors," ACS Nano, vol. 9, no. 1, pp. 117-123, 2015.
21. W. Gao, A. Pei, and J. Wang, "Water-driven micromotors," ACS Nano, vol. 6, no. 9, pp. 8432-8438, 2012.
22. S. R. Platt, J. A. Hawks, and M. E. Rentschler, "Vision and task assistance using modular wireless in vivo surgical robots," IEEE Trans. Biomed. Eng., vol. 56, no. 6, pp. 1700-1710, Jun. 2009.
23. K. Harada et al., "A reconfigurable modular robotic endoluminal surgical system: Vision and preliminary results," Robotica, vol. 28, no. 2, pp. 171-183, 2010.
24. A. C. Lehman, N. A. Wood, S. Farritor, M. R. Goede, and D. Oleynikov, "Dexterous miniature robot for advanced minimally invasive surgery," Surgical Endoscopy, vol. 25, no. 1, pp. 119-123, 2011.
25. J. S. Randhawa, K. E. Laflin, N. Seelam, and D. H. Gracias, "Microchemomechanical systems," Adv. Funct. Mater., vol. 21, no. 13, pp. 2395-2410, 2011.
26. M. A. C. Stuart et al., "Emerging applications of stimuli-responsive polymer materials," Nature Mater., vol. 9, no. 2, pp. 101-113, 2010.
27. R. R. Syms, E. M. Yeatman, V. M. Bright, and G. M. Whitesides, "Surface tension-powered self-assembly of microstructures-The stateof-the-art," J. Microelectromech. Syst., vol. 12, no. 4, pp. 387-417, Aug. 2003.
28. T. G. Leong, C. L. Randall, B. R. Benson, N. Bassik, G. M. Stern, and D. H. Gracias, "Tetherless thermobiochemically actuated microgrippers," Proc. Nat. Acad. Sci. USA, vol. 106, no. 3, pp. 703-708, 2009.
29. F. Ilievski, A. D. Mazzeo, R. F. Shepherd, X. Chen, and G. M. Whitesides, "Soft robotics for chemists," Angew. Chem., vol. 123, no. 8, pp. 1930-1935, 2011.
30. J. C. Breger et al., "Self-folding thermo-magnetically responsive soft microgrippers," ACS Appl. Mater. Interfaces, vol. 7, no. 5, pp. 3398-3405, 2015.
31. J. Troccaz and Y. Delnondedieu, "Semi-active guiding systems in surgery. A two-dof prototype of the passive arm with dynamic constraints (PADyC)," Mechatronics, vol. 6, no. 4, pp. 399-421, 1996.
32. M. Jakopec, F. Rodriguez y Baena, S. J. Harris, P. Gomes, J. Cobb, and B. L. Davies, "The hands-on orthopaedic robot 'acrobot': Early clinical trials of total knee replacement surgery," IEEE Trans. Robot. Autom., vol. 19, no. 5, pp. 902-911, 2003.
33. C. Pacchierotti, M. Abayazid, S. Misra, and D. Prattichizzo, "Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback," IEEE Trans. Haptics, vol. 7, no. 4, pp. 551-556, Apr. 2014.
34. A. M. Okamura, "Methods for haptic feedback in teleoperated robotassisted surgery," Ind. Robot, Int. J., vol. 31, no. 6, pp. 499-508, 2004.
35. Z. F. Quek, S. B. Schorr, I. Nisky, W. R. Provancher, and A. M. Okamura, "Sensory substitution and augmentation using 3-degreeof-freedom skin deformation feedback," IEEE Trans. Haptics, vol. 8, no. 2, pp. 209-221, Apr. 2015.
36. C. Pacchierotti, L. Meli, F. Chinello, M. Malvezzi, and D. Prattichizzo, "Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems," Int. J. Robot. Res., vol. 34, no. 14, pp. 1773-1787, 2015.
37. A. M. Jarc and I. Nisky, "Robot-assisted surgery: An emerging platform for human neuroscience research," Frontiers Human Neurosci., vol. 9, p. 315, Jun. 2015.
38. C. W. Kennedy, T. Hu, J. P. Desai, A. S. Wechsler, and J. Y. Kresh, "A novel approach to robotic cardiac surgery using haptics and vision," Cardiovascular Eng., vol. 2, no. 1, pp. 15-22, 2002.
39. S. Salcudean, S. Ku, and G. Bell, "Performance measurement in scaled teleoperation for microsurgery," in Proc. 1st Joint Conf. Comput. Vis. Virtual Reality Robot. Med. Robot. Comput.-Assisted Surgery, 1997, pp. 789-798.
40. A. Kazi, "Operator performance in surgical telemanipulation," Presence, Teleoperators Virtual Environ., vol. 10, no. 5, pp. 495-510, 2001.
41. C. Pacchierotti, D. Prattichizzo, and K. J. Kuchenbecker, "Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery," IEEE Trans. Biomed. Eng., vol. 63, no. 2, pp. 278-287, Feb. 2016.
42. A. Pillarisetti, M. Pekarev, A. D. Brooks, and J. P. Desai, "Evaluating the effect of force feedback in cell injection," IEEE Trans. Autom. Sci. Eng., vol. 4, no. 3, pp. 322-331, Jul. 2007.
43. W. Wang, X. Liu, and Y. Sun, "Autonomous zebrafish embryo injection using a microrobotic system," in Proc. IEEE Conf. Autom. Sci. Eng. (CASE), Sep. 2007, pp. 363-368.
44. C. Pacchierotti, S. Scheggi, D. Prattichizzo, and S. Misra, "Haptic feedback for microrobotics applications: A review," Frontiers Robot. AI, vol. 3, no. 53, 2016.
45. S.-G. Kim and M. Sitti, "Task-based and stable telenanomanipulation in a nanoscale virtual environment," IEEE Trans. Autom. Sci. Eng., vol. 3, no. 3, pp. 240-247, Jul. 2006.
46. M. Mehrtash, N. Tsuda, and M. B. Khamesee, "Bilateral macro-micro teleoperation using magnetic levitation," IEEE/ASME Trans. Mechatronics, vol. 16, no. 3, pp. 459-469, Mar. 2011.
47. A. Bolopion and S. Régnier, "A review of haptic feedback teleoperation systems for micromanipulation and microassembly," IEEE Trans. Autom. Sci. Eng., vol. 10, no. 3, pp. 496-502, Jul. 2013.
48. C. Pacchierotti, V. Magdanz, M. Medina-Sánchez, O. G. Schmidt, D. Prattichizzo, and S. Misra, "Intuitive control of self-propelled microjets with haptic feedback," J. Micro-Bio Robot., vol. 10, nos. 1-4, pp. 37-53, 2015.
49. A. Ghanbari, B. Horan, S. Nahavandi, X. Chen, and W. Wang, "Haptic microrobotic cell injection system," IEEE Syst. J., vol. 8, no. 2, pp. 371-383, Jun. 2014.
50. F. Ongaro, C. Pacchierotti, C. Yoon, D. Prattichizzo, D. H. Gracias, and S. Misra, "Evaluation of an electromagnetic system with haptic feedback for control of untethered, soft grippers affected by disturbances," in Proc. IEEE Int. Conf. Biomed. Robot. Biomechatron., Jun. 2016, pp. 900-905.
51. S. Süsstrunk, R. Buckley, and S. Swen, "Standard RGB color spaces," in Proc. Color Imag. Conf., 1999, pp. 127-134.
52. W. K. Pratt, "Image feature extraction," in Digital Image Processing. Hoboken, NJ, USA: Wiley, 1978, pp. 535-577.
53. G. Reeb, "Sur les points singuliers d'une forme de Pfaff complètement intégrable ou d'une fonction numérique," CR Acad. Sci., vol. 222, pp. 847-849, 1946.
54. C. L. Bajaj, V. Pascucci, and D. R. Schikore, "The contour spectrum," in Proc. 8th Conf. Vis., 1997, pp. 167-173.
55. H. Carr, J. Snoeyink, and M. van de Panne, "Simplifying flexible isosurfaces using local geometric measures," in Proc. Conf. Vis., 2004, pp. 497-504.
56. S. Suzuki and K. Abe, "Topological structural analysis of digitized binary images by border following," Comput. Vis., Graph., Image Process., vol. 30, no. 1, pp. 32-46, 1985.
57. F. van der Heijden, Image Based Measurement Systems: Object Recognition and Parameter Estimation. Hoboken, NJ, USA: Wiley, 1994.
58. F. van der Heijden, "On the discretization error of Fourier descriptors of planar closed curves," Signal Process. III, Theories Appl., pp. 669-672, 1986.
59. C.-L. Huang and D.-H. Huang, "A content-based image retrieval system," Image Vis. Comput., vol. 16, no. 3, pp. 149-163, 1998.
60. M. D. Heath, S. Sarkar, T. Sanocki, and K. W. Bowyer, "A robust visual method for assessing the relative performance of edge-detection algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 12, pp. 1338-1359, Dec. 1997.
61. K. Bowyer and P. J. Phillips, Empirical Evaluation Techniques in Computer Vision. Los Alamitos, CA, USA: IEEE Computer Society Press, 1998.
62. M. Abayazid, C. Pacchierotti, P. Moreira, R. Alterovitz, D. Prattichizzo, and S. Misra, "Experimental evaluation of co-manipulated ultrasoundguided flexible needle steering," Int. J. Med. Robot. Comput. Assist. Surgery, vol. 12, no. 2, pp. 219-230, 2016.
63. I. S. M. Khalil, V. Magdanz, S. Sanchez, O. G. Schmidt, and S. Misra, "The control of self-propelled microjets inside a microchannel with time-varying flow rates," IEEE Trans. Robot., vol. 30, no. 1, pp. 49-58, Feb. 2014.
64. I. S. M. Khalil, L. Abelmann, and S. Misra, "Magnetic-based motion control of paramagnetic microparticles with disturbance compensation," IEEE Trans. Magn., vol. 50, no. 10, pp. 1-10, Oct. 2014.
65. A. S. Bahaj, P. A. B. James, and F. D. Moeschler, "An alternative method for the estimation of the magnetic moment of nonspherical magnetotactic bacteria," IEEE Trans. Magn., vol. 32, no. 5, pp. 5133-5135, Sep. 1996.
66. R. Penrose, "A generalized inverse for matrices," Math. Proc. Cambridge Philos. Soc., vol. 51, no. 3, pp. 406-413, 1955.
67. J. G. Ziegler and N. B. Nichols, "Optimum settings for automatic controllers," ASME J. Dyn. Syst., Meas., Control, vol. 115, no. 2B, pp. 220-222, 1993.
68. M. Franken, S. Stramigioli, S. Misra, C. Secchi, and A. Macchelli, "Bilateral telemanipulation with time delays: A two-layer approach combining passivity and transparency," IEEE Trans. Robot., vol. 27, no. 4, pp. 741-756, Aug. 2011.
69. Z. Ni, C. Pacoret, R. Benosman, and S. Regnier, Haptic Feedback Teleoperation of Optical Tweezers. Hoboken, NJ, USA: Wiley, 2014.
70. R. W. Cholewiak and A. A. Collins, "Sensory and physiological bases of touch," in The Psychology of Touch. Hillsdale, NJ, USA: Lawrence Erlbaum Associates, 1991, pp. 23-60.
71. J. J. Faraway, "Practical regression and ANOVA using R," Tech. Rep., 2002. [Online]. Available: ftp://cran.r-project.org/pub/R/doc/contrib/Faraway-PRA.pdf
72. C. B. Zilles and J. K. Salisbury, "A constraint-based god-object method for haptic display," in Proc. IEEE Int. Conf. Human Robot Interact. Cooper. Robots, vol. 3, Aug. 1995, pp. 146-151.
73. K. Hashtrudi-Zaad and S. E. Salcudean, "Transparency in time-delayed systems and the effect of local force feedback for transparent teleoperation," IEEE Trans. Robot. Autom., vol. 18, no. 1, pp. 108-114, Feb. 2002.
74. K. Malachowski, M. Jamal, Q. Jin, B. Polat, C. J. Morris, and D. H. Gracias, "Self-folding single cell grippers," Nano Lett., vol. 14, no. 7, pp. 4164-4170, 2014.
75. A. Sánchez, V. Magdanz, O. G. Schmidt, and S. Misra, "Magnetic control of self-propelled microjets under ultrasound image guidance," in Proc. 5th IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatron., Aug. 2014, pp. 169-174.