Hand gesture guided robot-assisted surgery based on a direct augmented reality interface

Computer Methods and Programs in Biomedicine

Volumn 116, Issue 2, 2014, Pages 68-80

  Wen, Rong ^a   Tay, Wei Liang ^a   Nguyen, Binh P ^a   Chng, Chin Boon ^a   Chui, Chee Kong ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Augmented interaction; Augmented reality; Human robot cooperation; Image guided surgery; Projector camera system; Visual guidance

Indexed keywords

ABLATION; AUGMENTED REALITY; HAPTIC INTERFACES; ROBOTS; SURGERY;

AUGMENTED INTERACTION; HUMAN-ROBOT COOPERATION; IMAGE GUIDED SURGERY; PROJECTOR-CAMERA SYSTEM; VISUAL GUIDANCE;

SURGICAL EQUIPMENT;

ABLATION THERAPY; ARTICLE; COMPUTER ASSISTED SURGERY; COMPUTER INTERFACE; DECISION MAKING; GESTURE; HAND GESTURE; HUMAN; LIVER RESECTION; RADIOFREQUENCY ABLATION; ROBOT ASSISTED SURGERY; ROBOTICS; AUDIOVISUAL EQUIPMENT; CATHETER ABLATION; DEVICES; IMAGE QUALITY; LIVER NEOPLASMS; NEEDLE; ROBOTIC SURGICAL PROCEDURE;

CATHETER ABLATION; GESTURES; HUMANS; LIVER NEOPLASMS; MANIKINS; NEEDLES; PHANTOMS, IMAGING; ROBOTIC SURGICAL PROCEDURES; ROBOTICS; SURGERY, COMPUTER-ASSISTED; USER-COMPUTER INTERFACE;

EID: 84903118467     PISSN: 01692607     EISSN: 18727565     Source Type: Journal    
DOI: 10.1016/j.cmpb.2013.12.018     Document Type: Article

References (24)

1. Aggarwal R., Crochet P., Dias A., Misra A., Ziprin P., Darzi A. Development of a virtual reality training curriculum for laparoscopic cholecystectomy. British Journal of Surgery 2009, 96:1086-1093.
2. Cai Y.Y., Chui C.K., Ye X.Z., Fan Z., Anderson J.H. Tactile VR for handeye coordination in simulated PTCA. Computers in Biology and Medicine 2006, 36:167-180.
3. Du Q., Zhang X. Research on robotic technologies for ultrasound guided radio frequency ablation surgery. IEEE Proceedings of the International Conference on Industrial Technology (ICIT 2008) 2008, 1-6.
4. Ferrari V., Megali G., Troia E., Pietrabissa A., Mosca F. A 3-D mixed-reality system for stereoscopic visualization of medical dataset. IEEE Transactions on Biomedical Engineering 2009, 56:2627-2633.
5. Fichtinger G., Deguet A., Masamune K., Balogh E., Fischer G.S., Mathieu H., Taylor R.H., Zinreich S.J., Fayad L.M. Image overlay guidance for needle insertion in CT scanner. IEEE Transactions on Biomedical Engineering 2005, 52:1415-1424.
6. Gallo L., Placitelli A.P., Ciampi M. Controller-free exploration of medical image data: experiencing the Kinect. IEEE Proceedings of the 24th International Symposium on Computer-Based Medical Systems (CBMS) 2011, 1-6.
7. Hartley R., Zisserman A. Multiple View Geometry in Computer Vision 2004, Cambridge University Press, Cambridge, UK. second ed.
8. Hostettler A., Nicolau S., Rémond Y., Marescaux J., Soler L. A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Progress in Biophysics and Molecular Biology 2010, 103:169-184.
9. Jacob M., Li Y.T., Akingba G., Wachs J.P. Gestonurse: a robotic surgical nurse for handling surgical instruments in the operating room. Journal of Robotic Surgery 2012, 6:53-63.
10. Kipshagen T., Graw M., Tronnier V., Bonsanto M., Hofmann U. Touch- and marker-free interaction with medical software. World Congress on Medical Physics and Biomedical Engineering, volume 25/6 of IFMBE Proceedings 2007, 75-78. Springer. O. Dössel, W.C. Schlegel (Eds.).
11. Nguyen-Duc-Thanh N., Lee S., Kim D. Two-stage hidden markov model in gesture recognition for human robot interaction. International Journal of Advanced Robotic Systems 2012, 9:1-10.
12. Sainani N.I., Shyn P.B., Tatli S., Morrison P.R., Tuncali K., Silverman S.G. PET/CT-guided radiofrequency and cryoablation: is tumor fluorine-18 fluorodeoxyglucose activity dissipated by thermal ablation?. Journal of Vascular and Interventional Radiology 2011, 22:354-360.
13. Tay W.L., Chui C.K., Ong S.H. Single camera-based remote pointing and recognition for monitor-based augmented reality surgical systems. ACM Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry 2012, 35-38.
14. The Johns Hopkins University Gesture Based Surgical Manipulation of a da Vinci Robot Using a Kinect 2013, http://www.youtube.com/watch?v=YsY_A0kLh-g (accessed 25.07.13).
15. Vitiello V., Lee S.L., Cundy T., Yang G.Z. Emerging robotic platforms for minimally invasive surgery. IEEE Reviews in Biomedical Engineering 2013, 6:111-126.
16. Wachs J., Stern H., Edan Y., Gillam M., Feied C., Smith M., Handler J. Gestix: a doctor-computer sterile gesture interface for dynamic environments. Soft Computing in Industrial Applications, volume 39 of Advances in Soft Computing 2007, 30-39. Springer. A. Saad, K. Dahal, M. Sarfraz, R. Roy (Eds.).
17. Wachs J.P., Jacob M., Li Y.T., Akingba G. Does a robotic scrub nurse improve economy of movements?. Medical Imaging 2012: Image-Guided Procedures, Robotic Interventions, and Modeling, volume 8316 of Proceedings of SPIE 2012, 83160E-1-83160E-7. D.R. Holmes, K.H. Wong (Eds.).
18. Weinberger K.Q., Saul L.K. Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research 2009, 10:207-244.
19. Wen R., Chng C.B., Chui C.K., Lim K.B., Ong S.H., Chang S.K.Y. Robot-assisted RF ablation with interactive planning and mixed reality guidance. Proceedings of the IEEE/SICE International Symposium on System Integration (SII 2012) 2012, 31-36.
20. Wen R., Chui C.K., Ong S.H., Lim K.B., Chang S.K.Y. Projection-based visual guidance for robot-aided RF needle insertion. International Journal Computer Assisted Radiology Surgery 2013, 8:1015-1025.
21. Wong J.A., Matsumoto E.D. Primer: cognitive motor learning for teaching surgical skill-how are surgical skills taught and assessed?. Nature Reviews Urology 2008, 5:47-54.
22. Yang L., Wen R., Qin J., Chui C.K., Lim K.B., Chang S.K.Y. A robotic system for overlapping radiofrequency ablation in large tumor treatment. IEEE/ASME Transactions on Mechatronics 2010, 15:887-897.
23. Yang X., Yu H., Choi Y., Lee W., Wang B., Yang J., Hwang H., Kim J., Song J., Cho B., You H. A hybrid semi-automatic method for liver segmentation based on level-set methods using multiple seed points. Computer Methods and Programs in Biomedicine 2014, 13:69-76.
24. Zheng X., Vaezy S. A targeting method based on acoustic backscatter for treatment planning in tissue ablation using focused ultrasound. IEEE Transactions on Biomedical Engineering 2010, 57:71-79.