From medical images to minimally invasive intervention: Computer assistance for robotic surgery

Computerized Medical Imaging and Graphics

Volumn 34, Issue 1, 2010, Pages 33-45

  Lee, Su Lin ^a   Lerotic, Mirna ^a   Vitiello, Valentina ^a   Giannarou, Stamatia ^a   Kwok, Ka Wai ^a   Visentini Scarzanella, Marco ^a   Yang, Guang Zhong ^a  

^a Wolfson Foundation Medical Image Computing Laboratory   (United Kingdom)

Author keywords

3D reconstruction; Active constraints; Augmented reality; Feature tracking; Haptic feedback; Image guidance; Medical imaging; Minimally invasive surgery; Robotic assisted surgery

Indexed keywords

3D RECONSTRUCTION; ACTIVE CONSTRAINTS; FEATURE-TRACKING; HAPTIC FEEDBACK; HAPTIC FEEDBACKS; MINIMALLY INVASIVE SURGERY;

AUGMENTED REALITY; COMPUTER SUPPORTED COOPERATIVE WORK; DEFORMATION; IMAGE RECONSTRUCTION; MACHINE DESIGN; MEDICAL IMAGING; REPAIR; ROBOTICS; ROBOTS; SURGERY; THREE DIMENSIONAL; VIRTUAL REALITY;

SURGICAL EQUIPMENT;

ARTICLE; COMPUTER ASSISTED SURGERY; COMPUTER INTERFACE; EQUIPMENT DESIGN; HUMAN; IMAGE ANALYSIS; IMAGE PROCESSING; IMAGE RECONSTRUCTION; INTRAOPERATIVE PERIOD; MINIMALLY INVASIVE SURGERY; NATURAL ORIFICE TRANSLUMINAL ENDOSCOPIC SURGERY; PREOPERATIVE PERIOD; PRIORITY JOURNAL; ROBOTICS; SURGICAL EQUIPMENT; SURGICAL TECHNIQUE; SURGICAL TECHNOLOGY; THREE DIMENSIONAL IMAGING; TISSUE INJURY; TREATMENT PLANNING;

COMPUTER SIMULATION; ELASTICITY IMAGING TECHNIQUES; HUMANS; IMAGING, THREE-DIMENSIONAL; MODELS, BIOLOGICAL; ROBOTICS; SURGERY, COMPUTER-ASSISTED; SURGICAL PROCEDURES, MINIMALLY INVASIVE; USER-COMPUTER INTERFACE;

EID: 72049132096     PISSN: 08956111     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.compmedimag.2009.07.007     Document Type: Article

References (90)

1. Dario P., Hannaford B., and Menciassi A. Smart surgical tools and augmenting devices. IEEE Transactions on Robotics and Automation 19 5 (2003) 782-792
2. Mack M.J. Minimally invasive and robotic surgery. JAMA 285 5 (2001) 568-572
3. Nakamura Y., Kishi K., and Kawakami H. Heartbeat synchronization for robotic cardiac surgery. Proceedings of the 2001 IEEE international conference on robotics and automation (ICRA), vol. 2 (2001) 2014-2019
4. Ginhoux R., Gangloff J., de Mathelin M., Soler L., Sanchez M.M.A., and Marescaux J. Active filtering of physiological motion in robotized surgery using predictive control. IEEE Transactions on Robotics 21 1 (2005) 67-79
5. Sauvée M., Noce A., Poignet P., Triboulet J., and Dombre E. Three-dimensional heart motion estimation using endoscopic monocular vision system: from artificial landmarks to texture analysis. Biomedical Signal Processing and Control 2 3 (2007) 199-207
6. Dey D., Gobbi D.G., Slomka P.J., Surry K.J.M., and Peters T.M. Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas. IEEE Transactions on Medical Imaging 21 1 (2002) 23-30
7. Thoranaghatte R., Zheng G., Langlotz F., and Nolte L. Endoscope-based hybrid navigation system for minimally invasive ventral spine surgeries. Computer Aided Surgery 10 5-6 (2005) 351-356
8. Siddique S., and Jaffray D. Towards active image guidance: tracking of a fiducial in the thorax during respiration under X-ray fluoroscopy. Proceedings of the SPIE international symposium on medical imaging 2007: visualization and image-guided procedures, vol. 6509 (2007) 29-41
9. Giannarou S, Visentini-Scarzanella M, Yang G-Z. Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery. IEEE international symposium on biomedical imaging (ISBI) 2009;1059-62.
10. Mountney P, Lo B, Thiemjarus S, Stoyanov D, Yang G-Z. A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2007. LNCS 4792;2007:34-41.
11. Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang G-Z. Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2005. LNCS 2005;3750:139-46.
12. Ortmaier T., Groger M., Boehm D.H., Falk V., and Hirzinger G. Motion estimation in beating heart surgery. IEEE Transactions on Biomedical Engineering 52 10 (2005) 1729-1740
13. Groeger M., Ortmaier T., Sepp W., and Hirzinger G. Tracking local motion on the beating heart. Proceedings of SPIE medical imaging 2002: visualization, image-guided procedures, and display, vol. 4681 (2002) 233-241
14. Burschka D., Li M., Ishii M., Taylor R.H., and Hager G.D. Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery. Medical Image Analysis 9 5 (2005) 413-426
15. Hager G.D., and Belhumeur P.N. Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 10 (1998) 1025-1039
16. Bourger F., Doignon C., Zanne P., and de Mathelin M. A model-free vision-based robot control for minimally invasive surgery using ESM tracking and pixels color selection. IEEE International Conference on Robotics and Automation (2007) 3579-3584
17. Scharstein D., and Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47 1 (2002) 7-42
18. Komodakis N., Tziritas G., and Paragios N. Performance vs computational efficiency for optimizing single and dynamic MRFs: setting the state of the art with primal-dual strategies. Computer Vision and Image Understanding 112 1 (2008) 14-29
19. Tappen M.F., and Freeman W.T. Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. Proceedings of the ninth IEEE international conference on computer vision, vol. 2 (2003) 900-906
20. Lau WW, Ramey NA, Corso JJ, Thakor NV, Hager GD. Stereo-based endoscopic tracking of cardiac surface deformation. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2004. LNCS 2004;3217:494-501.
21. Richa R, Poignet P, Liu C. Efficient 3D tracking for motion compensation in beating heart surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:684-91.
22. Stoyanov D., Darzi A., and Yang G.-Z. A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery. Computer Aided Surgery 10 4 (2005) 199-208
23. Horn BKP. Shape from shading: a method for obtaining the shape of a smooth opaque object from one view. Ph.D. MIT; 1970.
24. Rashid H., and Burger P. Differential algorithm for the determination of shape from shading using a point light source. Image and Vision Computing 10 2 (1992) 119-127
25. Deguchi K., and Okatani T. Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center. Proceedings of the workshop on mathematical methods in biomedical image analysis (1996) 290-298
26. Rouy E., and Tourin A. A viscosity solutions approach to shape-from-shading. SIAM Journal on Numerical Analysis 29 3 (1992) 867-884
27. Tankus A., Sochen N., and Yeshurun Y. Reconstruction of medical images by perspective shape-from-shading. 17th international conference on pattern recognition (ICPR'04), vol. 3 (2004) 778-781
28. Kimmel R., and Bruckstein A.M. Global shape from shading. Computer Vision and Image Understanding 62 3 (1995) 360-369
29. Yuen S.Y., Tsui Y.Y., and Chow C.K. A fast marching formulation of perspective shape from shading under frontal illumination. Pattern Recognition Letters 28 7 (2007) 806-824
30. Prados E., Camilli F., and Faugeras O. A unifying and rigorous shape from shading method adapted to realistic data and applications. Journal of Mathematical Imaging and Vision 25 3 (2006) 307-328
31. Yuille A., and Kersten D. Vision as Bayesian inference: analysis by synthesis?. Trends in Cognitive Sciences 10 7 (2006) 301-308
32. Du W, Piater J. A probabilistic approach to integrating multiple cues in visual tracking. In: Computer Vision - ECCV 2008. LNCS 2008;5303:225-38.
33. Saxena A., Chung S.H., and Ng A.Y. 3-D depth reconstruction from a single still image. International Journal of Computer Vision 76 1 (2008) 53-69
34. White R., and Forsyth D.A. Combining cues: shape from shading and texture. 2006 IEEE computer society conference on computer vision and pattern recognition, vol. 2 (2006) 1809-1816
35. Lo BPL, Visentini-Scarzanella M, Stoyanov D, Yang G-Z. Belief propagation for depth cue fusion in minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:104-12.
36. Stoyanov D, Mylonas G, Yang G-Z. Gaze-contingent 3D control for focused energy ablation in robotic assisted surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:347-55.
37. Masood S., Yang G.-Z., Pennell D.J., and Firmin D.N. Investigating intrinsic myocardial mechanics: the role of MR tagging, velocity phase mapping, and diffusion imaging. Journal of Magnetic Resonance Imaging 12 6 (2000) 873-883
38. Axel L., and Dougherty L. MR imaging of motion with spatial modulation of magnetization. Radiology 171 3 (1989) 841-845
39. Osman N.F., Sampath S., Atalar E., and Prince J.L. Imaging longitudinal cardiac strain on short-axis images using strain-encoded MRI. Magnetic Resonance in Medicine 46 2 (2001) 324-334
40. Aletras A.H., Ding S., Balaban R.S., and Wen H. DENSE: displacement encoding with stimulated echoes in cardiac functional MRI. Journal of Magnetic Resonance 137 (1999) 247-252
41. Bryant D.J., Payne J.A., Firmin D.N., and Longmore D.B. Measurement of flow with NMR imaging using a gradient pulse and phase difference technique. Journal of Computer Assisted Tomography 8 4 (1984) 588-593
42. Nayler G.L., Firmin D.N., and Longmore D.B. Blood flow imaging by cine magnetic resonance. Journal of Computer Assisted Tomography 10 5 (1986) 715-722
43. Huntbatch A, Lee S-L, Firmin D, Yang G-Z. Bayesian motion recovery framework for myocardial phase-contrast velocity MRI. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:79-86.
44. Peyrat J.M., Sermesant M., Pennec X., Delingette H., Chenyang X., McVeigh E.R., et al. A computational framework for the statistical analysis of cardiac diffusion tensors: application to a small database of canine hearts. IEEE Transactions on Medical Imaging 26 11 (2007) 1500-1514
45. Tseng W-YI, Dou J., Reese T.G., and Wedeen V.J. Imaging myocardial fiber disarray and intramural strain hypokinesis in hypertrophic cardiomyopathy with MRI. Journal of Magnetic Resonance Imaging 23 1 (2006) 1-8
46. Cootes T., Taylor C., Cooper D., and Graham J. Active shape models-their training and application. Computer Vision and Imaging Understanding 61 1 (1995) 38-59
47. Horkaew P, Yang G-Z. Construction of 3D dynamic statistical deformable models for complex topological shapes. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2004. LNCS 2004;3216:217-24.
48. Lee S.-L., Horkaew P., Caspersz W., Darzi A., and Yang G.-Z. Assessment of shape variation of the levator ani with optimal scan planning and statistical shape modeling. Journal of Computer Assisted Tomography 29 2 (2005) 154-162
49. Lee S.-L., Tan E., Khullar V., Gedroyc W., Darzi A., and Yang G.-Z. Physical-based statistical shape modeling of the levator ani. IEEE Transactions on Medical Imaging 28 6 (2009) 926-936
50. Lorenzo-Valdés M., Sanchez-Ortiz G.I., Elkington A.G., Mohiaddin R.H., and Rueckert D. Segmentation of 4D cardiac MR images using a probabilistic atlas and the EM algorithm. Medical Image Analysis 8 3 (2004) 255-265
51. Lekadir K, Yang G-Z. Optimal feature point selection and automatic initialization in active shape model search. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5241:434-41.
52. Ecabert O., Peters J., Schramm H., Lorenz C., von Berg J., Walker M.J., et al. Automatic model-based segmentation of the heart in CT images. IEEE Transactions on Medical Imaging 27 9 (2008) 1189-1201
53. Masood S., Gao J., and Yang G.-Z. Virtual tagging: numerical considerations and phantom validation. IEEE Transactions on Medical Imaging 21 9 (2002) 1123-1131
54. Lee S-L, Wu Q, Huntbatch A, Yang G-Z. Predictive K-PLSR myocardial contractility modeling with phase contrast MR velocity mapping. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2007;4792:866-73.
55. Lee S-L, Huntbatch A, Yang G-Z. Contractile analysis with Kriging based on MR myocardial velocity imaging. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5241:892-9.
56. Taylor R.H., and Stoianovici D. Medical robotics in computer-integrated surgery. IEEE Transactions on Robotics and Automation 19 5 (2003) 765-781
57. Peters T.M. Image-guidance for surgical procedures. Physics in Medicine and Biology 51 (2006) R505-R540
58. Bajura M., Fuchs H., and Ohbuchi R. Merging virtual objects with the real world: seeing ultrasound imagery within the patient. 19th annual conference on computer graphics and interactive techniques (1992) 203-210
59. Abolmaesumi P., Salcudean S.E., and Zhu W.H. Visual servoing for robot-assisted diagnostic ultrasound. Proceedings of the 22nd annual international conference of the IEEE, vol. 4. Engineering in Medicine and Biology Society (2000) 2532-2535
60. Bainbridge D., Jones D.L., Guiraudon G.M., and Terrence M.P. Ultrasound image and augmented reality guidance for off-pump, closed, beating, intracardiac surgery. Artificial Organs 32 11 (2008) 840-845
61. Rickers C., Jerosch-Herold M., Hu X., Murthy N., Wang X., Kong H., et al. Magnetic resonance image-guided transcatheter closure of atrial septal defects. Circulation 107 1 (2003) 132-138
62. Johnson L.G., Edwards P., and Hawkes D. Surface transparency makes stereo overlays unpredictable: the implications for augmented reality. Studies in Health Technology and Informatics 94 (2003) 131-136
63. Sielhorst T, Bichlmeier C, Heining SM, Navab N. Depth perception-a major issue in medical AR: evaluation study by twenty surgeons. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2006. LNCS 2006;4190:364-72.
64. Bichlmeier C, Heining M, Feuerstein M, Navab N. The virtual mirror: a new interaction paradigm for augmented reality environments. IEEE Transactions on Medical Imaging; in press.
65. Lerotic M, Chung AJ, Mylonas G, Yang G-Z. pq-space based non-photorealistic rendering for augmented reality. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2007. LNCS 2007;4792:102-9.
66. Horn B.K.P. Height and gradient from shading. International Journal of Computer Vision 5 1 (1990) 37-75
67. Chiu A.M., Dey D., Drangova M., Boyd W.D., and Peters T.M. 3-D image guidance for minimally invasive robotic coronary artery bypass. The Heart Surgery Forum 3 3 (2000) 224-231
68. Coste-Manière E., Adhami L., Mourgues F., and Carpentier A. Planning, simulation, and augmented reality for robotic cardiac procedures: the STARS system of the ChIR team. Seminars in Thoracic and Cardiovascular Surgery 15 2 (2003) 141-156
69. Dombre E, Michelin M, Pierrot F, Poignet P, Bidaud P, Morel G, et al. MARGE project: design, modeling and control of assistive devices for minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2004. LNCS 2004;3217:1-8.
70. Michelin M., Poignet P., and Dombre E. Dynamic task/posture decoupling for minimally invasive surgery motions: simulation results. Proceedings of the 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS 2004), vol. 4 (2004) 3625-3630
71. Taylor R., Jensen P., Whitcomb L., Barnes A., Kumar R., Stoianovici D., et al. A steady-hand robotic system for microsurgical augmentation. The International Journal of Robotics Research 18 12 (1999) 1201-1210
72. Ang WT, Riviere CN, Khosla PK. An active hand-held instrument for enhanced microsurgical accuracy. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2000. LNCS 2000;1935:878-86.
73. Patronik NA, Zenati MA, Riviere CN. Crawling on the heart: a mobile robotic device for minimally invasive cardiac interventions. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2004. LNCS 2004;3217:9-16.
74. D'Attanasio S., Tonet O., Megali G., Carrozza M., and Dario P. A semi-automatic handheld mechatronic endoscope with collision-avoidance capabilities. IEEE International Conference on Robotics and Automation 2 (2000) 1586-1591
75. Swain P. A justification for NOTES-natural orifice translumenal endosurgery. Gastrointestinal Endoscopy 65 3 (2007) 514-516
76. Lerotic M, Chung AJ, Clark J, Valibeik S, Yang G-Z. Dynamic view expansion for enhanced navigation in natural orifice transluminal endoscopic surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:467-75.
77. Craige J.C. Tactile pattern perception and its perturbations. The Journal of the Acoustical Society of America 77 1 (1985) 238-246
78. Okamura A.M. Methods for haptic feedback in teleoperated robot-assisted surgery. Industrial Robot: An International Journal 31 5 (2004) 499-508
79. Pezzementi Z., Ursu D., Misra S., and Okamura A.M. Modeling realistic tool-tissue interactions with haptic feedback: a learning-based method. Symposium on haptic interfaces for virtual environment and teleoperator systems (Haptics 2008) (2008) 209-215
80. Mahvash M., and Okamura A.M. Friction compensation for enhancing transparency of a teleoperator with compliant transmission. IEEE Transactions on Robotics 23 6 (2007) 1240-1246
81. Basdogan C., De S., Kim J., Muniyandi M., Kim H., and Srinivasan M.A. Haptics in minimally invasive surgical simulation and training. IEEE Computer Graphics and Applications 24 2 (2004) 56-64
82. Davies B., Jakopec M., Harris S.J., Baena F.R.Y., Barrett A., Evangelidis A., et al. Active-constraint robotics for surgery. Proceedings of the IEEE 94 9 (2006) 1696-1704
83. Jakopec M., Baena F.R.Y., Harris S.J., Gomes P., Cobb J., and Davies B.L. The hands-on orthopaedic robot "Acrobot": early clinical trials of total knee replacement surgery. IEEE Transactions on Robotics and Automation 19 5 (2003) 902-911
84. Rosenberg L.B. Virtual fixtures: perceptual tools for telerobotic manipulation. Proceedings of the IEEE annual international symposium on virtual reality (1993) 76-82
85. Burschka D., Corso J.J., Dewan M., Lau W., Li M., Lin H., et al. Navigating inner space: 3-D assistance for minimally invasive surgery. Robotics and Autonomous Systems 52 1 (2005) 5-26
86. Ren J., Patel R.V., McIsaac K.A., Guiraudon G., and Peters T.M. Dynamic 3-D virtual fixtures for minimally invasive beating heart procedures. IEEE Transactions on Medical Imaging 27 8 (2008) 1061-1070
87. Mylonas G.P., Darzi A., and Yang G.Z. Gaze-contingent control for minimally invasive robotic surgery. Computer Aided Surgery 11 5 (2006) 256-266
88. Mylonas G, Kwok KW, Darzi A, Yang G-Z. Gaze-contingent motor channelling and haptic constraints for minimally invasive robotic surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2008. LNCS 2008;5242:676-83.
89. Fofi D., Sliwa T., and Voisin Y. A comparative survey on invisible structured light. Proceedings of SPIE 5303 (2004) 90-98
90. Mylonas GP, Darzi A, Yang G-Z. Gaze contingent depth recovery and motion stabilisation for minimally invasive robotic surgery. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2004. LNCS 2004;3150:311-9.