Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation

Computerized Medical Imaging and Graphics

Volumn 40, Issue , 2015, Pages 147-159

  Wang, Junchen ^a   Suenaga, Hideyuki ^b   Liao, Hongen ^c   Hoshi, Kazuto ^b   Yang, Liangjing ^a   Kobayashi, Etsuko ^a   Sakuma, Ichiro ^a  

^a UNIVERSITY OF TOKYO   (Japan)

^b UNIVERSITY OF TOKYO HOSPITAL   (Japan)

^c Tsinghua University   (China)

Author keywords

3D image calibration; Augmented reality; Integral imaging; Stereo tracking; Surgical navigation

Indexed keywords

AUGMENTED REALITY; CALIBRATION; COMPUTER GRAPHICS EQUIPMENT; GEOMETRY; GRAPHICS PROCESSING UNIT; IMAGE REGISTRATION; IMAGE RESOLUTION; MEDICAL IMAGING; NAVIGATION; OPTICAL VARIABLES MEASUREMENT; PIPELINE PROCESSING SYSTEMS; PROGRAM PROCESSORS; RENDERING (COMPUTER GRAPHICS); STEREO IMAGE PROCESSING; SURGERY; THREE DIMENSIONAL COMPUTER GRAPHICS;

3-D IMAGE; AUTOSTEREOSCOPIC 3D DISPLAYS; COMPUTER-GENERATED INTEGRAL IMAGING; INTEGRAL IMAGING; ORAL AND MAXILLOFACIAL SURGERIES; QUANTITATIVE EVALUATION; STEREO TRACKING; SURGICAL NAVIGATION;

THREE DIMENSIONAL DISPLAYS;

ARTICLE; AUGMENTED REALITY NAVIGATION; CALIBRATION; COMPUTER ASSISTED SURGERY; GEOMETRY; MAXILLOFACIAL SURGERY; MEASUREMENT ACCURACY; ORAL SURGERY; PHANTOM; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; SIMULATION; THREE DIMENSIONAL IMAGING; VIRTUAL REALITY; ALGORITHM; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; COMPUTER INTERFACE; COMPUTER SYSTEM; HUMAN; IMAGE ENHANCEMENT; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STANDARDS;

ALGORITHMS; CALIBRATION; COMPUTER SYSTEMS; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; ORAL SURGICAL PROCEDURES; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SURGERY, COMPUTER-ASSISTED; USER-COMPUTER INTERFACE;

EID: 84922990022     PISSN: 08956111     EISSN: 18790771     Source Type: Journal    
DOI: 10.1016/j.compmedimag.2014.11.003     Document Type: Article

References (29)

1. Nicolau S., Soler L., Mutter D., Marescaux J. Augmented reality in laparoscopic surgical oncology. Surg Oncol 2011, 20(3):189-201. 10.1016/j.suronc.2011.07.002.
2. Shekhar R., Dandekar O., Bhat V., Philip M., Lei P., Godinez C., et al. Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 2010, 24(8):1976-1985. 10.1007/s00464-010-0890-8.
3. Su L.-M., Vagvolgyi B.P., Agarwal R., Reiley C.E., Taylor R.H., Hager G.D. Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology 2009, 73(4):896-900. 10.1016/j.urology.2008.11.040.
4. Ferrari V., Megali G., Troia E., Pietrabissa A., Mosca F. A 3-d mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 2009, 56(11):2627-2633. 10.1109/TBME.2009.2028013.
5. Figl M., Ede C., Hummel J., Wanschitz F., Ewers R., Bergmann H., et al. A fully automated calibration method for an optical see-through head-mounted operating microscope with variable zoom and focus. IEEE Trans Med Imaging 2005, 24(11):1492-1499. 10.1109/TMI.2005.856746.
6. Teber D., Guven S., Simpfendrfer T., Baumhauer M., Gven E.O., Yencilek F., et al. Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? preliminary in vitro and in vivo results. Eur Urol 2009, 56(2):332-338. 10.1016/j.eururo.2009.05.017.
7. Masamune K., Sato I., Liao H., Dohi T. Non-metal slice image overlay display system used inside the open type MRI. Lecture notes in computer science 2008, vol. 5128:385-392. Springer, Berlin/Heidelberg. 10.1007/978-3-540-79982-5_42. T. Dohi, I. Sakuma, H. Liao (Eds.).
8. Fichtinger G., Deguet A., Masamune K., Balogh E., Fischer G., Mathieu H., et al. Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng 2005, 52(8):1415-1424. 10.1109/TBME.2005.851493.
9. Wen R., Chui C.-K., Ong S.-H., Lim K.-B., Chang S.-Y. Projection-based visual guidance for robot-aided RF needle insertion. Int J Comput Assist Radiol Surg 2013, 8(6):1015-1025. 10.1007/s11548-013-0897-4.
10. Osorio A., Galan J.-A., Nauroy J., Donars P. Real time planning, guidance and validation of surgical acts using 3D segmentations, augmented reality projections and surgical tools video tracking. Proc SPIE 2010, 7625. 10.1117/12.844039.
11. Krempien R., Hoppe H., Kahrs L., Daeuber S., Schorr O., Eggers G., et al. Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol Biol Phys 2008, 70(3):944-952. 10.1016/j.ijrobp.2007.10.048.
12. Dodgson N.A. Autostereoscopic 3D displays. Computer 2005, 8:31-36.
13. Lambooij M., Fortuin M., Heynderickx I., IJsselsteijn W. Visual discomfort and visual fatigue of stereoscopic displays: a review. J Imaging Sci 2009, 53(3). 10.2352/J.ImagingSci.Technol.2009.53.3.030201.
14. Martinez-Cuenca R., Saavedra G., Martinez-Corral M., Javidi B. Progress in 3-D multiperspective display by integral imaging. Proc IEEE 2009, 97(6):1067-1077. 10.1109/JPROC.2009.2016816.
15. Collier R. Optical holography 2013, Elsevier.
16. Lippmann G. Epreuves reversibles donnant la sensation du relief. J Phys Theor Appl 1908, 7(1):821-825.
17. Park J.-H., Hong K., Lee B. Recent progress in three-dimensional information processing based on integral imaging. Appl Opt 2009, 48(34):H77-H94. 10.1364/AO.48.000H77.
18. Stern A., Javidi B. Three-dimensional image sensing, visualization, and processing using integral imaging. Proc IEEE 2006, 94(3):591-607. 10.1109/JPROC.2006.870696.
19. Okano F., Hoshino H., Arai J., Yuyama I. Real-time pickup method for a three-dimensional image based on integral photography. Appl Opt 1997, 36(7):1598-1603. 10.1364/AO.36.001598.
20. Arimoto H., Javidi B. Integral three-dimensional imaging with digital reconstruction. Opt Lett 2001, 26(3):157-159. 10.1364/OL.26.000157.
21. Hong S.-H., Jang J.-S., Javidi B. Three-dimensional volumetric object reconstruction using computational integral imaging. Opt Exp 2004, 12(3):483-491. 10.1364/OPEX.12.000483.
22. Liao H., Iwahara M., Katayama Y., Hata N., Dohi T. Three-dimensional display with a long viewing distance by use of integral photography. Opt Lett 2005, 30(6):613-615. 10.1364/OL.30.000613.
23. Kim Y., Choi H., Kim J., Cho S.-W., Kim Y., Park G., et al. Depth-enhanced integral imaging display system with electrically variable image planes using polymer-dispersed liquid-crystal layers. Appl Opt 2007, 46(18):3766-3773. 10.1364/AO.46.003766.
24. Liao H., Hata N., Nakajima S., Iwahara M., Sakuma I., Dohi T. Surgical navigation by autostereoscopic image overlay of integral videography. IEEE Trans Inform Technol Biomed 2004, 8(2):114-121. 10.1109/TITB.2004.826734.
25. Liao H., Inomata T., Sakuma I., Dohi T. 3-d augmented reality for mri-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 2010, 57(6):1476-1486. 10.1109/TBME.2010.2040278.
26. Liao H., Ishihara H., Tran H.H., Masamune K., Sakuma I., Dohi T. Precision-guided surgical navigation system using laser guidance and 3d autostereoscopic image overlay. Comput Med Imaging Gr 2010, 34(1):46-54. 10.1016/j.compmedimag.2009.07.003.
27. Wang J., Suenaga H., Hoshi K., Yang L., Kobayashi E., Sakuma I., et al. Augmented reality navigation with automatic marker-free image registration using 3D image overlay for dental surgery. IEEE Trans Biomed Eng 2014, 61(4):1295-1304. 10.1109/TBME.2014.2301191.
28. Engel K., Kraus M., Ertl T. High-quality pre-integrated volume rendering using hardware-accelerated pixel shading. Proceedings of the ACM SIGGRAPH/EUROGRAPHICS workshop on graphics hardware, HWWS'01 2001, 9-16. ACM, New York, NY, USA. 10.1145/383507.383515.
29. Fitzpatrick J., West J. The distribution of target registration error in rigid-body point-based registration. IEEE Trans Med Imaging 2001, 20(9):917-927. 10.1109/42.952729.