High-quality see-through surgical guidance system using enhanced 3-D autostereoscopic augmented reality

IEEE Transactions on Biomedical Engineering

Volumn 64, Issue 8, 2017, Pages 1815-1825

  Zhang, Xinran ^a   Chen, Guowen ^a   Liao, Hongen ^a  

^a Tsinghua University   (China)

Author keywords

Image guided surgery (IGS); Integral videography (IV); Microsurgery; Real 3 D see through device; Surgical guidance

Indexed keywords

AUGMENTED REALITY; DIAGNOSIS; END EFFECTORS; MEDICAL IMAGING; MEDICAL PROBLEMS; REMOTE CONTROL; STEREO IMAGE PROCESSING; SURGERY; TRANSPLANTATION (SURGICAL); VIDEO RECORDING;

IMAGE GUIDED SURGERY; INTEGRAL VIDEOGRAPHY; MICROSURGERY; REAL 3-D SEE-THROUGH DEVICE; SURGICAL GUIDANCE;

DISPLAY DEVICES;

ADULT; DEGRADATION; FEASIBILITY STUDY; HUMAN; IMAGE DISPLAY; MICROSCOPY; MICROSURGERY; PERCEPTION; PRECLINICAL STUDY; RELIABILITY; SURGEON; SURGERY; COMPUTER INTERFACE; DEVICE FAILURE ANALYSIS; DEVICES; EQUIPMENT DESIGN; EVALUATION STUDY; MINIMALLY INVASIVE SURGERY; NEURONAVIGATION; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

EQUIPMENT DESIGN; EQUIPMENT FAILURE ANALYSIS; MICROSURGERY; MINIMALLY INVASIVE SURGICAL PROCEDURES; NEURONAVIGATION; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; USER-COMPUTER INTERFACE;

EID: 85029232736     PISSN: 00189294     EISSN: 15582531     Source Type: Journal    
DOI: 10.1109/TBME.2016.2624632     Document Type: Article

References (36)

1. K. Cleary and T. Peters, "Image-guided interventions: Technology review and clinical applications," Annu. Rev. Biomed. Eng., vol. 12, pp. 119-142, 2010.
2. H. Liao and P. J. Edwards, "Introduction to the special issues of mixed reality guidance of therapy-towards clinical implementation," Comput. Med. Imaging Graph., vol. 2, no. 37, pp. 81-82, 2013.
3. X. Zhang et al., "3D augmented reality based orthopaedic interventions," Computational Radiology for Orthopaedic Interventions. New York, NY, USA: Springer, 2015, pp. 71-90.
4. J. Mcintire et al., "Stereoscopic 3D displays and human performance: A comprehensive review," Displays, vol. 13, no. 1, pp. 18-26, 2014.
5. K. Gavaghan et al., "A portable image overlay projection device for computer-aided open liver surgery," IEEE Trans. Biomed. Eng., vol. 58, no. 6, pp. 1855-1864, Jun. 2011.
6. G. Fichtinger et al., "Image overlay guidance for needle insertion in CT scanner," IEEE Trans. Biomed. Eng., vol. 52, no. 8, pp. 1415-1424, Aug. 2005.
7. K. Kihara et al., "New three-dimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: Procedures for gasless single-port radical nephrectomy," Int. J. Urol., vol. 19, no. 9, pp. 886-889, 2012.
8. X. Chen et al., "Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display," J. Biomed. Inf., vol. 55, no. C, pp. 124-131, 2014.
9. S. Mondal et al., "Binocular goggle augmented imaging and navigation system provides real-time fluorescence image guidance for tumor resection and sentinel lymph node mapping," Sci. Rep., vol. 5, 2015, Art. no. 12117.
10. S. Nicolau et al., "Augmented reality in laparoscopic surgical oncology," Surgical Oncol., vol. 20, no. 3, pp. 189-201, 2011.
11. P. Surman, "Stereoscopic and autostereoscopic displays," 3D-TV System With Depth-Image-Based Rendering. New York, NY, USA: Springer, 2013, pp. 375-411.
12. I. Cabrilo et al., "Augmented reality in the surgery of cerebral aneurysms: A technical report," Operative Neurosurgery, vol. 10, no. 2, pp. 252-261, 2014.
13. S. Min et al., "New characteristic equation of three- dimensional integral imaging system and its applications," Jpn. J. Appl. Phys., vol. 44, no. 1L, pp. L71-L74, 2005.
14. Y. Lim et al., "Resolution-enhanced integral imagingmicroscopy that uses lens array shifting," Opt. Express, vol. 17, no. 21, pp. 19253-19263, 2009.
15. H. Liao et al., "Scalable high-resolution integral videography autostereoscopic display with a seamlessmultiprojection system," Appl.Opt., vol. 44, no. 3, pp. 305-315, 2005.
16. J. Jang and B. Javidi, "Improvement of viewing angle in integral imaging by use of moving lenslet arrays with low fill factor," Appl. Opt., vol. 42, no. 11, pp. 1996-2002, 2003.
17. K. Katz, "Noise in the operating room," Anesthesiology, vol. 121, no. 4, pp. 894-898, 2014.
18. S. Nakajima et al., "Three-dimensional display system for medical imagingwith computer-generated integral photography," Electron. Imaging Int. Soc. Opt. Photon., vol. 3957, pp. 60-67, 2000.
19. V. Mahajan, Optical Imaging and Aberrations, Part I: Ray Geometrical Optics. Bellingham, WA, USA: SPIE, 1998, pp. 141-202.
20. H. Liao, "Super long viewing distance light homogeneous emitting threedimensional display," Sci. Rep., vol. 5, 2015, Art. no. 9523.
21. Y. Kim et al., "Analysis of off-axis integral floating system using concave mirror," J. Opt. Soc. Korea, vol. 16, no. 3, pp. 270-276, 2012.
22. Y. Itoh and G. Klinker, "Simultaneous direct and augmented view distortion calibration of optical see-through head-mounted displays," in Proc. 2015 IEEE Int. Symp. Mixed Augmented Reality, 2015, pp. 43-48.
23. X. Zhang et al., "A high accuracy surgical augmented reality system using enhanced integral videography image overlay," in Proc. 2015 37th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., 2015, pp. 4210-4213.
24. J. Wang et al., "A hybrid flexible rendering pipeline for real-time 3D medical imaging using GPU-accelerated integral videography," Int. J. Comput. Assisted Radiol. Surgery, vol. 8, Supp. 1 pp. 287-288, 2013.
25. M. Lippmann, "Preuves reversibles donnant la sensation du relief," J. de Physique, vol. 7, pp. 821-825, 1908.
26. H. Liao et al., "High quality autostereoscopic surgical display using antialiased integral videography imaging," in Proc. 7th Int. Cinf. Med. Image Comput. Comput.-Assisted Intervention, 2004, pp. 462-469.
27. J. Wang et al., "Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation," Comput. Med. Imaging Graph., vol. 40, pp. 147-159, 2015.
28. W. Zinmmerman, "Exact raytracing and spherical aberration," Learn Optics. Nov. 2008, pp. 63-83, ch. 2. [Online]. Available: http://www. learnoptics.com/Optics%20Chapters/opticschap2.pdf
29. H. Liao et al., "3D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay," IEEE Trans. Biomed. Eng., vol. 57, no. 6, pp. 1476-1486, Jun. 2010.
30. K. Aurn et al., "Least-squares fitting of two 3-D point sets," IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, no. 5, pp. 698-700, Sep. 1987.
31. Z. Fan et al., "3D quantitative evaluation system for autostereoscopic display," J. Display Technol., vol. 12, no. 10, pp. 1185-1196, Oct. 2016.
32. Z. Fan et al., "3D quantitative evaluation system for integral photography based 3D autostereoscopic medical display," in Proc. World Congr. Med. Phys. Biomed. Eng., 2015, pp. 850-853.
33. A. Davis and F. Kühnlenz, "Optical design using Fresnel lenses," Optik Photonik, vol. 2, no. 4, pp. 52-55, 2007.
34. N. Navab et al., "Action- And workflow-driven augmented reality for computer-aided medical procedures," IEEE Comput. Graph. Appl. vol. 25, no. 5, pp. 10-14, 2007.
35. C. Bichlmeier et al., "The virtual mirror: A new interaction paradigm for augmented reality environments," IEEE Trans. Med. Imag., vol. 28, no. 9, pp. 1498-1510, Sep. 2009.
36. C. Bohil et al., "Virtual reality in neuroscience research and therapy," Nature Rev. Neurosci., vol. 12, no. 12, pp. 752-762, 2011.