Registration and fusion quantification of augmented reality based nasal endoscopic surgery

Medical Image Analysis

Volumn 42, Issue , 2017, Pages 241-256

  Chu, Yakui ^a   Yang, Jian ^a   Ma, Shaodong ^a   Ai, Danni ^a   Li, Wenjie ^a   Song, Hong ^a   Li, Liang ^b   Chen, Duanduan ^a   Chen, Lei ^b   Wang, Yongtian ^a  

^a BEIJING INSTITUTE OF TECHNOLOGY   (China)

^b CHINESE PLA GENERAL HOSPITAL   (China)

Author keywords

Augmented reality; Endoscope; Fusion; Image registration; Image guided surgery; Optical tracking

Indexed keywords

AUGMENTED REALITY; CALIBRATION; COMPUTERIZED TOMOGRAPHY; ECONOMIC AND SOCIAL EFFECTS; ERROR ANALYSIS; FUSION REACTIONS; IMAGE REGISTRATION; RENDERING (COMPUTER GRAPHICS); SURGERY;

IMAGE GUIDED SURGERY; INNOVATIVE APPROACHES; OPTICAL TRACKING; PERFORMANCE EVALUATIONS; REGISTRATION ACCURACY; SPATIAL CALIBRATION; STRUCTURAL RELIABILITY; TARGET REGISTRATION;

ENDOSCOPY;

ANALYTICAL ERROR; ARTICLE; AUGMENTED REALITY BASED NASAL ENDOSCOPIC SURGERY; CADAVER; CALIBRATION; CANCER SURGERY; ENDOSCOPE; EYE TRACKING; HUMAN; MINIMALLY INVASIVE SURGERY; PRIORITY JOURNAL; SKULL BASE TUMOR; SURGICAL NAVIGATION SYSTEM; VALIDITY; VISUAL INFORMATION; ALGORITHM; COMPUTER ASSISTED SURGERY; COMPUTER INTERFACE; DIAGNOSTIC IMAGING; ENDOSCOPY; NOSE DISEASE; PROCEDURES; THREE DIMENSIONAL IMAGING; X-RAY COMPUTED TOMOGRAPHY;

ALGORITHMS; CALIBRATION; ENDOSCOPY; HUMANS; IMAGING, THREE-DIMENSIONAL; NOSE DISEASES; SURGERY, COMPUTER-ASSISTED; TOMOGRAPHY, X-RAY COMPUTED; USER-COMPUTER INTERFACE;

EID: 85028710943     PISSN: 13618415     EISSN: 13618423     Source Type: Journal    
DOI: 10.1016/j.media.2017.08.003     Document Type: Article

References (50)

1. Aboul-Hosn Centenero, S., Hernández-Alfaro, F., 3D planning in orthognathic surgery: CAD/CAM surgical splints and prediction of the soft and hard tissues results - our experience in 16 cases. J. Cranio-Maxillofacial Surg. 40 (2012), 162–168, 10.1016/j.jcms.2011.03.014.
2. Armstrong, D.G., Rankin, T.M., Giovinco, N.A., Mills, J.L., Matsuoka, Y., A heads-up display for diabetic limb salvage surgery: a view through the google looking glass. J. Diabetes Sci. Technol. 8 (2014), 951–956, 10.1177/1932296814535561.
3. Bankman, I., Handbook of Medical Image Processing and Analysis, 13, 2009, Optics Photonics News, 984 pages.
4. Baumhauer, M., Feuerstein, M., Meinzer, H.-P., Rassweiler, J., Navigation in endoscopic soft tissue surgery: perspectives and limitations. J. Endourol. 22 (2008), 751–766, 10.1089/end.2007.9827.
5. Benet, A., Rincon-Torroella, J., Lawton, M.T., González Sánchez, J.J., Novel embalming solution for neurosurgical simulation in cadavers. J. Neurosurg. 120 (2014), 1229–1237, 10.3171/2014.1.JNS131857.
6. Besharati Tabrizi, L., Mahvash, M., Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J. Neurosurg. 123 (2015), 206–211, 10.3171/2014.9.JNS141001.
7. Brouwer, O.R., Buckle, T., Bunschoten, A., Kuil, J., Vahrmeijer, A.L., Wendler, T., Valdés-Olmos, R.a., van der Poel, H.G., van Leeuwen, F.W.B., Image navigation as a means to expand the boundaries of fluorescence-guided surgery. Phys. Med. Biol. 57 (2012), 3123–3136, 10.1088/0031-9155/57/10/3123.
8. Chen, X., Xu, L., Wang, Y., Wang, H., Wang, F., Zeng, X., Wang, Q., Egger, J., Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J. Biomed. Inf. 55 (2015), 124–131, 10.1016/j.jbi.2015.04.003.
9. Chen, Y., Shi, L., Feng, Q., Yang, J., Shu, H., Luo, L., Coatrieux, J.L., Chen, W., Artifact suppressed dictionary learning for low-dose CT image processing. IEEE Trans. Med. Imaging 33 (2014), 2271–2292, 10.1109/TMI.2014.2336860.
10. Cong, W., Yang, J., Deng, W., Zhu, J., Automatic radial distortion correction for endoscope image. Proceedings of the 2013 6th International Congress on Image and Signal Processing, 2013, CISP, 932–937, 10.1109/CISP.2013.6745298 2013.
11. Daly, M.J., Muhanna, N., Chan, H., Wilson, B.C., Irish, J.C., Jaffray, D.A., A surgical navigation system for non-contact diffuse optical tomography and intraoperative cone-beam CT. Prog. Biomed. Opt. Imaging Proc. SPIE, 8937, 2014, 893703, 10.1117/12.2040176.
12. Dixon, B.J., Daly, M.J., Chan, H., Vescan, A., Witterick, I.J., Irish, J.C., Augmented real-time navigation with critical structure proximity alerts for endoscopic skull base surgery. Laryngoscope, 2014, 853–859, 10.1002/lary.24385.
13. Eggers, G., Mühling, J., Marmulla, R., Image-to-patient registration techniques in head surgery. Int. J. Oral Maxillofac. Surg., 2006, 10.1016/j.ijom.2006.09.015.
14. Fan, J., Yang, J., Chu, Y., Kinect based real-time position calibration for nasal endoscopic surgical navigation system. SPIE Med. … 9786 (2016), 1–7, 10.1117/12.2216599.
15. Fan, J., Yang, J., Lu, F., Ai, D., Zhao, Y., Wang, Y., 3-points convex hull matching (3PCHM) for fast and robust point set registration. Neurocomputing., 2015, 10.1016/j.neucom.2016.01.078.
16. Fan, J., Yang, J., Zhao, Y., Ai, D., Liu, Y., Wang, G., Wang, Y., Convex hull aided registration method (CHARM). IEEE Trans. Vis. Comput. Graph., 2016, 10.1109/TVCG.2016.2602858.
17. Feuerstein, M., Mussack, T., Heining, S.M., Navab, N., Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans. Med. Imaging 27 (2008), 355–369, 10.1109/TMI.2007.907327.
18. Fitzpatrick, J.M., The role of registration in accurate surgical guidance. Proc. Inst. Mech. Eng. H. 224 (2010), 607–622, 10.1243/09544119JEIM589.
19. Hayashi, Y., Misawa, K., Oda, M., Hawkes, D.J., Mori, K., Clinical application of a surgical navigation system based on virtual laparoscopy in laparoscopic gastrectomy for gastric cancer. Int. J. Comput. Assist. Radiol. Surg., 2015, 827–836, 10.1007/s11548-015-1293-z.
20. Ieiri, S., Uemura, M., Konishi, K., Souzaki, R., Nagao, Y., Tsutsumi, N., Akahoshi, T., Ohuchida, K., Ohdaira, T., Tomikawa, M., Tanoue, K., Hashizume, M., Taguchi, T., Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device. Pediatr. Surg. Int. 28 (2012), 341–346, 10.1007/s00383-011-3034-x.
21. Li, L., Yang, J., Chu, Y., Wu, W., Xue, J., Liang, P., Chen, L., A novel augmented reality navigation system for endoscopic sinus and skull base surgery: a feasibility study. PLoS One 11 (2016), 1–17, 10.1371/journal.pone.0146996.
22. Liao, H., Inomata, T., Sakuma, I., Dohi, T., 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans. Biomed. Eng. 57 (2010), 1476–1486, 10.1109/TBME.2010.2040278.
23. Lorensen, W.E., Cline, H.E., Marching cubes: a high resolution 3D surface construction algorithm. Proc. 14th Annu. Conf. Comput. Graph. Interact. Tech. - SIGGRAPH ‘87, 21, 1987, 163–169, 10.1145/37402.37422.
24. Luebbers, H.T., Messmer, P., Obwegeser, J.A., Zwahlen, R.A., Kikinis, R., Graetz, K.W., Matthews, F., Comparison of different registration methods for surgical navigation in cranio-maxillofacial surgery. J. Cranio-Maxillofacial Surg. 36 (2008), 109–116, 10.1016/j.jcms.2007.09.002.
25. Luo, X., Kitasaka, T., Mori, K., Externally navigated bronchoscopy using 2-D motion sensors: dynamic phantom validation. IEEE Trans. Med. Imaging 32 (2013), 1745–1764, 10.1109/TMI.2013.2263152.
26. Luo, Z., Cai, J., Wang, S., Zhao, Q., Peters, T.M., Gu, L., Magnetic navigation for thoracic aortic stent-graft deployment using ultrasound image guidance. IEEE Trans. Biomed. Eng. 60 (2013), 862–871, 10.1109/TBME.2012.2206388.
27. McLaughlin, N., Carrau, R.L., Kassam, A.B., Kelly, D.F., Neuronavigation in endonasal pituitary and skull base surgery using an autoregistration mask without head fixation: an assessment of accuracy and practicality. J. Neurol. Surgery Part A Cent. Eur. Neurosurg. 73 (2012), 351–357, 10.1055/s-0032-1326943.
28. Mirota, D.J., Uneri, A., Schafer, S., Nithiananthan, S., Reh, D.D., Ishii, M., Gallia, G.L., Taylor, R.H., Hager, G.D., Siewerdsen, J.H., Evaluation of a system for high-accuracy 3D image-based registration of endoscopic video to c-arm cone-beam CT for image-guided skull base surgery. IEEE Trans. Med. Imaging 32 (2013), 1215–1226, 10.1109/TMI.2013.2243.
29. Mirota, D.J., Wang, H., Taylor, R.H., Ishii, M., Gallia, G.L., Hager, G.D., A system for video-Based navigation for endoscopic endonasal skull base surgery. IEEE Trans. Med. Imaging 31 (2012), 963–976, 10.1109/TMI.2011.2176500.
30. Nickel, F., Kenngott, H.G., Neuhaus, J., Sommer, C.M., Gehrig, T., Kolb, A., Gondan, M., Radeleff, B.A., Schaible, A., Meinzer, H.P., Gutt, C.N., Muller-Stich, B.P., Navigation system for minimally invasive esophagectomy: experimental study in a porcine model. Surg Endosc. 27 (2013), 3663–3670, 10.1007/s00464-013-2941-4.
31. Nicolau, S.A., Pennec, X., Soler, L., Buy, X., Gangi, A., Ayache, N., Marescaux, J., An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med. Image Anal. 13 (2009), 494–506, 10.1016/j.media.2009.02.003.
32. Nicolau, S., Soler, L., Mutter, D., Marescaux, J., Augmented reality in laparoscopic surgical oncology. Surg. Oncol., 2011, 10.1016/j.suronc.2011.07.002.
33. Okamoto, T., Onda, S., Yanaga, K., Suzuki, N., Hattori, A., Clinical application of navigation surgery using augmented reality in the abdominal field. Surg. Today., 2015, 10.1007/s00595-014-0946-9.
34. Ramakrishnan, V.R., Orlandi, R.R., Citardi, M.J., Smith, T.L., Fried, M.P., Kingdom, T.T., The use of image-guided surgery in endoscopic sinus surgery: an evidence-based review with recommendations. Int. Forum Allergy Rhinol. 3 (2013), 236–241, 10.1002/alr.21094.
35. Schulze, F., Bühler, K., Neubauer, A., Kanitsar, A., Holton, L., Wolfsberger, S., Intra-operative virtual endoscopy for image guided endonasal transsphenoidal pituitary surgery. Int. J. Comput. Assist. Radiol. Surg. 5 (2010), 143–154, 10.1007/s11548-009-0397-8.
36. Shamir, R.R., Joskowicz, L., Shoshan, Y., Fiducial optimization for minimal target registration error in image-guided neurosurgery. IEEE Trans. Med. Imaging 31 (2012), 725–737, 10.1109/TMI.2011.2175939.
37. Shamir, R.R., Joskowicz, L., Spektor, S., Shoshan, Y., Target and trajectory clinical application accuracy in neuronavigation. Neurosurgery, 68, 2011, 10.1227/NEU.0b013e31820828d9.
38. Shamir, R.R., Joskowicz, L., Spektor, S., Shoshan, Y., Localization and registration accuracy in image guided neurosurgery: a clinical study. Int. J. Comput. Assist. Radiol. Surg. 4 (2009), 45–52, 10.1007/s11548-008-0268-8.
39. Simpfendörfer, T., Baumhauer, M., Müller, M., Gutt, C.N., Meinzer, H.-P., Rassweiler, J.J., Guven, S., Teber, D., Augmented reality visualization during laparoscopic radical prostatectomy. J. Endourol. 25 (2011), 1841–1845, 10.1089/end.2010.0724.
40. Souzaki, R., Ieiri, S., Uemura, M., Ohuchida, K., Tomikawa, M., Kinoshita, Y., Koga, Y., Suminoe, A., Kohashi, K., Oda, Y., Hara, T., Hashizume, M., Taguchi, T., An augmented reality navigation system for pediatric oncologic surgery based on preoperative CT and MRI images. J. Pediatric Surg., 2013, 2479–2483, 10.1016/j.jpedsurg.2013.08.025.
41. Sugimoto, M., Recent advances in visualization, imaging, and navigation in hepatobiliary and pancreatic sciences. J. Hepatobiliary. Pancreat. Sci. 17 (2010), 574–576, 10.1007/s00534-009-0192-5.
42. Sun, B., Liu, L., Hu, C., Meng, M.Q.H., Surgical instrument recognition and calibration for optical tracking system. 2010 IEEE International Conference on Robotics and Biomimetics, ROBIO 2010, 2010, 1376–1381, 10.1109/ROBIO.2010.5723530.
43. Tuceryan, M., Greer, D.S., Whitaker, R.T., Breen, D., Crampton, C., Rose, E., Ahlers, K.H., Calibration requirements and procedures for augmented reality. IEEE Trans. Vis. Comput. Graph. 1 (1995), 255–273, 10.1109/2945.466720 1, 3.
44. Uneri, A., Schafer, S., Mirota, D., Nithiananthan, S., Otake, Y., Reaungamornrat, S., Yoo, J., Stayman, J.W., Reh, D., Gallia, G.L., Khanna, A.J., Hager, G., Taylor, R.H., Kleinszig, G., Siewerdsen, J.H., Architecture of a high-performance surgical guidance system based on C-arm cone-beam CT: software platform for technical integration and clinical translation. Proc. SPIE, 2011, 10.1117/12.878191.
45. Volonté, F., Pugin, F., Bucher, P., Sugimoto, M., Ratib, O., Morel, P., Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J. Hepatobiliary. Pancreat. Sci. 18 (2011), 506–509, 10.1007/s00534-011-0385-6.
46. Wang, J., Suenaga, H., Hoshi, K., Yang, L., Kobayashi, E., Sakuma, I., Liao, H., Augmented reality navigation with automatic marker-free image registration using 3-d image overlay for dental surgery. IEEE Trans. Biomed. Eng. 61 (2014), 1295–1304, 10.1109/TBME.2014.2301191.
47. Wang, M.Y., Maurer, C.R., Fitzpatrick, J.M., Maciunas, R.J., An automatic technique for finding and localizing externally attached markers in CT and MR volume images of the head. IEEE Trans. Biomed. Eng. 43 (1996), 627–637, 10.1109/10.495282.
48. West, J., Fitzpatrick, J.M., Wang, M.Y., Dawant, B.M., Maurer, C.R., Kessler, R.M., Maciunas, R.J., Barillot, C., Lemoine, D., Collignon, A., Maes, F., Suetens, P., Vandermeulen, D., van den Elsen, P.A., Napel, S., Sumanaweera, T.S., Harkness, B., Hemler, P.F., Hill, D.L., Hawkes, D.J., Studholme, C., Maintz, J.B., Viergever, M.A., Malandain, G., Woods, R.P., Comparison and evaluation of retrospective intermodality image registration techniques. Int. Soc. Opt. Photonics, 1996, 332–347, 10.1097/00004728-199707000-00007.
49. West, J.B., Maurer, C.R., Designing optically tracked instruments for image-guided surgery. IEEE Trans. Med. Imaging 23 (2004), 533–545, 10.1109/TMI.2004.825614.
50. Zang, X., Yang, J., Weng, D., Wang, Y., Liu, Y., Augmented reality based surgery navigation system. International Conference on Optical Instrumentation and Technology, 2009, 10.1117/12.839658 p. 75132T–75132T–8.