Video see-through augmented reality for oral and maxillofacial surgery

International Journal of Medical Robotics and Computer Assisted Surgery

Volumn 13, Issue 2, 2017, Pages

  Wang, Junchen ^a,b   Suenaga, Hideyuki ^c   Yang, Liangjing ^b   Kobayashi, Etsuko ^b   Sakuma, Ichiro ^b  

^a BEIHANG UNIVERSITY   (China)

^b UNIVERSITY OF TOKYO   (Japan)

^c UNIVERSITY OF TOKYO HOSPITAL   (Japan)

Author keywords

3D 2D image registration; augmented reality; object tracking; oral and maxillofacial surgery

Indexed keywords

CAMERAS; COMPUTERIZED TOMOGRAPHY; COPYRIGHTS; IMAGE REGISTRATION; SURGERY; TRACKING (POSITION); VIRTUAL REALITY;

3D-2D IMAGE REGISTRATIONS; IMAGE GUIDANCES; IMAGES REGISTRATION; MARKERLESS; MATCHING METHODS; OBJECT TRACKING; ORAL AND MAXILLOFACIAL SURGERIES; REAL- TIME; REGISTRATION METHODS; SHAPE MATCHING;

AUGMENTED REALITY;

ARTICLE; AUGMENTED REALITY; CAMERA; COMPUTER ASSISTED TOMOGRAPHY; CONTROLLED STUDY; FEASIBILITY STUDY; IMAGING PHANTOM; INTRAOPERATIVE PERIOD; MAXILLOFACIAL SURGERY; MEASUREMENT ACCURACY; MEASUREMENT PRECISION; ORAL SURGERY; VALIDATION PROCESS; VIDEORECORDING; VIRTUAL REALITY; COMPUTER ASSISTED SURGERY; COMPUTER INTERFACE; DIAGNOSTIC IMAGING; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; JAW; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; SURGERY; X-RAY COMPUTED TOMOGRAPHY;

HUMANS; JAW; ORAL SURGICAL PROCEDURES; PHANTOMS, IMAGING; RADIOGRAPHIC IMAGE ENHANCEMENT; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE; SURGERY, COMPUTER-ASSISTED; TOMOGRAPHY, X-RAY COMPUTED; USER-COMPUTER INTERFACE; VIDEO RECORDING;

EID: 84978252306     PISSN: 14785951     EISSN: 1478596X     Source Type: Journal    
DOI: 10.1002/rcs.1754     Document Type: Article

References (36)

1. Azagury DE, Dua MM, Barrese JC, et al. Image-guided surgery. Curr Probl Surg 2015; 52(12): 476–520.
2. Maintz JBA, Viergever MA. A survey of medical image registration. Med Image Anal 1998; 2(1): 1–36.
3. Bouchard C, Magill JC, Nikonovskiy V, et al. Osteomark: a surgical navigation system for oral and maxillofacial surgery. Int J Oral Max Surg 2012; 41(2): 265–270.
4. Casap N, Nadel S, Tarazi E, Weiss EI. Evaluation of a navigation system for dental implantation as a tool to train novice dental practitioners. J Oral Maxil Surg 2011; 69(10): 2548–2556.
5. Venosta D, Sun Y, Matthews F, et al. Evaluation of two dental registration-splint techniques for surgical navigation in cranio-maxillofacial surgery. J Cranio Maxill Surg 2014; 42(5): 448–453.
6. Yu HB, Shen SG, Wang XD, et al. The indication and application of computer-assisted navigation in oral and maxillofacial surgery – Shanghai's experience based on 104 cases. J Cranio Maxill Surg 2013; 41(8): 770–774.
7. Badiali G, Ferrari V, Cutolo F, et al. Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning. J Cranio Maxill Surg 2014; 42(8): 1970–1976.
8. Cutolo F, Badiali G, Ferrari V. Human-PnP: ergonomic AR interaction paradigm for manual placement of rigid bodies. In Augmented Environments for Computer-Assisted Interventions (AE-CAI), Linte CA, Yaniv Z, Fallavollita P (eds.). Springer International Publishing: Cham, Switzerland, 2015; 50–60.
9. Navab N, Heining SM, Traub J. Camera augmented mobile C-arm (CAMC): calibration, accuracy study, and clinical applications. IEEE Trans Med Imag 2010; 29(7): 1412–1423.
10. Wang J, Suenaga H, Yang L, et al. Real-time marker-free patient registration and image-based navigation using stereovision for dental surgery. In Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, Liao H, Linte CA, Masamune K, Peters T, Zheng G (eds.). Springer Berlin Heidelberg: Heidelberg, 2013; 9–18.
11. Wang J, Suenaga H, Hoshi K, et al. Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 2014; 61(4): 1295–1304.
12. Wang J, Suenaga H, Liao H, et al. Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput Med Imag Grap 2015; 40: 147–159.
13. Ulrich M, Wiedemann C, Steger C. Combining scale-space and similarity-based aspect graphs for fast 3D object recognition. IEEE Trans Pattern Anal 2012; 34(10): 1902–1914.
14. Kalal Z, Mikolajczyk K, Matas J. Tracking-learning-detection. IEEE Trans Pattern Anal 2012; 34(7): 1409–1422.
15. Wang J, Suenaga H, Yang L, et al. 3D surgical overlay with markerless image registration using a single camera. In Augmented Environments for Computer-Assisted Interventions (AE-CAI), Linte CA, Yaniv Z, Fallavollita P (eds.). Springer International Publishing: Cham, Switzerland, 2015; 124–133.
16. Su LM, Vagvoigyi BP, Agarwal R, et al. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 2009; 73(4): 896–900.
17. Otake Y, Leonard S, Reiter A, et al. Rendering-based video-CT registration with physical constraints for image-guided endoscopic sinus surgery. Proc SPIE Int Soc Opt Eng 2015; 9415: 94150A-1–94150A-6.
18. Mirota DJ, Hanzi W, Taylor RH, et al. A system for video-based navigation for endoscopic endonasal skull base surgery. IEEE Trans Med Imag 2012; 31(4): 963–976.
19. Mirota DJ, Uneri A, Schafer S, et al. Evaluation of a system for high-accuracy 3D image-based registration of endoscopic video to C-arm cone-beam CT for image-guided skull base surgery. IEEE Trans Med Imag 2013; 32(7): 1215–1226.
20. Bernhardt S, Nicolauc SA, Agnus V, et al. Automatic localization of endoscope in intraoperative CT image: a simple approach to augmented reality guidance in laparoscopic surgery. Med Image Anal 2016; 30: 130–143.
21. Helferty JP, Sherbondy AJ, Kiraly AP, et al. Computer-based system for the virtual-endoscopic guidance of bronchoscopy. Comput Vis Image Und 2007; 108: 171–187.
22. Merritt SA, Khare R, Bascom R, Higgins WE. Interactive CT-video registration for the continuous guidance of bronchoscopy. IEEE Trans Med Imag 2013; 32(8): 1376–1396.
23. Deguchi D, Mori K, Feuerstein M, et al. Selective image similarity measure for bronchoscope tracking based on image registration. Med Image Anal 2009; 13(4): 621–633.
24. Luo X, Mori K. A discriminative structural similarity measure and its application to video-volume registration for endoscope three-dimensional motion tracking. IEEE Trans Med Imag 2014; 33(6): 1248–1261.
25. Gold S, Rangarajan A, Lu C-P, et al. New algorithms for 2D and 3D point matching: pose estimation and correspondence. Pattern Recog 1998; 31(8): 1019–1031.
26. Rosenhahn B, Perwass C, Sommer G. Pose estimation of 3D free-form contours. Int J Comput Vision 2005; 62(3): 267–289.
27. Steger C. Occlusion, clutter, and illumination invariant object recognition. Int Archives Photogramm Remote Sens 2002; 34: 345–350.
28. Cyr CM, Kimia BB. A similarity-based aspect-graph approach to 3D object recognition. Int J Comput Vision 2004; 57: 5–22.
29. Wang J, Kobayashi E, Sakuma I. Coarse-to-fine dot array marker detection with accurate edge localization for stereo visual tracking. Biomed Signal Process 2015; 15: 49–59.
30. Di Zenzo S. A note on the gradient of a multi-image. Comput Vision Graphics Image Process 1986; 33(1): 116–125.
31. Wang J, Ohya T, Liao H, et al. Intravascular catheter navigation using path planning and virtual visual feedback for oral cancer treatment. Int J Med Robot Comp 2011; 7(2): 214–224.
32. Sielhorst T, Bichlmeier C, Heining SM, Navab N. Depth perception - a major issue in medical AR: evaluation study by twenty surgeons. In Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer Berlin/ Heidelberg: Heidelberg, 2006; 364–372.
33. Kersten-Oertel M, Jannin P, Collins DL. The state of the art of visualization in mixed reality image guided surgery. Comput Med Imag Grap 2013; 37: 98–112.
34. Bichlmeier C, Heining SM, Feuerstein M, Navab N. The vrtual mirror: a new interaction paradigm for augmented reality environments. IEEE Trans Med Imag 2009; 28(9): 1498–1510.
35. Bichlmeier C, Euler E, Blum T. Navab N. Evaluation of the virtual mirror as a navigational aid for augmented reality driven minimally invasive procedures. In 9th IEEE International Symposium on Mixed and Augmented Reality. IEEE: Seoul, 2010; 91–97.
36. Bichlmeier C, Wimmer F, Heining SM. Navab N. Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. In 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE: Nara, 2007; 129–138.