Augmented reality navigation with automatic marker-free image registration using 3-d image overlay for dental surgery

IEEE Transactions on Biomedical Engineering

Volumn 61, Issue 4, 2014, Pages 1295-1304

  Wang, Junchen ^a   Suenaga, Hideyuki ^b   Hoshi, Kazuto ^c   Yang, Liangjing ^a   Kobayashi, Etsuko ^a   Sakuma, Ichiro ^a   Liao, Hongen ^a,d  

^a UNIVERSITY OF TOKYO   (Japan)

^b UNIVERSITY OF TOKYO HOSPITAL   (Japan)

^c UNIVERSITY OF TOKYO   (Japan)

^d Tsinghua University   (China)

Author keywords

3 D image overlay; Augmented reality (AR); dental surgery; integral photography (IP); stereo tracking; surgical navigation

Indexed keywords

AUGMENTED REALITY; CAMERAS; DENTISTRY; DEPTH PERCEPTION; DISPLAY DEVICES; GEOMETRICAL OPTICS; IMAGE REGISTRATION; NAVIGATION SYSTEMS; PROGRAM PROCESSORS; SURGERY; SURGICAL EQUIPMENT;

3-D IMAGE; DENTAL SURGERY; INTEGRAL PHOTOGRAPHY; STEREO TRACKING; SURGICAL NAVIGATION;

THREE DIMENSIONAL;

ACCURACY; ALGORITHM; ARTICLE; AUTOMATION; CALIBRATION; CAMERA; DENTAL SURGERY; DEPTH PERCEPTION; HUMAN; MANDIBLE; MOLAR TOOTH; MOTION; PHANTOM; SURGICAL NAVIGATION SYSTEM; THICKNESS; THREE DIMENSIONAL IMAGING; TOOTH;

DENTISTRY, OPERATIVE; HEAD; HUMANS; IMAGING, THREE-DIMENSIONAL; MODELS, BIOLOGICAL; PHANTOMS, IMAGING; PHOTOGRAPHY, DENTAL; USER-COMPUTER INTERFACE;

EID: 84897129185     PISSN: 00189294     EISSN: 15582531     Source Type: Journal    
DOI: 10.1109/TBME.2014.2301191     Document Type: Article

References (35)

1. P. Pohlenz, A. Gröbe, A. Petersik, N. von Sternberg, B. Pflesser, A. Pommert, K.-H. Hhne, U. Tiede, I. Springer, and M. Heiland, "Virtual dental surgery as a new educational tool in dental school, " J. Cranio Maxill. Surg., vol. 38, no. 8, pp. 560-564, 2010.
2. D. Wang, Y. Zhang, Y. Wang, Y.-S. Lee, P. Lu, and Y. Wang, "Cutting on triangle mesh: Local model-based haptic display for dental preparation surgery simulation, " IEEE Trans. Vis. Comput. Graph., vol. 11, no. 6, pp. 671-683, Nov./Dec. 2005.
3. N. Casap, A. Wexler, N. Persky, A. Schneider, and J. Lustmann, "Navigation surgery for dental implants: Assessment of accuracy of the image guided implantology system, " J. Oral Maxillofac. Surg., vol. 62, pp. 116-119, 2004. (Pubitemid 39485814)
4. N. Casap, S. Nadel, E. Tarazi, and E. I. Weiss, "Evaluation of a navigation system for dental implantation as a tool to train novice dental practitioners, " J. Oral Maxillofac. Surg., vol. 69, no. 10, pp. 2548-2556, 2011.
5. H. Yu, S. G. Shen, X. Wang, L. Zhang, and S. Zhang, "The indication and application of computer-assisted navigation in oral and maxillofacial surgery Shanghai's experience based on 104 cases, " J. Cranio Maxill. Surg., vol. 41, no. 8, pp. 770-774, 2013.
6. C. Bouchard, J. Magill, V. Nikonovskiy, M. Byl, B. Murphy, L. Kaban, and M. Troulis, "Osteomark: A surgical navigation system for oral and max-illofacial surgery, " Int. J. Oral Maxillofac. Surg., vol. 41, no. 2, pp. 265-270, 2012.
7. M. Tsuji, N. Noguchi, M. Shigematsu, Y. Yamashita, K. Ihara, M. Shikimori, and M. Goto, "A new navigation system based on cephalo-grams and dental casts for oral and maxillofacial surgery, " Int. J. Oral Max. Surg., vol. 35, no. 9, pp. 828-836, 2006. (Pubitemid 44070648)
8. S. Yamaguchi, T. Ohtani, H. Yatani, and T. Sohmura, "Augmented reality system for dental implant surgery, " in Virtual and Mixed Reality, (Lecture Notes in Computer Science series), R. Shumaker, Ed. vol. 5622Berlin, Germany: Springer-Verlag, 2009, pp. 633-638.
9. H.-T. Luebbers, P. Messmer, J. A. Obwegeser, R. A. Zwahlen, R. Kikinis, K. W. Graetz, and F. Matthews, "Comparison of different registration methods for surgical navigation in cranio-maxillofacial surgery, " J. Cranio Maxill. Surg., vol. 36, no. 2, pp. 109-116, 2008.
10. J. Maintz and M. A. Viergever, "A survey of medical image registration, " Med. Image Anal., vol. 2, no. 1, pp. 1-36, 1998. (Pubitemid 128500719)
11. D. Venosta, Y. Sun, F. Matthews, A. L. Kruse, M. Lanzer, T. Gander, K. W. Grtz, and H.-T. Lbbers, "Evaluation of two dental registration-splint techniques for surgical navigation in cranio-maxillofacial surgery, " J. Cranio Maxill. Surg., (2013, Jul. 5). [Online].
12. N. Dodgson, "Autostereoscopic 3-D displays, " Computer, vol. 38, no. 8, pp. 31-36, 2005.
13. W. Birkfellner, M. Figl, K. Huber, F. Watzinger, F. Wanschitz, J. Hummel, R. Hanel, W. Greimel, P. Homolka, R. Ewers, and H. Bergmann, "A head-mounted operating binocular for augmented reality visualization in medicine-Design and initial evaluation, " IEEE Trans. Med. Imag., vol. 21, no. 8, pp. 991-997, Aug. 2002. (Pubitemid 35476602)
14. M. Figl, C. Ede, J. Hummel, F. Wanschitz, R. Ewers, H. Bergmann, and W. Birkfellner, "A fully automated calibration method for an optical see-through head-mounted operating microscope with variable zoom and focus, " IEEE Trans. Med. Imag., vol. 24, no. 11, pp. 1492-1499, Nov. 2005. (Pubitemid 41619225)
15. V. Ferrari, G. Megali, E. Troia, A. Pietrabissa, and F. Mosca, "A 3-D mixed-reality system for stereoscopic visualization of medical dataset, " IEEE Trans. Biomed. Eng., vol. 56, no. 11, pp. 2627-2633, Nov. 2009.
16. R. Wen, C.-K. Chui, S.-H. Ong, K.-B. Lim, and S.-K. Chang, "Projection-based visual guidance for robot-aided RF needle insertion, " Int. J. Comput. Assist. Radiol. Surg., vol. 8, pp. 1015-1025, 2013.
17. A. Stern and B. Javidi, "Three-dimensional image sensing, visualization, and processing using integral imaging, " Proc. IEEE, vol. 94, no. 3, pp. 591-607, Mar. 2006.
18. J.-H. Park, K. Hong, and B. Lee, "Recent progress in three-dimensional information processing based on integral imaging, " Appl. Opt., vol. 48, no. 34, pp. H77-H94, Dec. 2009.
19. H. Liao, T. Inomata, I. Sakuma, and T. Dohi, "3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay, " IEEE Trans. Biomed. Eng., vol. 57, no. 6, pp. 1476-1486, Jun. 2010.
20. H. Liao, N. Hata, S. Nakajima, M. Iwahara, I. Sakuma, and T. Dohi, "Surgical navigation by autostereoscopic image overlay of integral videogra-phy, " IEEE Trans. Inf. Technol. Biomed., vol. 8, no. 2, pp. 114-121, Jun. 2004.
21. H. Liao, H. Ishihara, H. H. Tran, K. Masamune, I. Sakuma, and T. Dohi, "Precision-guided surgical navigation system using laser guidance and 3D autostereoscopic image overlay, " Comput. Med. Imag. Graph., vol. 34, no. 1, pp. 46-54, 2010.
22. H. Tran, H. Suenaga, K. Kuwana, K. Masamune, T. Dohi, S. Nakajima, and H. Liao, "Augmented reality system for oral surgery using 3D auto stereoscopic visualization, " in Proc. Med. Image Comput Comput.-Assist. Intervention, 2011, vol. 6891, pp. 81-88.
23. J. Wang, H. Suenaga, L. Yang, H. Liao, E. Kobayashi, T. Takato, and I. Sakuma, "Real-time marker-free patient registration and image-based navigation using stereovision for dental surgery, " in Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions, (Lecture Notes in Computer Science Series). vol. 8090, Berlin, Germany: Springer-Verlag, 2013, pp. 9-18.
24. G. Lippmann, "Epreuves reversible donnant la sensation du relief, " J. Phys. Theor. Appl., vol. 7, no. 1, pp. 821-825, 1908.
25. H. Liao, T. Dohi, and K. Nomura, "Autostereoscopic 3-D display with long visualization depth using referential viewing area-based integral photography, " IEEE Trans. Vis. Comput. Graph., vol. 17, no. 11, pp. 1690-1701, Nov. 2011.
26. J. Wang, I. Sakuma, and H. Liao, "A hybrid flexible rendering pipeline for real-time 3D medical imaging using GPU-accelerated integral videogra-phy, " Int. J. Comput. Assist. Radiol. Surg., vol. 8, pp. S287-S288, 2013.
27. Z. Zhang, "A flexible new technique for camera calibration, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 11, pp. 1330-1334, Nov. 2000.
28. A. Fusiello, E. Trucco, and A. Verri, "A compact algorithm for rectification of stereo pairs, " Mach. Vision Appl., vol. 12, no. 1, pp. 16-22, 2000.
29. B. K. P. Horn, H. M. Hilden, and S. Negahdaripour, "Closed-form solution of absolute orientation using orthonormal matrices, " J. Opt. Soc. Amer. A, vol. 5, no. 7, pp. 1127-1135, Jul. 1988.
30. G. Welch and G. Bishop. (1995). "An introduction to the Kalman filter, " [Online]. Available: http://clubs.ens-cachan.fr/krobot/old/data/ positionnement/kalman.pdf
31. K. Briechle and U. D. Hanebeck, "Template matching using fast normalized cross correlation, " Proc. SPIE, Opt. Pattern Recog. XII, vol. 4387, pp. 95-102, 2001. (Pubitemid 32638472)
32. S. Rusinkiewicz and M. Levoy, "Efficient variants of the ICP algorithm, " in Proc. 3rd Int. Conf. 3-D Digital Imag. Model., 2001, pp. 145-152.
33. J. Fitzpatrick and J. West, "The distribution of target registration error in rigid-body point-based registration, " IEEE Trans. Med. Imag., vol. 20, no. 9, pp. 917-927, Sep. 2001. (Pubitemid 32992594)
34. R. Destrez, S. Treuillet, Y. Lucas, and B. Albouy-Kissi, "Semi-automatic registration of 3D orthodontics models from photographs, " Proc. SPIE, Med. Imag., vol. 8669, pp. 86 691E1-86 691E12, 2013.
35. J. Fitzpatrick, J. West, and C. R. Maurer, Jr., "Predicting error in rigid-body point-based registration, " IEEE Trans. Med. Imag., vol. 17, no. 5, pp. 694-702, Oct. 1998. (Pubitemid 128747725)