Augmented Reality in Neurosurgery: A Review of Current Concepts and Emerging Applications

Canadian Journal of Neurological Sciences

Volumn 44, Issue 3, 2017, Pages 235-245

  Guha, Daipayan ^a   Alotaibi, Naif M ^a   Nguyen, Nhu ^b   Gupta, Shaurya ^c   McFaul, Christopher ^a   Yang, Victor X D ^a,c,d  

^a UNIVERSITY OF TORONTO   (Canada)

^b RYERSON UNIVERSITY   (Canada)

^c UNIVERSITY OF TORONTO   (Canada)

^d Sunnybrook Health Sciences Centre   (Canada)

Author keywords

augmented reality; image guidance; neuronavigation; virtual reality

Indexed keywords

AUGMENTED REALITY; CANCER SURGERY; DEPTH PERCEPTION; FUTUROLOGY; HISTORY OF MEDICINE; HUMAN; MEASUREMENT ERROR; MEDICAL LITERATURE; MEDLINE; NEUROIMAGING; NEURONAVIGATION; NEUROSURGERY; OPERATING ROOM; PRIORITY JOURNAL; REVIEW; ROBOT ASSISTED SURGERY; SCOPUS; SKULL SURGERY; SPINE SURGERY; STEREOTAXIC SURGERY; SURGICAL TECHNOLOGY; SYSTEMATIC REVIEW; TOUCH; VIRTUAL REALITY; VISION; WEB OF SCIENCE; BRAIN DISEASE; COMPUTER ASSISTED SURGERY; COMPUTER INTERFACE; DIAGNOSTIC IMAGING; IMAGE PROCESSING; PROCEDURES; SPINE DISEASE; TRENDS;

BRAIN DISEASES; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; NEUROSURGICAL PROCEDURES; SPINAL DISEASES; SURGERY, COMPUTER-ASSISTED; USER-COMPUTER INTERFACE; VIRTUAL REALITY;

EID: 85019229177     PISSN: 03171671     EISSN: None     Source Type: Journal    
DOI: 10.1017/cjn.2016.443     Document Type: Review

References (69)

1. Tang SL, Kwoh CK, Teo MY, Sing NW, Ling K V. Augmented reality systems for medical applications. IEEE Eng Med Biol Mag. 1998;17:49-58.
2. Shuhaiber JH. Augmented reality in surgery. Arch Surg. 2004;139:170-4.
3. Alaraj A, Charbel FT, Birk D, et al. Role of cranial and spinal virtual and augmented reality simulation using immersive touch modules in neurosurgical training. Neurosurgery. 2013;72(Suppl 1): 115-23.
4. Sielhorst T, Feuerstein M, Navab N. Advanced medical displays: a literature review of augmented reality. Disp Technol J. 2008;4:451-67.
5. Sutherland IE. A head-mounted three dimensional display. Proc AFIPS Fall Jt Comput Conf. 1968:757-64.
6. Roberts DW, Strohbehn JW, Hatch JF, Murray W, Kettenberger H. A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. J Neurosurg. 1986;65:545-9.
7. Rankin TM, Slepian MJ, Armstrong DG. Augmented reality in surgery. Technological Advances in Surgery, and Trauma Critical Care. New York: Springer New York; 2015, p. 59-71.
8. Kersten-Oertel M, Jannin P, Collins DL. The state of the art of visualization in mixed reality image guided surgery. Comput Med Imaging Graph. 2013;37:98-112.
9. Tagaytayan R, Kelemen A, Sik-Lanyi C. Augmented reality in neurosurgery. Arch Med Sci. 2016:1-7.
10. Bastien S, Peuchot B, Tanguy A. Augmented reality in spine surgery: critical appraisal and status of development. Stud Health Technol Inform. 2002;88:153-6.
11. Lerotic M, Chung AJ, Mylonas G, Yang G-Z. Pq-space based non-photorealistic rendering for augmented reality. Med Image Comput Comput Assist Interv. 2007;10:102-9.
12. Tuceryan M, Greer DS, Whitaker RT, et al. Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System. Trans Vis Comput Graph. 1995;1:255-73.
13. Cabrilo I, Schaller K, Bijlenga P. Augmented reality-assisted bypass surgery: embracing minimal invasiveness. World Neurosurg. 2015;83:596-602.
14. Grimson WL, Ettinger GJ, White SJ, Lozano-Perez T, Wells WM, Kikinis R. An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization. IEEE Trans Med Imaging. 1996;15:129-40.
15. Besharati Tabrizi L, Mahvash M. Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg. 2015;123:206-11.
16. Edwards PJ, Hawkes DJ, Hill DLG, et al. Augmentation of reality using an operating microscope for otolaryngology and neurosurgical guidance. J Image Guid Surg. 1995;1:172-8.
17. Gleason PL, Kikinis R, Altobelli D, et al. Video registration virtual reality for nonlinkage stereotactic surgery. Stereotact Funct Neurosurg. 1994;63:139-43.
18. Gildenberg PL, Ledoux R, Cosman E, Labuz J. The exoscope-a frame-based video/graphics system for intraoperative guidance of surgical resection. Stereotact Funct Neurosurg. 1994;63:23-5.
19. Masutani Y, Dohi T, Yamane F, Iseki H, Takakura K. Augmented reality visualization system for intravascular neurosurgery. Comput Aided Surg. 1998;3:239-47.
20. Kawamata T, Iseki H, Shibasaki T, Hori T. Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery. 2002;50:1393-7.
21. Paul P, Fleig O, Jannin P. Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging. 2005;24:1500-11.
22. Pandya A, Siadat MR, Auner G. Design, implementation and accuracy of a prototype for medical augmented reality. Comput Aided Surg. 2005;10:23-35.
23. Gildenberg PL, Labuz J. Use of a volumetric target for image-guided surgery. Neurosurgery. 2006;59:651-9.
24. Lovo EE, Quintana JC, Puebla MC, et al. A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery. 2007;60:362-6.
25. Kockro RA, Tsai YT, Ng I, et al. Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery. 2009;65:795-8.
26. Low D, Lee CK, Dip LLT, Ng WH, Ang BT, Ng I. Augmented reality neurosurgical planning and navigation for surgical excision of parasagittal, falcine and convexity meningiomas. Br J Neurosurg. 2010;24:69-74.
27. Bisson M, Cheriet F, Parent S. 3D visualization tool for minimally invasive discectomy assistance. Stud Health Technol Inform. 2010;158:55-60.
28. Navab N, Heining SM, Traub J. Camera augmented mobile C-arm (CAMC): calibration, accuracy study, and clinical applications. IEEE Trans Med Imaging. 2010;29:1412-23.
29. Luciano CJ, Banerjee PP, Bellotte B, et al. Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery. 2011;69:ons14-9; discussion ons19.
30. Wang A, Mirsattari SM, Parrent AG, Peters TM. Fusion and visualization of intraoperative cortical images with preoperative models for epilepsy surgical planning and guidance. Comput Aided Surg. 2011;16:149-60.
31. Shenai MB, Dillavou M, Shum C, et al. Virtual interactive presence and augmented reality (VIPAR) for remote surgical assistance. Neurosurgery. 2011;68:200-7.
32. Weiss CR, Marker DR, Fischer GS, Fichtinger G, Machado AJ, Carrino JA. Augmented reality visualization using image-overlay for MR-guided interventions: system description, feasibility, and initial evaluation in a spine phantom. AJR Am J Roentgenol. 2011;196:W305-7.
33. Azimi E, Doswell J, Kazanzides P. Augmented reality goggles with an integrated tracking system for navigation in neurosurgery. IEEE Virtual Real Conf 2012 Proc. 2012:123-4.
34. Chang YZ, Hou JF, Tsao YH, Lee ST. Application of real-time single camera SLAM technology for image-guided targeting in neurosurgery. In: Tescher AG, editor. Applied Digital Image Processing, Vol. 8499, Bellingham: Spie-Int Soc Optical Engineering; 2012.
35. Abe Y, Sato S, Kato K, et al. A novel 3D guidance system using augmented reality for percutaneous vertebroplasty: technical note. J Neurosurg Spine. 2013;19:492-501.
36. Mahvash M, Besharati Tabrizi L. A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir. 2013;155:943-7.
37. Inoue D, Cho B, Mori M, et al. Preliminary study on the clinical application of augmented reality neuronavigation. J Neurol Surg. 2013;74:71-6.
38. Yudkowsky R, Luciano C, Banerjee P, et al. Practice on an augmented reality/haptic simulator and library of virtual brains improves residents' ability to perform a ventriculostomy. Simul Heal. 2013;8:25-31.
39. Hooten KG, Lister JR, Lombard G, Lizdas DE, Lampotang S, Rajon DA, et al. Mixed reality ventriculostomy simulation: experience in neurosurgical residency. Neurosurgery. 2014;10(Suppl 4): 576-81.
40. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir. 2014;156:1769-74.
41. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery. 2014; 10(Suppl 2):251-2.
42. Deng W, Li F, Wang M, Song Z. Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg. 2014;92:17-24.
43. Kersten-Oertel M, Gerard I, Drouin S, et al. Augmented reality in neurovascular surgery: first experiences. In: Linte CA, Yaniv Z, Fallavollita P, Abolmaesumi P, Holmes DR, editors. Augmented Environments for Computer Assisted Interventions, Vol. 8678, Berlin: Springer-Verlag, Berlin, 2014, p. 80-9.
44. Fritz J, U-Thainual P, Ungi T, et al. MR-guided vertebroplasty with augmented reality image overlay navigation. Cardiovasc Intervent Radiol. 2014;37:1589-96.
45. Wu JR, Wang ML, Liu KC, Hu MH, Lee PY. Real-time advanced spinal surgery via visible patient model and augmented reality system. Comput Methods Programs Biomed. 2014;113:869-81.
46. Watson JR, Martirosyan N, Skoch J, Lemole GM, Anton R, Romanowski M. Augmented microscopy with near-infrared fluorescence detection. In: Pogue BW, Gioux S, editors. Proc. SPIE, Molecular-Guided Surgery: Molecules, Devices, and Applications, Vol. 9311, Bellingham: Spie-Int Soc Optical Engineering; 2015.
47. Kersten-Oertel M, Gerard I, Drouin S, et al. Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. Int J Comput Assist Radiol Surg. 2015;10: 1823-36.
48. Abhari K, Baxter JSH, Chen ECS, et al. Training for planning tumour resection: augmented reality and human factors. IEEE Trans Biomed Eng. 2015;62:1466-77.
49. Watanabe E, Satoh M, Konno T, Hirai M, Yamaguchi T. The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg. 2016;87:399-405.
50. Eftekhar B. A smartphone app to assist scalp localization of superficial supratentorial lesions-technical note. World Neurosurg. 2016;85:359-63.
51. Hill DL, Maurer CR, Maciunas RJ, Barwise JA, Fitzpatrick JM, Wang MY. Measurement of intraoperative brain surface deformation under a craniotomy. Neurosurgery. 1998;43:514-26-8.
52. Reinertsen I, Lindseth F, Askeland C, Iversen DH, Unsgård G. Intra-operative correction of brain-shift. Acta Neurochir (Wien). 2014;156:1301-10.
53. Gerard IJ, Kersten-Oertel M, Drouin S, et al. Improving patient specific neurosurgical models with intraoperative ultrasound and augmented reality visualizations in a neuronavigation environment. In: Oyarzun Laura C, Shekhar R, Wesarg S, et al. editors Clinical Image-Based Procedures. Translational Research in Medical Imaging, Vol. 9401. Lecture Notes in Computer Science; 2016, p. 28-35.
54. Shin BJ, James AR, Njoku IU, Hartl R, Härtl R. Pedicle screw navigation: a systematic review and meta-analysis of perforation risk for computer-navigated versus freehand insertion. J Neurosurg Spine. 2012;17:113-22.
55. Tuceryan M, Genc Y, Navab N. Single-point active alignment method (SPAAM) for optical see-through HMD calibration for augmented reality. Presence Teleoperators Virtual Environ. 2002;11:259-76.
56. Watt SJ, Akeley K, Ernst MO, Banks MS. Focus cues affect perceived depth. J Vis. 2005;5:834-62.
57. Bando T, Iijima A, Yano S. Visual fatigue caused by stereoscopic images and the search for the requirement to prevent them: a review. Displays. 2012;33:76-83.
58. Hu X, Hua H. An optical see-through multi-focal-plane stereoscopic display prototype enabling nearly correct focus cues. Proc SPIE. 2013;8648:86481A-6.
59. Nagata S. How to reinforce perception of depth in single twodimensional pictures. Proc SID. 1983;25:239-46.
60. Kersten-Oertel M, Chen SS, Drouin S, Sinclair DS, Collins DL. Augmented reality visualization for guidance in neurovascular surgery. Stud Heal Technol Informatics. 2012;173:225-9.
61. Ware C, Balakrishnan R. Reaching for objects in VR displays: lag and frame rate. ACM Trans Comput Interact. 1994;1: 331-56.
62. Hughes-Hallett A, Mayer EK, et al. Inattention blindness in surgery. Surg Endosc. 2015;29:3184-9.
63. Simons DJ, Chabris CF. Gorillas in our midst: sustained inattentional blindness for dynamic events. Perception. 1999;28:1059-74.
64. Dixon BJ, Daly MJ, Chan HH, Vescan A, Witterick IJ, Irish JC. Inattentional blindness increased with augmented reality surgical navigation. Am J Rhinol Allergy. 2014;28:433-7.
65. Dixon BJ, Daly MJ, Chan H, Vescan AD, Witterick IJ, Irish JC. Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc. 2013;27: 454-61.
66. Marcus HJ, Pratt P, Hughes-Hallett A, et al. Comparative effectiveness and safety of image guidance systems in surgery: a preclinical randomised study. Lancet. 2015;385:S64.
67. Davis MC, Can DD, Pindrik J, Rocque BG, Johnston JM. Virtual interactive presence in global surgical education: international collaboration through augmented reality. World Neurosurg. 2016;86:103-11.
68. Arshad H, Chowdhury SA, Chun LM, Parhizkar B, Obeidy WK. A freeze-object interaction technique for handheld augmented reality systems. Multimed Tools Appl. 2016;75:5819-39.
69. Tang JKT, Tewell J. Emerging human-toy interaction techniques with augmented and mixed reality. New York: Springer International Publishing; 2015, p. 77-105.