On mixed reality environments for minimally invasive therapy guidance: Systems architecture, successes and challenges in their implementation from laboratory to clinic

Computerized Medical Imaging and Graphics

Volumn 37, Issue 2, 2013, Pages 83-97

  Linte, Cristian A ^a   Davenport, Katherine P ^b   Cleary, Kevin ^b   Peters, Craig ^b   Vosburgh, Kirby G ^c   Navab, Nassir ^d   Edwards, Philip Eddie ^e   Jannin, Pierre ^f   Peters, Terry M ^g   Holmes, David R ^a   Robb, Richard A ^a  

^a MAYO CLINIC   (United States)

^b CHILDREN'S NATIONAL MEDICAL CENTER   (United States)

^c HARVARD UNIVERSITY   (United States)

^d TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^e IMPERIAL COLLEGE LONDON   (United Kingdom)

^f UNIVERSITÉ DE RENNES 1   (France)

^g ROBARTS RESEARCH INSTITUTE   (Canada)

Author keywords

Augmented and mixed reality environments; Clinical translation; Image guided interventions; Minimally invasive surgery and therapy; Virtual

Indexed keywords

CLINICAL TRANSLATION; IMAGE-GUIDED INTERVENTION; MINIMALLY INVASIVE SURGERY AND THERAPIES; MIXED-REALITY ENVIRONMENT; VIRTUAL;

MEDICAL APPLICATIONS; SURGICAL EQUIPMENT;

VIRTUAL REALITY;

ARTICLE; AUTOMATION; CLINICAL EVALUATION; CLINICAL PRACTICE; COMPUTER ASSISTED TOMOGRAPHY; COMPUTER GRAPHICS; COMPUTER SYSTEM; DIAGNOSTIC IMAGING; FLUOROSCOPY; HUMAN; INTERVENTIONAL MAGNETIC RESONANCE IMAGING; MEDICAL TECHNOLOGY; MINIMALLY INVASIVE PROCEDURE; MITRAL VALVE REPAIR; OPERATING ROOM; PRIORITY JOURNAL; RADIATION EXPOSURE; THREE DIMENSIONAL IMAGING; ULTRASOUND TRANSDUCER;

COMPUTER GRAPHICS; ENVIRONMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; SOFTWARE; SURGERY, COMPUTER-ASSISTED; SURGICAL PROCEDURES, MINIMALLY INVASIVE; TRANSLATIONAL MEDICAL RESEARCH; USER-COMPUTER INTERFACE;

EID: 84876998702     PISSN: 08956111     EISSN: 18790771     Source Type: Journal    
DOI: 10.1016/j.compmedimag.2012.12.002     Document Type: Article

References (186)

1. Milgram P., Takemura H., Utsumi A., Kishino F. Augmented reality: a class of displays on the reality-virtuality continuum. Proc SPIE 1994: Telemanipulator and Telepresence Technology, vol. 2351 1994, 282-292.
2. Milgram P., Kishino F. A taxonomy of mixed reality visual displays. IEICE Trans Inform Syst 1994, ED-77:1-15.
3. Metzger P.J. Adding reality to the virtual. Proc IEEE Virtual Reality Int Symp 1993, 7-13.
4. Takemura H., Kishino F. Cooperative work environment using virtual workspace. Proc Comput Support Cooper Work 1992, 226-232.
5. Kaneko M., Kishino F., Shimamura K., Harashima H. Toward the new era of visual communication. IEICE Trans Commun 1993, E76-B(6):577-591.
6. Utsumi A., Milgram P., Takemura H., Kishino F. Investigation of errors in perception of stereoscopically presented virtual object locations in real display space. Proc Hum Factors Ergon Soc 1994, 38:250-254.
7. Grigg E.R.N. The trail of the invisible light 1965, Charles C. Thomas Pub Ltd, Springfield, IL.
8. Jaugeas F. Précis de radiodiagnostic 1918, Masson & Cie, Paris, France.
9. Fisher S.S. Virtual environments, personal simulation and telepresence. Virtual reality: theory, practice and promise 1991, Meckler Publishing, Westport, CT. S. Helsed, J. Roth (Eds.).
10. Fisher S.S., McGreevy M., Humphries J., Robinet W. Virtual environment display system. ACM Workshop on Interactive 3D Graphics 1986.
11. Paul P., Fleig O., Jannin P. Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: methods and performance evaluation. IEEE Trans Med Imaging 2005, 24:1500-1511.
12. Shuhaiber J.H. Augmented reality in surgery. Arch Surg 2004, 139:170-174.
13. Peters T.M. Image-guidance for surgical procedures. Phys Med Biol 2006, 51:R505-40.
14. Burdea G., Coiffet P. Virtual reality technology 1994, Wiley, New York.
15. Akay M., March A. Information technologies in medicine 2001, Wiley, New York.
16. Kelly P.J., Alker G.J., Goerss S. Computer-assisted stereotactic microsurgery for the treatment of intracranial neoplasms. Neurosurgery 1982, 10:324-331.
17. Roberts D.W., Strohbehn J.W., Hatch j.F., Murray W., Kettenberger H. A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. J Neurosurg 1986, 65:545-549.
18. Friets E.M., Strohbehn J.W., Hatch J.F., Roberts D.W. A frameless stereotaxic operating microscope for neurosurgery. IEEE Trans Biomed Eng 1989, 36:608-617.
19. Satava R., Cybersurgery Advanced technologies for surgical practice 1998, Wiley, New York.
20. Robb R.A. Virtual endoscopy: development and evaluation using the Visible Human datasets. Comput Med Imaging Graph 2000, 24:133-151.
21. Robb R.A., Hanson D.P. Biomedical image visualization research using the visible human datasets. Clin Anat 2006, 18:240-253.
22. Robb R.A. Biomedical imaging: visualization and analysis 2000, Wiley, New York.
23. Edwards P.J., King A.P., Maurer C.R., de Cunha D.A., Hawkes D.J., Hill D.L.G., et al. Design and evaluation of a system for microscope-assisted guided interventions (MAGI). IEEE Trans Med Imaging 2000, 19:1082-1093.
24. Edwards P.J., Johnson L.G., Hawkes D.J., Fenlon M.R., Strong A.J., Gleeson M.J. Clinical experience and perception in stereo augmented reality surgical navigation. Proc Medical Imaging and Augmented Reality, vol. 3150 of Lect Notes Comput Sci 2004, 369-376.
25. Ettinger G.L., Leventon M.E., Grimson W.E.L., Kikinis R., Gugino L., Cote W., et al. Experimentation with a transcranial magnetic stimulation system for functional brain mapping. Med Image Anal 1998, 2:477-486.
26. Feifer A., Delisle J., Anidjar M. Hybrid augmented reality simulator: preliminary construct validation of laparoscopic smoothness in a urology residency program. J Urol 2008, 180:1455-1459.
27. Magee D., Zhu Y., Ratnalingam R., Gardner P., Kessel D. An augmented reality simulator for ultrasound guided needle placement training. Med Biol Eng Comput 2007, 45:957-967.
28. Kerner K.F., Imielinska C., Rolland J., Tang H. Augmented reality for teaching endotracheal intubation: MR imaging to create anatomically correct models. Proc Annu AMIA Symp 2003, 88:8-9.
29. Rolland J.P., Wright D.L., Kancherla A.R. Towards a novel augmented-reality tool to visualize dynamic 3-D anatomy. Stud Health Technol Inform 1997, 39:337-348.
30. Botden S.M., Jakimowicz J.J. What is going on in augmented reality simulation in laparoscopic surgery?. Surg Endosc 2009, 23:1693-1700.
31. Koehring A., Foo J.L., Miyano G., Lobe T., Winer E. A framework for interactive visualization of digital medical images. J Laparoendosc Adv Surg Tech 2008, 18:697-706.
32. Lovo E.E., Quintana J.C., Puebla M.C., Torrealba G., Santos J.L., Lira I.H., et al. A novel, inexpensive method of image coregistration for applications in image-guided surgery using augmented reality. Neurosurgery 2007, 60:366-371.
33. Friets E.M., Strohbehm J.W., Roberts D.W. Curvature-based nonfiducial registration for the stereotactic operating microscope. IEEE Trans Biomed Eng 1995, 42:477-486.
34. Kaufman S., Poupyrev I., Miller E., Billinghurst M., Oppenheimer P., Weghorst S. New interface metaphors for complex information space visualization: an ECG monitor object prototype. Stud Health Technol Inform 1997, 39:131-140.
35. Vosburgh K.G., San José Estépar R. Natural Orifice Transluminal Endoscopic Surgery (NOTES): an opportunity for augmented reality guidance. Proc MMVR, vol. 125 of Stud Health Technol Inform 2007, 485-490.
36. Vogt S., Khamene A., Niemann H., Sauer F. An AR system with intuitive user interface for manipulation and visualization of 3D medical data. Proc MMVR, vol. 98 of Stud Health Technol Inform 2004, 397-403.
37. Teber D., Guven S., Simpfend”orfer T., Baumhauer M., Güven E.O., Yencilek F., et al. Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol 2009, 56:332-338.
38. Nakamoto M., Nakada K., Sato Y., Konishi K., Hashizume M., Tamura S. Intraoperative magnetic tracker calibration using a magneto-optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic surgery. IEEE Trans Med Imaging 2008, 27:255-270.
39. Peters T.M., Cleary K. Image-guided interventions: technology and applications 2008, Springer, Heidelberg, Germany.
40. Hounsfield G.N. Computerized transverse axial scanning (tomography). Part 1. Description of system. Br J Radiol 1973, 46:1016-1022.
41. Mori S., Endo M., Tsunoo T., Kandatsu S., Kusakabe M., et al. Physical performance evaluation of a 256-slice CT-scanner for four-dimensional imaging. Med Phys 2004, 31:1348-1356.
42. Klass O., Kleinhans S., Walker M.J., Olszewski M., Feuerlein S., Juchems M., et al. Coronary plaque imaging with 256-slice multidetector computed tomography: interobserver variability of volumetric lesion parameters with semiautomatic plaque analysis software. Int J Cardiovasc Imaging 2010, 26:711-720.
43. van Mieghem C.A., van der Ent M., de Feyter P.J. Percutaneous coronary intervention for chronic total occlusions: value of pre-procedural multislice CT guidance. Heart 2007, 93:1492-1493.
44. Hoffmann H., Dubel H.P., Laube H., Hamm B., Dewey M. Triage of patients with suspected coronary artery disease using multislice computed tomography. Acad Radiol 2007, 14:901-909.
45. Wong K. Imaging modalities. Image-guided interventions: technology and applications 2008, 241-273. Springer, Heidelberg, Germany. T.M. Peters, K. Cleary (Eds.).
46. Otsuka M., Sugahara S., Umeda K., Nakamura M., Nakamura A., Bonkohara Y., et al. Utility of multislice computed tomography as a strategic tool for complex percutaneous coronary intervention. Int J Cardiovasc Imaging 2008, 24:201-210.
47. Bronskill M., Sprawls P. The physics of MRI. Proc AAPM Summer School 1993.
48. Black P.M., Moriarty T., Alexander E., Stieg P., Kikinis R., Woodard E.J., et al. Development and implementation of intraoperative magnetic resonance imaging and its neurosurgical applications. Neurosurgery 1997, 41:831-842.
49. Schenck J.F., Jolesz F.A., Roemer P.B., Cline H.E., Kikinis R., Lorensen W.E., et al. L.E.T. Superconducting open-configuration MR imaging system for image-guided therapy. Radiology 1995, 195:805-814.
50. Moriarty T.M., Kikinis R., Jolesz F.A., Black P.M., Alexander E. Magnetic resonance imaging therapy. Intraoperative MR imaging. Neurosurg Clin N Am 1996, 7:323-331.
51. Steinmeier R., Fahlbusch R., Ganslandt O., Nimsky C., Huk W., Buchfelder M., et al. Intraoperative magnetic resonance imaging with the Magnetom Open scanner: concepts, neurosurgical indications, and procedures: a preliminary report. Neurosurgery 1998, 43:739-747.
52. Seebauer C.J., Bail H.J., Rump J.C., Walter T., Teichgraeber U.K. Advancements in orthopedic intervention: retrograde drilling and bone grafting of osteochondral lesions of the knee using MRI guidance. Cardiovasc Intervent Radiol 2010, 33:1230-1234.
53. Baumann F., Schmid C., Bernays R.L. Intraoperative magnetic resonance imaging-guided transsphenoidal surgery for giant pituitary adenomas. Neurosurg Rev 2010, 33:83-90.
54. van der Weide R., Zuiderveld K.J., Bakker C.J., Hoogenboom T., van Vaals J.J., Viergever M.A. Image guidance of endovascular interventions on a clinical MR scanner. IEEE Trans Med Imaging 1998, 17:779-785.
55. Hoult D.I., Saunders J.K., Sutherland G.R., Sharp J., Tomanek B. The engineering of an interventional MRI with a movable 1.5T magnet. J Magn Reson Imaging 2001, 13:78-86.
56. McVeigh E.R., Guttman M.A., Lederman R.J., Li M., Kocatruk O., Hunt T., et al. Real-time interactive MRI-guided cardiac surgery: aortic valve replacement using a direct apical approach. Magn Reson Med 2006, 56:958-964.
57. Rhode K.S., Sermesant M., Brogan D., Hegde S., Hipwell J., Lambiase P., et al. A system for real-time XMR guided cardiovascular intervention. IEEE Trans Med Imaging 2005, 24:1428-1440.
58. Szabo T. Diagnostic ultrasound imaging: inside and out 2004, Academic Press, Burlington, MA.
59. van Velthoven V., Auer L.M. Practical application of intraoperative ultrasound imaging. Acta Neurochir (Wien) 1990, 105:5-13.
60. Suematsu Y., Marx G.R., Stoll J.A., DuPont P.E., Howe R.D., Cleveland R.O., et al. Three-dimensional echocardiography-guided beating-heart surgery without cardiopulmonary bypass: a feasibility study. J Thorac Cardiovasc Surg 2004, 128:579-587.
61. Novotny P.M., Stoll J.A., Vasilyev N.V., del Nido P.J., Dupont P.E., Zickler T.E., et al. GPU based real-time instrument tracking with three-dimensional ultrasound. Med Image Anal 2007, 11:458-464.
62. Gobbi D.G., Comeau R.M., Peters T.M. Ultrasound probe tracking for real-time ultrasound/MRI overlay and visualization of brain shift. Proc Med Image Comput Comput Assist Interv, vol. 1679 of Lect Notes Comput Sci 1999, 920-927.
63. Linte C.A., Wiles A.D., Hill N., Moore J., Wedlake C., Guiraudon G.M., et al. An augmented reality environment for image-guidance of off-pump mitral valve implantation. Proc SPIE Medical Imaging 2007: Visualization and Image-Guided Procedures, vol. 6509, 65090N-12 2007.
64. Karnik V.V., Fenster A., Bax J., Cool D.W., Gardi L., Gracskov I., et al. Assessment of image registration accuracy in three-dimensional transrectal ultrasound guided prostate biopsy. Med Phys 2010, 37:802-813.
65. Linte C.A., Rettmann M.E., Dilger B., Gunawan M.S., Arunachalam S.P., Holmes D.R., et al. Calibration and evaluation of a magnetically tracked ICE probe for guidance of left atrial ablation therapy. Proc SPIE Medical Imaging 2012: Image-guided Procedures, Robotic Interventions and Modeling, vol. 8316, 83162P-10 2012.
66. Rohling R.N., Gee A.H., Berman L. Automatic registration of 3-D ultrasound images. Ultrasound Med Biol 1998, 24:841-854.
67. Fenster A., Lee D., Sherebrin S., Rankin R., Downey D. Three-dimensional ultrasound imaging of the vasculature. Ultrasonics 1998, 36:629-633.
68. Khamene A., Sauer F. A novel phantom-less spatial and temporal ultrasound calibration method. Proc Med Image Comput Comput Assist Interv, vol. 3750 of Lect Notes Comput Sci 2004, 671-678.
69. Pace D.F., Wiles A.D., Moore J., Wedlake C., Gobbi D.G., Peters T.M. Validation of four-dimensional ultrasound for targeting in minimally-invasive beating-heart surgery. Proc SPIE Med Imaging 2009: Visualization, Image-Guided Procedures and Modeling, 7261, 726115-1-12 2009.
70. Feuerstein M., Reichl T., Vogel J., Traub J., Navab N. Magneto-optical tracking of flexible laparoscopic ultrasound: model-based online detection and correction of magnetic tracking errors. IEEE Trans Med Imaging 2009, 28:951-967.
71. Comeau R., Sadikot A.F., Fenster A., Peters T.M. Intraoperative ultrasound for guidance and tissue shift correction in image-guided neurosurgery. Med Phys 2000, 27:787-800.
72. Barratt D.C., Chan C.S.K., Edwards P.J., Penney G.P., Slomczykowski M., Carter T.J., et al. Cadaver validation of ultrasound-based instantiation and registration of 3D statistical bone models for computer-aided orthopaedic surgery. Med Image Anal 2008, 12:358-374.
73. Sun Y., Kadoury S., Li Y., John M., Sauer F., Resnick J., et al. Image guidance of intracardiac ultrasound with fusion of pre-operative images. Proc Med Image Comput Comput Assist Interv, vol. 10 of Lect Notes Comput Sci, 60-7 2007.
74. Huang X., Moore J., Guiraudon G.M., Jones D.L., Bainbridge D., Ren J., et al. Dynamic 2D ultrasound and 3D CT image registration of the beating heart. IEEE Trans Med Imaging 2009, 28:1179-1189.
75. Stetten G.D., Chib V.S. Overlaying ultrasound images on direct vision. J Ultrasound Med 2001, 20:235-240.
76. Hendee W.R. The information perspective of radiology. Am J Roentgenol 1992, 158:1387-1389.
77. Peters T.M. Image-guided surgery: from X-rays to virtual reality. Comput Methods Biomech Biomed Eng 2000, 4:27-57.
78. Masamune K., Fichtinger G., Patriciu A., Susil R.C., Taylor R.H., Kavoussi L.R., et al. System for robotically assisted percutaneous procedures with computed tomography guidance. Comput Aided Surg 2001, 6:370-383.
79. Solomon S.B., Patriciu A., Bohlman M.E., Kavoussi L.R., Stoianovici D. Robotically driven interventions: a method of using CT fluoroscopy without radiation exposure to the physician. Radiology 2002, 225:277-282.
80. Sakurai H., Mitsuhashi N., Harashima K., Muramatsu H., Nakano T. CT-fluoroscopy guided interstitial brachytherapy with image-based treatment planning for unresectable locally recurrent rectal carcinoma. Brachytherapy 2004, 3:222-230.
81. Fahrig R., Holdsworth D.W. Three-dimensional computed tomographic reconstruction using a C-arm mounted XRII: image-based correction of gantry motion nonidealities. Med Phys 2000, 27:30-38.
82. Jaffray D.A., Siewerdsen J.H., Wong J.W., Martinez A.A. Flat-panel cone-beam computed tomography for image-guided radiation therapy. Int J Radiat Oncol Biol Phys 2002, 53:1337-1349.
83. Siewerdsen J.H., Moseley D.J., Burch S., Bisland S.K., Jaffray D.A., Bogaards A., et al. Volume CT with a flat-panel detector on a mobile, isocentric C-arm: Pre-clinical investigation in guidance of minimally invasive surgery. Med Phys 2005, 32:241-254.
84. Lauriola W., Nardella M., Strizzi V., Calì A., D'Angelo V., Florio F. 3D angiography in the evaluation of intracranial aneurysms before and after treatment: Initial experience. Radiol Med 2005, 109:98-107.
85. Manzke R., Reddy V.Y., Dalal S., Hanekamp A., Rasche V., Chan R.C. Intra-operative volume imaging of the left atrium and pulmonary veins with rotational X-ray angiography. Proc Med Image Comput Comput Assist Interv, vol. 9 of Lect Notes Comput Sci 2006, 604-611.
86. Meyer C., Manzke R., Peters J., Ecabert O., Weese J., Kneser R., et al. Automatic intra-operative generation of geometric left atrium/pulmonary vein models from rotational X-ray angiography. Proc Med Image Comput Comput Assist Interv, vol. 11 of Lect Notes Comput Sci, Vol. 6 2008, 1-9.
87. Manzke R., Meyer C., Ecabert O., Peters J., Noordhoek N.J., Thiagalingam A., et al. Automatic segmentation of rotational X-ray images for anatomic intra-procedural surface generation in atrial fibrillation ablation procedures. IEEE Trans Med Imaging 2009, 29:260-272.
88. Robb R.A. 3-D visualization in biomedical applications. Annu Rev Biomed Eng 1999, 1:377-399.
89. Ecabert O., Peters J., Schramm H., Lorenz C., von Berg J., Walker M.J., et al. Automatic model-based segmentation of the heart in CT images. IEEE Trans Med Imaging 2008, 27:1189-1201.
90. Huang J., Huang X., Metaxas D., Axel L. Adaptive metamorphs model for 3D medical image segmentation. Proc Med Image Comput Comput Assist Interv, vol. 4792 of Lect Notes Comput Sci 2007, 302-310.
91. Ben Ayed I., Li S., Ross I. Level set image segmentation with a statistical overlap constraint. Inf Process Med Imaging 2009, 21:589-601.
92. Frangi A.F., Rueckert D., Schnabel J.A., Niessen W.J. Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling. IEEE Trans Med Imaging 2002, 21:1151-1166.
93. Renno M.S., Shang Y., Sweeney J., Dossel O. Segmentation of 4D cardiac images: investigation on statistical shape models. Proc IEEE Eng Med Biol 2006, 3086-3089.
94. van Rikxoort E.M., Isgum I., Arzhaeva Y., Staring M., Klein S., Viergever M.A., et al. Adaptive local multi-atlas segmentation: application to the heart and the caudate nucleus. Med Image Anal 2010, 14:39-49.
95. Zhuang X., Rhode K., Arridge S., Razavi R., Hill D., Hawkes D., et al. An atlas-based segmentation propagation framework locally affine registration-application to automatic whole heart segmentation. Proc Med Image Comput Comput Assist Interv, vol. 5241 of Lect Notes Comput Sci 2008, 425-433.
96. Niessen W. Model-based image segmentation for image-guided interventions. Image-guided interventions: technology and applications 2008, 219-240. Springer, Heidelberg, Germany. T.M. Peters, K. Cleary (Eds.).
97. Lorensen W., Cline H. Marching cubes: a high-resolution 3D surface reconstruction algorithm. Comput Graph 1987, 21:163-169.
98. Renze K.J., Oliver J.H. Generalized unstructured decimation. IEEE Comput Graph Appl 1996, 16:24-32.
99. Crouch D., Robb R.A. A new algorithm for efficient polygon decimation for virtual reality applications in medicine. Proc SPIE Med Imaging 1997, 3031:514-517.
100. Zhang Q., Eagleson R., Peters T.M. Dynamic real-time 4D cardiac MDCT image display using GPU-accelerated volume rendering. Comput Med Imaging Graph 2009, 33:461-476.
101. Wiles A.D., Thompson D.G., Frantz D.D. Accuracy assessment and interpretation for optical tracking systems. Proc SPIE Medical Imaging 2007: Visualization, Image-Guided Procedures and Display, 5367 2004, 421-432.
102. Birkfellner W., Watzinger F., Wanschitz F., Enislidis G., Kollmann C., Rafolt D., et al. Systematic distortions in magnetic position digitizers. Med Phys 1998, 25:2242-2248.
103. Frantz D.D., Wiles A.D., Leis S.E., Kirsch S.R. Accuracy assessment protocols for electromagnetic tracking systems. Phys Med Biol 2003, 48:2241-2251.
104. Hummel J.B., Bax M.R., Figl M.L., Kang Y., Maurer C., Birkfellner W.W., et al. Design and application of an assessment protocol for electromagnetic tracking systems. Med Phys 2005, 32:2371-2379.
105. Nolte L.P., Zamorano L., Visarius H., Berlemann U., Langlotz F., Arm E., et al. Clinical evaluation of a system for precision enhancement in spine surgery. Clin Biomech 1995, 10:293-303.
106. Rohling R., Munger P., Hollerbach J.M., Peter T. Comparison of relative accuracy between a mechanical and an optical position tracker for image-guided neurosurgery. J Image Guid Surg 1995, 1:30-34.
107. Eljamel M.S. Accuracy, efficacy, and clinical applications of the Radionics Operating Arm System. Comput Aided Surg 1997, 2:292-297.
108. Anon J.B. Computer-aided endoscopic sinus surgery. Laryngoscope 1998, 108:949-961.
109. Watzinger F., Birkfellner W., Wanschitz F., Millesi W., Schopper C., Sinko K., et al. Positioning of dental implants using computer-aided navigation and an optical tracking system: case report and presentation of a new method. J Craniomaxillofac Surg 1999, 27:77-81.
110. Khadem R., Yeh C.C., Sadeghi-Tehrani M., Bax M.R., Johnson J.A., Welch J.N., et al. Comparative tracking error analysis of five different optical tracking systems. Comput Aided Surg 2000, 5:98-107.
111. Schmerber S., Chassat F. Accuracy evaluation of a CAS system: laboratory protocol and results with 6D localizers, and clinical experiences in otorhinolaryngology. Comput Aided Surg 2001, 6:1-13.
112. Linte C.A., Moore J., Wiles A.D., Wedlake C., Peters T.M. Targeting accuracy under model-to-subject misalignments in model-guided cardiac surgery. Proc Med Image Comput Comput Assist Interv., vol. 5761 of Lect Notes Comput Sci 2009, 361-368.
113. Wood B.J., Zhang H., Glossop D.A.N., Durrani A., Ranjan S., Lindisch D., et al. Navigation with electromagnetic tracking for interventional radiology procedures: a feasibility study. J Vasc Interv Radiol 2005, 16:493-505.
114. Wiles A.D., Linte C.A., Moore J., Wedlake C., Peters T.M. Object identification accuracy under ultrasound enhanced virtual reality for minimally invasive cardiac surgery. Proc SPIE Medical Imaging 2008: Visualization, Image-Guided Procedures and Modeling, 6918, 69180E1-12 2008.
115. Sauer F. Image registration enabling technology for image-guided surgery and therapy. Proc IEEE Eng Med Biol 2005, 7242-7245.
116. Hajnal J.V., Hill D.L.G., Hwakes D.J. Introduction. Medical image registration 2001, CRC Press, Boca Raton, FL, USA, p. 1-8.
117. Fitzpatrick J.M., West J.B., Maurer C.R.J. Predicting error in rigid-body point-based registration. IEEE Trans Med Imaging 1998, 17:694-702.
118. Yaniv Z. Rigid registration. Image-guided interventions: technology and applications 2008, 159-192. Springer, Heidelberg, Germany. T.M. Peters, K. Cleary (Eds.).
119. Crum W.R., Griffin L.D., Hill D.L., Hawkes D.J. Zen and the art of medical image registration: correspondence, homology, and quality. Neuroimage 2003, 20:1425-1437.
120. Pelizzari C.A., Chen G.T., Spelbring D.R., Weichselbaum R.R., Chen C.T. Accurate three-dimensional registration of CT, PET, and/or MR images of the brain. J Comput Assist Tomogr 1989, 13:20-26.
121. Besl P.J., McKay N.D. A method for registration of 3D shapes. IEEE Trans Pattern Anal Machine Intell 1992, 14:239-255.
122. Hawkes D.J., Barratt D., Blackall J.M., Chan C., Edwards P.J., Rhode K., et al. Tissue deformation and shape models in image-guided interventions: a discussion paper. Med Image Anal 2005, 163-175.
123. Guo T., Parent A., Peters T.M. Surgical targeting accuracy analysis of six methods for subthalamic nucleus deep brain stimulation. Comput Aided Surg 2007, 12:325-334.
124. lavallee S., Merloz P., Stindel E., Kilian P., Troccaz J., Cinquin P., et al. Introducing an intra-operative imaging modality to reconstruct 3D bone surfaces for minimally invasive surgery. Proc CAOS 2004, 38-39.
125. Yao J., Taylor R. A multiple-layed flexible mesh template matching method for nonrigid registration between a pelvis model and CT images. Proc SPIE Med Imaging: Image Process 2003, 5032:1117-1125.
126. McClelland J.R., Blackall J.M., Tarte S., Chandler A.C., Hughes S., Ahmad S., et al. A continuous 4D motion model from multiple respiratory cycles for use in lung radiotherapy. Med Phys 2006, 33:3348-3358.
127. Penney G.P., Blackall J.M., Hamady M.S., Sabharwal T., Adam A., Hawkes D.J. Registration of freehand 3D ultrasound and magnetic resonance liver images. Med Image Anal 2004, 8:81-91.
128. Wierzbicki M., Drangova M., Guiraudon G.M., Peters T.M. Validation of dynamic heart models obtained using non-linear registration for virtual reality training, planning, and guidance of minimally invasive cardiac surgeries. Med Image Anal 2004, 8:387-401.
129. Perperidis D., Mohiaddin R., Rueckert D. Construction of a 4D statistical atlas of the cardiac anatomy and its use in classification. Proc Med Image Comput Comput Assist Interv, Lect Notes Comput Sci 2005, 402-410.
130. Hajnal J.V., Hill D.L.G., Hawkes D.J. Medical image registration 2001, CRC Press, Boca Raton, FL, USA.
131. Gibson S., Fyock C., Grimson E., Kanade T., Kikinis R., Lauer H., et al. Volumetric object modeling for surgical simulation. Med Image Anal 1998, 2:121-132.
132. Robb R.A., Hanson D.P., Karwoski R.A., Larson A.G., Workman E.L., et al. Analyze: a comprehensive, operator-interactive software package for multidimensional medical image display and analysis. Comput Med Imaging Graph 1989, 13:433-454.
133. Shahidi R. Clinical applications of three-dimensional rendering of medical data sets. Proc IEEE Virtual Augment Real Med 1998, 86:555-568.
134. Linte C.A., Augustine K.E., Huddleston P.H., Stans A.A., Holmes D.R., Robb R.A. Enhanced planning of interventions for spinal deformity correction using virtual modeling and visualization techniques. Proc Augment Envir Comput Assist Interv, 2011, vol. 7264 of Lect Notes Comput Sci 2012, 93-105.
135. Jannin P., Morandi X. Surgical models for computer-assisted neurosurgery. Neuroimage 2007, 37:783-791.
136. Edwards P., Hawkes D.J., Hill D.L., Jewell D., Gleeson M. Augmentation of reality using an operating microscope for otolaryngology and neurosurgical guidance. J Image Guid Surg 1995, 1:172-178.
137. Birkfellner W., Figl M., Huber K., Watzinger F., Wanschitz F., Hummel J., et al. A head-mounted operating binocular for augmented reality visualization in medicine - design and initial evaluation. IEEE Trans Med Imaging 2002, 21:991-997.
138. Birkfellner W., Huber K., Watzinger F., Figl M., Wanschitz F., Hanel R., et al. Development of the varioscopear: a see-through HMD for computer-aided surgery. Proc IEEE ACM Int Symp Augment Real 2000, 5:4-9.
139. Masutani Y., Iwahara M., Samuta O., Nishi Y., Suzuki M., Suzuki N., et al. Development of integral photography-based enhanced reality visualization system for surgical support. Proc Int Symp Comput Aided Surg 1995, 1:6-7.
140. Masamune K., Masutani Y., Nakajima S., Sakuma I., Dohi T., Iseki H., et al. Three-dimensional slice image overlay system with accurate depth perception for surgery. Proc Med Image Comput Comput Assist Interv, Lect Notes Comput Sci 2000, 395-402.
141. Blackwell M., Nikou C., DiGioia A.M., Kanade T. An image overlay system for medical data visualization. Med Image Anal 2000, 4:67-72.
142. Sauer F., Vogt S., Khamene A. Augmented reality. Image-guided interventions: technology and applications 2008, 81-119. Springer, Heidelberg, Germany. T.M. Peters, K. Cleary (Eds.).
143. Sielhorst T., Feuerstein M., Navab N. Advanced medical displays: a literature review of augmented reality. J Disp Technol 2008, 4:451-467.
144. Cohen D., Mayer E., Chen D., Anstee A., Vale J., Yang G.Z., et al. Augmented reality image guidance in minimally invasive prostatectomy. Proc Prostate Cancer Imaging, vol. 6367 of Lect Notes Comput Sci 2010, 101-110.
145. Moore J., Wiles A., Wedlake C., Kiaii B., Peters T.M. Integration of trans-esophageal echocardiography with magnetic tracking technology for cardiac interventions. Proc SPIE Medical Imaging 2010: Visualization, Image-guided Procedures and Modeling, 7625, 76252Y-1-10 2010.
146. Rousseau F., Hellier P., Letteboer M.M., Niessen W.J., Barillot C. Quantitative evaluation of three calibration methods for 3-D freehand ultrasound. IEEE Trans Med Imaging 2006, 25:1492-1501.
147. Shahidi R., Bax M.R., Maurer C.R., Johnson J.A., Wilkinson E.P., Wang B., et al. Implementation, calibration and accuracy testing of an image-enhanced endoscopy system. IEEE Trans Med Imaging 2002, 21:1524-1535.
148. Azuma RT. Predictive tracking for augmented reality. PhD thesis, UNC-Chapel Hill; 1995.
149. Holmes D.R., Rettmann M., Robb R.A. Visualization in image-guided interventions. Image-guided interventions: technology and applications 2008, 45-80. Springer, Heidelberg, Germany. T.M. Peters, K. Cleary (Eds.).
150. Azuma R.T. A survey of augmented reality. Teleoper Virtual Environ 1997, 6:355-385.
151. Figl M., Ede C., Hummel J., Wanschitz F., Seemann R., Ewers R., Bergmann H., Birkfellner W. Latency in medical augmented reality systems. Proc MICCAI - Augment Environ Med Imaging Comput Aided Surg 2006, 1:45-47.
152. Li B., Young A.A., Cowan B.R. GPU accelerated non-rigid registration for the evaluation of cardiac function. Proc Med Image Comput Comput Assist Interv, vol. 5242 of Lect Notes Comput Sci 2008, 880-887.
153. Gu X., Pan H., Liang Y., Castillo R., Jiang S.B., Yang D., et al. Implementation and evaluation of various demons deformable image registration algorithms on a GPU. Phys Med Biol 2010, 55:207-219.
154. Kutter O., Shams R., Navab N. Visualization and GPU-accelerated simulation of medical ultrasound from CT images. Comput Methods Programs Biomed 2009, 94:250-266.
155. Viant W.J. The development of an evaluation framework for the quantitative assessment of computer-assisted surgery and augmented reality accuracy performance. Stud Health technol Inform 2001, 81:534-540.
156. Jannin P., Fitzpatrick J.M., Hawkes D.J., Pannec X., Shahidi R., Vannier M.W. Validation of medical image processing in image-guided therapy. IEEE Trans Med Imaging 2002, 21:1445-1449.
157. Jannin P., Korb W. Assessment of Image Guided Interventions. Image-guided interventions: technology and applications 2008, Springer-Verlag, Berlin/Heidelberg. T.M. Peters, K. Cleary (Eds.).
158. Neumuth T., Jannin P., Strauss G., Meixensberger J., Burgert O. Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 2009, 16:72-80.
159. Bowman D.A., Kruijff E., laViola J.J., Poupyrev I. 3D user interfaces: theory and practice 2004, Addison Wesley Longman Publishing, Redwood City, CA, USA.
160. Reitinger B., Bornik A., Beichel R., Schmalsteig D. Liver surgery planning using virtual reality. IEEE Comput Graph Appl 2006, 2:36-47.
161. Satava R.M., Jones S.B. Current and future application of virtual reality in medicine. Proc IEEE Virtual Augment Real Med 1998, 86:484-489.
162. Ahmadi S.A., Sielhorst T., Stauder M., Horn M., Feussner H., Navab N. Recovery of surgical workflow without explicit models. Proc Med Image Comput Comput Assist Interv, vol. 4190 of Lect Notes Comput Sci 2006, 420-428.
163. Linte C.A., Moore J., Wiles A., Wedlake C., Peters T. Virtual reality-enhanced ultrasound guidance: a novel technique for intracardiac interventions. Comput Aided Surg 2008, 13:82-94.
164. Moore J., Wedlake C., Bainbridge D., Guiraudon G.M., Chu M., Kiaii B., et al. A navigation platform for guidance of beating heart transapical mitral valve repair. Proc Inform Proces for Computer Assist Interv, 2012, vol. 7330 of Lect Notes Comput Sci 2012, 84-93.
165. Moore J., Clark C., Bainbridge D., Wedlake C., Wiles A., Pace D.F., et al. Image guidance for spinal facet injections using tracked ultrasound. Proc Med Image Comput Comput Assist Interv, vol. 5761 of Lect Notes Comput Sci 2009, 516-523.
166. Rettmann M.E., Holmes D.R., Cameron B., Robb R.A. An event-driven distributed processing architecture for image-guided cardiac ablation therapy. Comput Methods Programs Biomed 2009, 95:95-104.
167. Rettmann M.E., Holmes D.R., Su Y., Cameron B.M., Camp J.J., Packer D.L., et al. An integrated system for real-time image-guided cardiac catheter ablation. Proc Medicine Meets Virtual Reality, vol. 119 of Stud Health Technol Inform 2006, 455-460.
168. Bajura M., Fuchs H., Ohbuchi R. Merging virtual objects with the real world: seeing ultrasound imagery within the patient. Comput Graph 1992, 26:203-210.
169. Fuchs H., Livingston M.A., Raskar R., Colucci D., Keller K., State A., et al. Augmented reality visualization for laparoscopic surgery. Med Image Comput Comput Assist Interv, vol. 1496 of Lect Notes Comput Sci 1998, 934-943.
170. Sauer F., Khamene A., Vogt S. An augmented reality navigation system with a single-camera tracker: system design and needle biopsy phantom trial. Proc Med Image Comput Comput Assist Interv, Lect Notes Comput Sci 2002, 116-124.
171. Maurer C., Sauer F., Brown C., Hu B., Bascle B., Geiger B., et al. Augmented reality visualization of brain structures with stereo and kinetic depth cues: System description and initial evaluation with head phantom. Proc SPIE Med Imaging 2001, 4319:445-456.
172. Sauer F., Khamene A., Bascle B., Rubino G. A head-mounted display system for augmented reality image guidance: towards surgical evaluation for iMRI neurosurgery. Proc Med Image Comput Comput Assist Interv, Lect Notes Comput Sci 2001, 707-716.
173. Das M., Sauer F., Schoepf U.J., Khamene A., Vogt S.K., Schaller S., et al. Augmented reality visualization for CT-guided interventions: system description, feasibility, and initial evaluation in an abdominal phantom. Radiology 2006, 230-235.
174. Grimson W., Kikinis R., Jolesz F., Black P. Image-guided surgery. Sci Am 1999, 280:62-69.
175. Grimson W.L., Ettinger G.J., White S.J., Lozano-Perez T., Wells W.M., Kikinis R. An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization. IEEE Trans Med Imaging 1996, 15:129-140.
176. Figl M., Rueckert D., Hawkes D., Casula R., Hu M., Pedro O., et al. Image guidance for robotic minimally invasive coronary artery bypass. Comput Med Imaging Graph 2010, 34:61-68.
177. Azuma R.T., Bishop G. Improving static and dynamic registration in a see-through HMD. Proc SIGGRAPH, Computer Graphics 1994, 197-204.
178. Bajura M., Neumann U. Dynamic registration correction in video-based augmented reality systems. IEEE Comput Graph Appl 1995, 15:52-60.
179. Mellor JP. Enhanced reality visualization in a surgical environment. PhD thesis, Dept. Electrical Eng., MIT; 1995.
180. Mellor J.P. Realtime camera calibration for enhanced reality visualization. Proc Computer Vision, Virtual Reality and Robotics in Medicine 1995, 471-475.
181. Neumann U., Cho Y. A self-tracking augmented reality system. Proc VRST 1996, 109-115.
182. State A., Hirota G., Chen D.T., Garrett B., Livingston M. Superior augmented reality registration by integrating landmark tracking and magnetic tracking. Proc SIGGRAPH, Computer Graphics 1996, 429-438.
183. Wierzbicki M., Moore J., Drangova M., Peters T.M. Subject-specific models for image-guided cardiac surgery. Phys Med Biol 2008, 53:5295-5312.
184. Birkfellner W., Watzinger F., Wanschitz F., Ewers R., Bergmann H. Calibration of tracking systems in a surgical environment. IEEE Trans Med Imaging 1998, 17:737-742.
185. Kwartowitz D.M., Rettmann M.E., Holmes D.R., Robb R.A. A novel technique for analysis of accuracy of magnetic tracking systems used in image-guided surgery. SPIE Medical Imaging Symposium: Visualization, Image-guided Procedures and Modeling, 7625, 76251L-1-8 2010.
186. Jolesz F. Advanced Multi-Modality Image Guided Operating (AMIGO) suite. Proc 4th Image-guided Therapy (NCIGT) Workshop 2011, 15-16.