Review of fluorescence guided surgery visualization and overlay techniques

Biomedical Optics Express

Volumn 6, Issue 10, 2015, Pages 3765-3782

  Elliott, Jonathan T ^a   Dsouza, Alisha V ^a   Davis, Scott C ^a   Olson, Jonathan D ^b   Paulsen, Keith D ^a   Roberts, David W ^b,c   Pogue, Brian W ^a,c  

^a DARTMOUTH COLLEGE   (United States)

^b DARTMOUTH HITCHCOCK MEDICAL CENTER   (United States)

^c DARTMOUTH MEDICAL SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COLOR VISION; DECISION MAKING; FLUORESCENCE; INFRARED DEVICES; SURGERY; VISUALIZATION;

CLINICAL DECISION; COLOR PERCEPTION; DISPLAY CHOICES; DYNAMIC RANGE COMPRESSION; HUMAN PERCEPTION; LOW CONCENTRATIONS; OVERLAY TECHNIQUES; SURGICAL ENVIRONMENT;

DATA VISUALIZATION;

COLOR VISION; COMPRESSION; CONSENSUS DEVELOPMENT; DECISION MAKING; FLUORESCENCE; HUMAN; INFRARED RADIATION; SURGERY;

EID: 84948774326     PISSN: None     EISSN: 21567085     Source Type: Journal    
DOI: 10.1364/BOE.6.003765     Document Type: Article

References (75)

1. M. I. Miga, K. D. Paulsen, J. M. Lemery, S. D. Eisner, A. Hartov, F. E. Kennedy, and D. W. Roberts, "Modelupdated image guidance: initial clinical experiences with gravity-induced brain deformation," IEEE Trans. Med. Imaging 18(10), 866-874 (1999).
2. D. W. Roberts, J. W. Strohbehn, J. F. Hatch, W. Murray, and H. Kettenberger, "A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope," J. Neurosurg. 65(4), 545-549 (1986).
3. R. M. Comeau, A. F. Sadikot, A. Fenster, and T. M. Peters, "Intraoperative ultrasound for guidance and tissue shift correction in image-guided neurosurgery," Med. Phys. 27(4), 787-800 (2000).
4. S. B. Sobottka, J. Bredow, B. Beuthien-Baumann, G. Reiss, G. Schackert, and R. Steinmeier, "Comparison of functional brain PET images and intraoperative brain-mapping data using image-guided surgery," Comput. Aided Surg. 7(6), 317-325 (2002).
5. B. T. Bethea, A. M. Okamura, M. Kitagawa, T. P. Fitton, S. M. Cattaneo, V. L. Gott, W. A. Baumgartner, and D. D. Yuh, "Application of haptic feedback to robotic surgery," J. Laparoendosc. Adv. Surg. Tech. A 14(3), 191-195 (2004).
6. L. Boni, G. David, A. Mangano, G. Dionigi, S. Rausei, S. Spampatti, E. Cassinotti, and A. Fingerhut, "Clinical applications of indocyanine green (ICG) enhanced fluorescence in laparoscopic surgery," Surg. Endosc. 29(7), 2046-2055 (2015).
7. W. Stummer, H. J. Reulen, A. Novotny, H. Stepp, and J. C. Tonn, "Fluorescence-guided resections of malignant gliomas-an overview," Acta Neurochir. Suppl. (Wien) 88, 9-12 (2003).
8. W. Stummer, U. Pichlmeier, T. Meinel, O. D. Wiestler, F. Zanella, and H. J. Reulen; ALA-Glioma Study Group, "Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial," Lancet Oncol. 7(5), 392-401 (2006).
9. K. S. Samkoe, K. Sexton, K. M. Tichauer, S. K. Hextrum, O. Pardesi, S. C. Davis, J. A. O'Hara, P. J. Hoopes, T. Hasan, and B. W. Pogue, "High vascular delivery of EGF, but low receptor binding rate is observed in AsPC-1 tumors as compared to normal pancreas," Mol. Imaging Biol. 14(4), 472-479 (2012).
10. C. H. Heath, N. L. Deep, L. Sweeny, K. R. Zinn, and E. L. Rosenthal, "Use of panitumumab-IRDye800 to image microscopic head and neck cancer in an orthotopic surgical model," Ann. Surg. Oncol. 19(12), 3879-3887 (2012).
11. K. R. Zinn, M. Korb, S. Samuel, J. M. Warram, D. Dion, C. Killingsworth, J. Fan, T. Schoeb, T. V. Strong, and E. L. Rosenthal, "IND-directed safety and biodistribution study of intravenously injected cetuximab-IRDye800 in cynomolgus macaques," Mol. Imaging Biol. 17(1), 49-57 (2015).
12. J. S. Mieog, A. L. Vahrmeijer, M. Hutteman, J. R. van der Vorst, M. Drijfhout van Hooff, J. Dijkstra, P. J. Kuppen, R. Keijzer, E. L. Kaijzel, I. Que, C. J. van de Velde, and C. W. Löwik, "Novel intraoperative near-infrared fluorescence camera system for optical image-guided cancer surgery," Mol. Imaging 9(4), 223-231 (2010).
13. R. W. Holt, J. L. Demers, K. J. Sexton, J. R. Gunn, S. C. Davis, K. S. Samkoe, and B. W. Pogue, "Tomography of epidermal growth factor receptor binding to fluorescent Affibody in vivo studied with magnetic resonance guided fluorescence recovery in varying orthotopic glioma sizes," J. Biomed. Opt. 20(2), 026001 (2015).
14. K. Sexton, K. Tichauer, K. S. Samkoe, J. Gunn, P. J. Hoopes, and B. W. Pogue, "Fluorescent affibody peptide penetration in glioma margin is superior to full antibody," PLoS One 8(4), e60390 (2013).
15. S. Oliveira, R. Heukers, J. Sornkom, R. J. Kok, and P. M. van Bergen En Henegouwen, "Targeting tumors with nanobodies for cancer imaging and therapy," J. Control. Release 172(3), 607-617 (2013).
16. M. Kijanka, F. J. Warnders, M. El Khattabi, M. Lub-de Hooge, G. M. van Dam, V. Ntziachristos, L. de Vries, S. Oliveira, and P. M. van Bergen En Henegouwen, "Rapid optical imaging of human breast tumour xenografts using anti-HER2 VHHs site-directly conjugated to IRDye 800CW for image-guided surgery," Eur. J. Nucl. Med. Mol. Imaging 40(11), 1718-1729 (2013).
17. H. Shi, W. Cui, X. He, Q. Guo, K. Wang, X. Ye, and J. Tang, "Whole cell-SELEX aptamers for highly specific fluorescence molecular imaging of carcinomas in vivo," PLoS One 8(8), e70476 (2013).
18. E. M. Sevick-Muraca, R. Sharma, J. C. Rasmussen, M. V. Marshall, J. A. Wendt, H. Q. Pham, E. Bonefas, J. P. Houston, L. Sampath, K. E. Adams, D. K. Blanchard, R. E. Fisher, S. B. Chiang, R. Elledge, and M. E. Mawad, "Imaging of lymph flow in breast cancer patients after microdose administration of a near-infrared fluorophore: feasibility study," Radiology 246(3), 734-741 (2008).
19. U. S. D. H. H. S. C. D. E., Research, "Guidance for industry, investigators, and reviewers exploratory IND studies," U. D. H. H. Services, ed. (Rockville, MD, 2006).
20. J. Glatz, P. Symvoulidis, P. B. Garcia-Allende, and V. Ntziachristos, "Robust overlay schemes for the fusion of fluorescence and color channels in biological imaging," J. Biomed. Opt. 19(4), 040501 (2014).
21. N. L. Martirosyan, J. Skoch, J. R. Watson, G. M. Lemole, Jr., M. Romanowski, and R. Anton, "Integration of indocyanine green videoangiography with operative microscope: augmented reality for interactive assessment of vascular structures and blood flow," Neurosurgery 11(Suppl 2), 252-258 (2015).
22. D. W. Roberts, P. A. Valdés, B. T. Harris, K. M. Fontaine, A. Hartov, X. Fan, S. Ji, S. S. Lollis, B. W. Pogue, F. Leblond, T. D. Tosteson, B. C. Wilson, and K. D. Paulsen, "Coregistered fluorescence-enhanced tumor resection of malignant glioma: relationships between δ-aminolevulinic acid-induced protoporphyrin IX fluorescence, magnetic resonance imaging enhancement, and neuropathological parameters. Clinical article," J. Neurosurg. 114(3), 595-603 (2011).
23. P. A. Valdés, A. Kim, F. Leblond, O. M. Conde, B. T. Harris, K. D. Paulsen, B. C. Wilson, and D. W. Roberts, "Combined fluorescence and reflectance spectroscopy for in vivo quantification of cancer biomarkers in lowand high-grade glioma surgery," J. Biomed. Opt. 16(11), 116007 (2011).
24. K. M. Tichauer, K. S. Samkoe, J. R. Gunn, S. C. Kanick, P. J. Hoopes, R. J. Barth, P. A. Kaufman, T. Hasan, and B. W. Pogue, "Microscopic lymph node tumor burden quantified by macroscopic dual-tracer molecular imaging," Nat. Med. 20(11), 1348-1353 (2014).
25. K. S. Samkoe, K. M. Tichauer, J. R. Gunn, W. A. Wells, T. Hasan, and B. W. Pogue, "Quantitative In Vivo Immunohistochemistry of Epidermal Growth Factor Receptor Using a Receptor Concentration Imaging Approach," Cancer Res. 74(24), 7465-7474 (2014).
26. E. R. Kandel, J. H. Schwartz, and T. M. Jessell, Principles of Neural Science (McGraw-Hill Companies, Inc., New York, N. Y., 2000).
27. G. Wald, "The molecular basis of visual excitation," Nature 219(5156), 800-807 (1968).
28. M. Niccoli, "Geophysics tutorial: how to evaluate and compare color maps," Leading Edge (Tulsa Okla.) 33, 910-912 (2014).
29. K. Moreland, "Diverging color maps for scientific visualization," in Advances in Visual Computing, G. Bebis, R. Boyle, B. Parvin, D. Koracin, Y. Kuno, J. Wang, R. Pajarola, P. Lindsrom, A. Hinkenjann, M. L. Encarnacao, C. T. Silva, and D. Coming, eds. (Springer-Verlag Berlin Heidelberg, 2009).
30. B. E. Rogowitz, and A. D. Kalvin, "The "Which Blair project": A quick visual method for evaluating perceptual color maps," Visualization 2001, Proceedings, 183-190 (2001).
31. J. M. Wolfe, "Guided Search 2. 0 A revised model of visual search," Psychon. Bull. Rev. 1(2), 202-238 (1994).
32. H. E. Egeth and S. Yantis, "Visual attention: control, representation, and time course," Annu. Rev. Psychol. 48(1), 269-297 (1997).
33. I. Rock and D. Gutman, "The effect of inattention on form perception," J. Exp. Psychol. Hum. Percept. Perform. 7(2), 275-285 (1981).
34. M. M. Chou, E. S. Ho, and Y. H. Lee, "Prenatal diagnosis of placenta previa accreta by transabdominal color Doppler ultrasound," Ultrasound Obstet. Gynecol. 15(1), 28-35 (2000).
35. K. Kitajima, Y. Suenaga, Y. Ueno, T. Kanda, T. Maeda, N. Makihara, Y. Ebina, H. Yamada, S. Takahashi, and K. Sugimura, "Value of fusion of PET and MRI in the detection of intra-pelvic recurrence of gynecological tumor: comparison with 18F-FDG contrast-enhanced PET/CT and pelvic MRI," Ann. Nucl. Med. 28(1), 25-32 (2014).
36. K. Fukuda, H. Kataoka, N. Nakajima, J. Masuoka, T. Satow, and K. Iihara, "Efficacy of FLOW 800 with indocyanine green videoangiography for the quantitative assessment of flow dynamics in cerebral arteriovenous malformation surgery," World. Neurosurg. 83(2), 203-210 (2015).
37. E. L. Jewell, J. J. Huang, N. R. Abu-Rustum, G. J. Gardner, C. L. Brown, Y. Sonoda, R. R. Barakat, D. A. Levine, and M. M. Leitao, Jr., "Detection of sentinel lymph nodes in minimally invasive surgery using indocyanine green and near-infrared fluorescence imaging for uterine and cervical malignancies," Gynecol. Oncol. 133(2), 274-277 (2014).
38. N. Tagaya, H. Aoyagi, A. Nakagawa, A. Abe, Y. Iwasaki, M. Tachibana, and K. Kubota, "A novel approach for sentinel lymph node identification using fluorescence imaging and image overlay navigation surgery in patients with breast cancer," World J. Surg. 35(1), 154-158 (2011).
39. S. L. Troyan, V. Kianzad, S. L. Gibbs-Strauss, S. Gioux, A. Matsui, R. Oketokoun, L. Ngo, A. Khamene, F. Azar, and J. V. Frangioni, "The FLARE intraoperative near-infrared fluorescence imaging system: a first-inhuman clinical trial in breast cancer sentinel lymph node mapping," Ann. Surg. Oncol. 16(10), 2943-2952 (2009).
40. B. E. Schaafsma, J. S. Mieog, M. Hutteman, J. R. van der Vorst, P. J. Kuppen, C. W. Löwik, J. V. Frangioni, C. J. van de Velde, and A. L. Vahrmeijer, "The clinical use of indocyanine green as a near-infrared fluorescent contrast agent for image-guided oncologic surgery," J. Surg. Oncol. 104(3), 323-332 (2011).
41. Q. T. Nguyen, E. S. Olson, T. A. Aguilera, T. Jiang, M. Scadeng, L. G. Ellies, and R. Y. Tsien, "Surgery with molecular fluorescence imaging using activatable cell-penetrating peptides decreases residual cancer and improves survival," Proc. Natl. Acad. Sci. U. S. A. 107(9), 4317-4322 (2010).
42. E. S. Olson, T. Jiang, T. A. Aguilera, Q. T. Nguyen, L. G. Ellies, M. Scadeng, and R. Y. Tsien, "Activatable cell penetrating peptides linked to nanoparticles as dual probes for in vivo fluorescence and MR imaging of proteases," Proc. Natl. Acad. Sci. U. S. A. 107(9), 4311-4316 (2010).
43. J. V. Frangioni, "In vivo near-infrared fluorescence imaging," Curr. Opin. Chem. Biol. 7(5), 626-634 (2003).
44. S. L. Gibbs-Strauss, K. A. Nasr, K. M. Fish, O. Khullar, Y. Ashitate, T. M. Siclovan, B. F. Johnson, N. E. Barnhardt, C. A. Tan Hehir, and J. V. Frangioni, "Nerve-highlighting fluorescent contrast agents for imageguided surgery," Mol. Imaging 10(2), 91-101 (2011).
45. S. Kim, Y. T. Lim, E. G. Soltesz, A. M. De Grand, J. Lee, A. Nakayama, J. A. Parker, T. Mihaljevic, R. G. Laurence, D. M. Dor, L. H. Cohn, M. G. Bawendi, and J. V. Frangioni, "Near-infrared fluorescent type II quantum dots for sentinel lymph node mapping," Nat. Biotechnol. 22(1), 93-97 (2004).
46. E. G. Soltesz, S. Kim, R. G. Laurence, A. M. DeGrand, C. P. Parungo, D. M. Dor, L. H. Cohn, M. G. Bawendi, J. V. Frangioni, and T. Mihaljevic, "Intraoperative sentinel lymph node mapping of the lung using near-infrared fluorescent quantum dots," Ann. Thorac. Surg. 79(1), 269-277 (2005).
47. P. S. Adusumilli, D. P. Eisenberg, Y. S. Chun, K. W. Ryu, L. Ben-Porat, K. J. Hendershott, M. K. Chan, R. Huq, C. C. Riedl, and Y. Fong, "Virally directed fluorescent imaging improves diagnostic sensitivity in the detection of minimal residual disease after potentially curative cytoreductive surgery," J. Gastrointest. Surg. 9, 1138-1146 (2005).
48. M. Hutteman, J. S. Mieog, J. R. van der Vorst, G. J. Liefers, H. Putter, C. W. Löwik, J. V. Frangioni, C. J. van de Velde, and A. L. Vahrmeijer, "Randomized, double-blind comparison of indocyanine green with or without albumin premixing for near-infrared fluorescence imaging of sentinel lymph nodes in breast cancer patients," Breast Cancer Res. Treat. 127(1), 163-170 (2011).
49. Q. T. Nguyen and R. Y. Tsien, "Fluorescence-guided surgery with live molecular navigation-a new cutting edge," Nat. Rev. Cancer 13(9), 653-662 (2013).
50. M. Jermyn, K. Kolste, J. Pichette, G. Sheehy, L. Angulo-Rodríguez, K. D. Paulsen, D. W. Roberts, B. C. Wilson, K. Petrecca, and F. Leblond, "Macroscopic-imaging technique for subsurface quantification of near-infrared markers during surgery," J. Biomed. Opt. 20(3), 036014 (2015).
51. P. A. Valdés, F. Leblond, V. L. Jacobs, B. C. Wilson, K. D. Paulsen, and D. W. Roberts, "Quantitative, spectrally-resolved intraoperative fluorescence imaging," Sci. Rep. 2, 798 (2012).
52. G. M. van Dam, G. Themelis, L. M. Crane, N. J. Harlaar, R. G. Pleijhuis, W. Kelder, A. Sarantopoulos, J. S. de Jong, H. J. Arts, A. G. van der Zee, J. Bart, P. S. Low, and V. Ntziachristos, "Intraoperative tumor-specific fluorescence imaging in ovarian cancer by folate receptor-α targeting: first in-human results," Nat. Med. 17(10), 1315-1319 (2011).
53. C. Chi, J. Ye, H. Ding, D. He, W. Huang, G. J. Zhang, and J. Tian, "Use of indocyanine green for detecting the sentinel lymph node in breast cancer patients: from preclinical evaluation to clinical validation," PLoS One 8(12), e83927 (2013).
54. B. Rogowitz, A. D. Kalvin, A. Pelah, and A. Cohen, "Which trajectories through which perceptually uniform color spaces produce appropriate colors scales for interval data?" in The Seventh Color Imaging Conference: Color Science, Systems, and Applications (Society for Imaging Science and Technology, 1999).
55. S. H. Keller, S. Holm, A. E. Hansen, B. Sattler, F. Andersen, T. L. Klausen, L. Højgaard, A. Kjær, and T. Beyer, "Image artifacts from MR-based attenuation correction in clinical, whole-body PET/MRI," MAGMA 26(1), 173-181 (2013).
56. D. W. Hosmer and S. Lemeshow, Applied Logistic Regression (Wiley-Intersceince, New York, 2000).
57. A. A. Boxwala, J. Kim, J. M. Grillo, and L. Ohno-Machado, "Using statistical and machine learning to help institutions detect suspicious access to electronic health records," J. Am. Med. Inform. Assoc. 18(4), 498-505 (2011).
58. M. Carpelan-Holmström, J. Louhimo, U. H. Stenman, H. Alfthan, H. Järvinen, and C. Haglund, "Estimating the probability of cancer with several tumor markers in patients with colorectal disease," Oncology 66(4), 296-302 (2004).
59. S. Gao, S. B. Mondal, N. Zhu, R. Liang, S. Achilefu, and V. Gruev, "Image overlay solution based on threshold detection for a compact near infrared fluorescence goggle system," J. Biomed. Opt. 20(1), 016018 (2015).
60. P. A. Valdes, V. L. Jacobs, B. C. Wilson, F. Leblond, D. W. Roberts, and K. D. Paulsen, "System and methods for wide-field quantitative fluorescence imaging during neurosurgery," Opt. Lett. 38(15), 2786-2788 (2013).
61. B. W. Pogue, S. Gibbs-Strauss, P. A. Valdés, K. Samkoe, D. W. Roberts, and K. D. Paulsen, "Review of Neurosurgical Fluorescence Imaging Methodologies," IEEE J. Sel. Top. Quantum Electron. 16(3), 493-505 (2010).
62. T. Nakajima, M. Mitsunaga, N. H. Bander, W. D. Heston, P. L. Choyke, and H. Kobayashi, "Targeted, activatable, in vivo fluorescence imaging of prostate-specific membrane antigen (PSMA) positive tumors using the quenched humanized J591 antibody-indocyanine green (ICG) conjugate," Bioconjug. Chem. 22(8), 1700-1705 (2011).
63. N. Thekkek, T. Muldoon, A. D. Polydorides, D. M. Maru, N. Harpaz, M. T. Harris, W. Hofstettor, S. P. Hiotis, S. A. Kim, A. J. Ky, S. Anandasabapathy, and R. Richards-Kortum, "Vital-dye enhanced fluorescence imaging of GI mucosa: metaplasia, neoplasia, inflammation," Gastrointest. Endosc. 75(4), 877-887 (2012).
64. K. J. Rosbach, M. D. Williams, A. M. Gillenwater, and R. R. Richards-Kortum, "Optical molecular imaging of multiple biomarkers of epithelial neoplasia: epidermal growth factor receptor expression and metabolic activity in oral mucosa," Transl. Oncol. 5(3), 160-171 (2012).
65. E. Reinhard, W. Heidrich, P. Debevec, S. Pattanaik, G. Ward, and K. Myszkowski, High dynamic range imaging: acquisition, display, and image-based lighting (Morgan Kaufmann, 2010).
66. S. Helge, H. Wolfgang, S. Wolfgang, W. Greg, W. Lorne, T. Matthew, G. Abhijeet, and V. Andrejs, "High dynamic range display systems," ACM Trans. Graph. 23(3), 760-768 (2004).
67. F. Drago, K. Myszkowski, T. Annen, and N. Chiba, "Adaptive Logarithmic Mapping For Displaying High Contrast Scenes," Comput. Graph. Forum 22(3), 419-426 (2003).
68. V. Dutt and J. F. Greenleaf, "Adaptive speckle reduction filter for log-compressed B-scan images," IEEE Trans. Med. Imaging 15(6), 802-813 (1996).
69. S. Keereweer, J. D. Kerrebijn, P. B. van Driel, B. Xie, E. L. Kaijzel, T. J. Snoeks, I. Que, M. Hutteman, J. R. van der Vorst, J. S. Mieog, A. L. Vahrmeijer, C. J. van de Velde, R. J. Baatenburg de Jong, and C. W. Löwik, "Optical image-guided surgery-where do we stand?" Mol. Imaging Biol. 13(2), 199-207 (2011).
70. A. Raabe, P. Nakaji, J. Beck, L. J. Kim, F. P. Hsu, J. D. Kamerman, V. Seifert, and R. F. Spetzler, "Prospective evaluation of surgical microscope-integrated intraoperative near-infrared indocyanine green videoangiography during aneurysm surgery," J. Neurosurg. 103(6), 982-989 (2005).
71. G. B. Hanna, S. M. Shimi, and A. Cuschieri, "Task performance in endoscopic surgery is influenced by location of the image display," Ann. Surg. 227(4), 481-484 (1998).
72. G. B. Hanna, S. M. Shimi, and A. Cuschieri, "Randomised study of influence of two-dimensional versus threedimensional imaging on performance of laparoscopic cholecystectomy," Lancet 351(9098), 248-251 (1998).
73. Y. Liu, Y. M. Zhao, W. Akers, Z. Y. Tang, J. Fan, H. C. Sun, Q. H. Ye, L. Wang, and S. Achilefu, "First inhuman intraoperative imaging of HCC using the fluorescence goggle system and transarterial delivery of nearinfrared fluorescent imaging agent: a pilot study," Transl. Res. 162(5), 324-331 (2013).
74. B. Funt, F. Ciurea, and J. McCann, "Retinex in Matlab," Eighth Color Imaging Conference: Color Science and Engineering Systems, Technologies, Applications, 112-121 (2000).
75. E. H. Land and J. J. McCann, "Lightness and retinex theory," J. Opt. Soc. Am. 61(1), 1-11 (1971).