Computer vision and augmented reality in gastrointestinal endoscopy

Gastroenterology Report

Volumn 3, Issue 3, 2015, Pages 179-184

  Mahmud, Nadim ^a   Cohen, Jonah ^c   Tsourides, Kleovoulos ^b   Berzin, Tyler M ^c  

^a BRIGHAM AND WOMEN S HOSPITAL   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c BETH ISRAEL DEACONESS MEDICAL CENTER   (United States)

Author keywords

Augmented reality; Colonoscopy; Computer vision; Gastrointestinal endoscopy

Indexed keywords

EID: 84973881052     PISSN: None     EISSN: 20520034     Source Type: Journal    
DOI: 10.1093/gastro/gov027     Document Type: Review

References (74)

1. Tang SL, Kwoh CK, Teo MY et al. Augmented reality systems for medical applications. IEEE Eng Med Biol Mag 1998;17:49–58.
2. Wright WG. Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds. Front Syst Neurosci 2014;8:56.
3. Graham M, Zook M, Boulton A. Augmented reality in urban places: contested content and the duplicity of code. Trans Inst Br Geogr 2013;38:464–79.
4. Chen BX. If you’re not seeing data, you’re not seeing. Wired Magazine 2009 August 25.
5. Maxwell K. Augmented Reality. Macmillan Dictionary Buzzword, 2010.
6. Azuma RT. A survey of augmented reality. Presence 1997;6: 355–85.
7. Vaughan-Nichols SJ. Augmented reality: No longer a novelty? Computer 2009;42:19–22.
8. Shuhaiber JH. Augmented reality in surgery. Arch Surg 2004; 139:170–4.
9. Kramers M, Armstrong R, Bakhshmand SM et al. Evaluation of a mobile augmented reality application for image guidance of neurosurgical interventions. Stud Health Technol Inform 2014; 196:204–8.
10. Mahvash M and Tabrizi LB. A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir (Wien) 2013;155:943–7.
11. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir (Wien) 2014;156: 1769–74.
12. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral aneurysms: a technical report. Neurosurgery 2014;10 Suppl 2:252–61.
13. Thoranaghatte RU, Giraldez JG, Zheng G. Landmark based augmented reality endoscope system for sinus and skull-base surgeries, In Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE, IEEE, 2008.
14. Thoranaghatte R, Garcia J, Caversaccio M et al. Landmark-based augmented reality system for paranasal and trans-nasal endoscopic surgeries. Int J Med Robot 2009;5:415–22.
15. Caversaccio M, Langlotz F, Nolte LP et al. Impact of a self-developed planning and self-constructed navigation system on skull base surgery: 10 years experience. Acta Otolaryngol 2007;127:403–7.
16. Okamoto T, Onda S, Yanaga K et al. Clinical application of navigation surgery using augmented reality in the abdominal field. Surg Today 2015;45:397–406.
17. López-Mir F, Naranjo V, Fuertes J et al. Design and validation of an augmented reality system for laparoscopic surgery in a real environment. Biomed Res Int 2013;2013:758491.
18. Kang X, Azizian M, Wilson E et al. Stereoscopic augmented reality for laparoscopic surgery. Surg Endosc 2014;28:2227–35.
19. TM robotic biliary surgery. Int J Surg Case Rep 2013;4:365–7.
20. Simpfendörfer T, Baumhauer M, Müller M et al. Augmented reality visualization during laparoscopic radical prostatec-tomy. J Endourol 2011;25:1841–5.
21. Peterhans M, vom Berg A, Dagon B et al. A navigation system for open liver surgery: design, workflow and first clinical applications. Int J Med Robot 2011;7:7–16.
22. Marzano E, Piardi T, Soler L et al. Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 2013;17:1980–3.
23. Kranzfelder M, Wilhelm D, Doundoulakis M et al. A probe-based electromagnetic navigation system to integrate computed tomography during upper gastrointestinal endoscopy. Endoscopy 2014;46:302–5.
24. Azagury D, Ryou M, Shaikh S et al. Real-time computed tomography-based augmented reality for natural orifice transluminal endoscopic surgery navigation. Br J Surg 2012; 99:1246–53.
25. Vosburgh KG and San José Estépar R. Natural Orifice Transluminal Endoscopic Surgery (NOTES): an opportunity for augmented reality guidance. Stud Health Technol Inform 2006;125:485–90.
26. Marr D. Vision: A computational investigation into the human representation and processing of visual information. New York, NY: Henry Holt and Co. Inc., 1982:2–46.
27. Rosenfeld A and Avinash C. KAK: Digital Picture Processing. Academic Press, 1982.
28. Barghout L and Lee L. Perceptual information processing system. Google Patents, 2003.
29. Horn B. Robot vision. MIT press, 1986.
30. Faugeras O. Three-dimensional computer vision: a geometric viewpoint. MIT press, 1993.
31. Berryman DR. Augmented reality: a review. Med Ref Serv Q 2012;31:212–18.
32. Schrock TR. Colonoscopy versus barium enema in the diagnosis of colorectal cancers and polyps. Philadelphia, PA: WB Saunders Co, 1995:513–38.
33. Seeff LC, Richards TB, Shapiro JA et al. How many endoscopies are performed for colorectal cancer screening? Results from CDC’s survey of endoscopic capacity. Gastroenterology 2004; 127:1670–7.
34. Ahn SB, Han DS, Bae JH et al. The miss rate for colorectal adenoma determined by quality-adjusted, back-to-back colonoscopies. Gut Liver 2012;6:64–70.
35. Rex DK, Cutler CS, Lemmel GT et al. Colonoscopic miss rates of adenomas determined by back-to-back colonoscopies. Gastroenterology 1997;112:24–8.
36. Postic G, Lewin D, Bickerstaff C et al. Colonoscopic miss rates determined by direct comparison of colonoscopy with colon resection specimens. Am J Gastroenterol 2002;97:3182–5.
37. Heresbach D, Barrioz T, Lapalus M et al. Miss rate for colorectal neoplastic polyps: a prospective multicenter study of back-to-back video colonoscopies. Endoscopy 2008;40:284–90.
38. Leufkens A, van Oijen M, Vleggaar F et al. Factors influencing the miss rate of polyps in a back-to-back colonoscopy study. Endoscopy 2012;44:470–5.
39. Hurlstone DP, Cross SS, Adam I et al. A prospective clinico-pathological and endoscopic evaluation of flat and depressed colorectal lesions in the United Kingdom. Am J Gastroenterol 2003;98:2543–9.
40. Parra-Blanco A, Gimeno-García AZ, Nicolás-Pérez D et al. Risk for high-grade dysplasia or invasive carcinoma in colorectal flat adenomas in a Spanish population. Gastroenterol Hepatol 2006;29:602–9.
41. Xiang L, Zhan Q, Zhao XH et al. Risk factors associated with missed colorectal flat adenoma: a multicenter retrospective tandem colonoscopy study. World J Gastroenterol 2014;20: 10927–37.
42. Choi HN, Kim HH, Oh JS et al. Factors influencing the miss rate of polyps in a tandem colonoscopy study. Korean J Gastroenterol 2014;64:24–30.
43. Aslanian HR, Shieh FK, Chan FW et al. Nurse observation during colonoscopy increases polyp detection: a randomized prospective study. Am J Gastroenterol 2013;108:166–72.
44. Lee CK, Park DI, Lee SH et al. Participation by experienced endoscopy nurses increases the detection rate of colon polyps during a screening colonoscopy: a multicenter, prospective, randomized study. Gastrointest Endosc 2011;74: 1094–102.
45. Buchner AM, Shahid MW, Heckman MG et al. Trainee participation is associated with increased small adenoma detection. Gastrointest Endosc 2011;73:1223–31.
46. Zauber AG, Winawer SJ, O’Brien MJ et al. Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths. N Engl J Med 2012;366:687–96.
47. Jacob BJ, Moineddin R, Sutradhar R et al. Effect of colonoscopy on colorectal cancer incidence and mortality: an instrumental variable analysis. Gastrointest Endosc 2012;76:355–64.e1.
48. Siersema PD, Rastogi A, Leufkens AM et al. Retrograde-viewing device improves adenoma detection rate in colonoscopies for surveillance and diagnostic workup. World J Gastroenterol 2012;18:3400–8.
49. Gralnek IM, Segol O, Suissa A et al. A prospective cohort study evaluating a novel colonoscopy platform featuring full-spectrum endoscopy. Endoscopy 2013;45:697–702.
50. Kiesslich R, Fritsch J, Holtmann M et al. Methylene blue-aided chromoendoscopy for the detection of intraepithelial neoplasia and colon cancer in ulcerative colitis. Gastroenterology 2003;124:880–8.
51. Hurlstone DP, Karajeh M, Cross SS et al. The role of high-magnification-chromoscopic colonoscopy in hereditary nonpolyposis colorectal cancer screening: a prospective “back-to-back” endoscopic study. Am J Gastroenterol 2005;100: 2167–73.
52. Longcroft-Wheaton G, Brown J, Cowlishaw D et al. High-definition vs. standard-definition endoscopy with indigo carmine for the in vivo diagnosis of colonic polyps. United European Gastroenterol J 2013;1:425–9.
53. Uraoka T, Saito Y, Matsuda T et al. Detectability of colorectal neoplastic lesions using a narrow-band imaging system: A pilot study. J Gastroenterol Hepatol 2008;23:1810–15.
54. Wallace MB. Improving colorectal adenoma detection: technology or technique? Gastroenterology 2007;132:1221–3.
55. Yotsumoto Y and Sekuler R. Out of mind, but not out of sight: Intentional control of visual memory. Mem Cogn 2006;34: 776–86.
56. Davila RE, Rajan E, Baron TH. ASGE guideline: colorectal cancer screening and surveillance. Gastrointest Endosc 2006;63: 546–57.
57. Huang CS, O’Brien MJ, Yang S et al. Hyperplastic polyps, serrated adenomas, and the serrated polyp neoplasia pathway. Am J Gastroenterol 2004;99:2242–55.
58. Rastogi A. Optical diagnosis of small colorectal polyp histology with high-definition colonoscopy using narrow band imaging. Clin Endosc 2013;46:120–9.
59. Takemura Y, Yoshida S, Tanaka S et al. Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video). Gastrointest Endosc 2012;75:179–85.
60. Mori Y, Kudo SE, Wakamura K et al. Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointest Endosc 2015;81:621–9.
61. Hummel J, Figl M, Bax M et al. 2D/3D registration of endoscopic ultrasound to CT volume data. Phys Med Biol 2008;53: 4303–16.
62. Chen Y, Kwok KW, Ge J et al. Augmented reality for improving catheterization in magnetic resonance imaging-guided cardiac electrophysiology therapy. J Med Dev 2014;8:020917.
63. Kappelman MD, Dorn SD, Peterson E et al. Quality of care for gastrointestinal conditions: a primer for gastroenterologists. Am J Gastroenterol 2011;106:1182–7.
64. Rex DK. Optimal withdrawal and examination in colonoscopy. Gastroenterol Clin North Am 2013;42:429–42.
65. Chaptini L, Chaaya A, Depalma F et al. Variation in polyp size estimation among endoscopists and impact on surveillance intervals. Gastrointest Endosc 2014;80:652–9.
66. Eichenseer PJ, Dhanekula R, Jakate S et al. Endoscopic mis-siz-ing of polyps changes colorectal cancer surveillance recommendations. Dis Colon Rectum 2013;56:315–21.
67. Kume K, Watanabe T, Yoshikawa I et al. Endoscopic measurement of polyp size using a novel calibrated hood. Gastroenterol Res Pract 2014;2014:714294.
68. Ferrante M, Colombel JF, Sandborn WJ et al. Validation of endoscopic activity scores in patients with Crohn’s disease based on a post hoc analysis of data from SONIC. Gastroenterology 2013;145:978–86.e5.
69. Ahmad H, Berzin TM, Yu HJ, et al. Central endoscopy reads in inflammatory bowel disease clinical trials: The role of the imaging core lab. Gastroenterol Rep (Oxf) 2014;2:201–6.
70. Calderwood AH, Schroy PC, 3rd, Lieberman DA et al. Boston Bowel Preparation Scale scores provide a standardized definition of adequate for describing bowel cleanliness. Gastrointest Endosc 2014;80:269–76.
71. Kim EJ, Park YI, Kim YS et al. A Korean experience of the use of Boston bowel preparation scale: a valid and reliable instrument for colonoscopy-oriented research. Saudi J Gastroenterol 2014;20:219–24.
72. Keller J, Fibbe C, Rosien U et al. Recent advances in capsule endoscopy: development of maneuverable capsules. Expert Rev Gastroenterol Hepatol 2012;6:561–6.
73. Swain P. The future of wireless capsule endoscopy. World J Gastroenterol 2008;14:4142–5.
74. Taigman Y, Yang M, Ranzato MA et al. Deepface: Closing the gap to human-level performance in face verification, In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, IEEE, 2014.