Interactive CT-video registration for the continuous guidance of bronchoscopy

IEEE Transactions on Medical Imaging

Volumn 32, Issue 8, 2013, Pages 1376-1396

  Merritt, Scott A ^a   Khare, Rahul ^b   Bascom, Rebecca ^c   Higgins, William E ^d  

^a X Nav Technologies LLC   (United States)

^b CHILDREN'S NATIONAL MEDICAL CENTER   (United States)

^c PENN STATE MILTON S HERSHEY MEDICAL CENTER   (United States)

^d PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

Bronchoscopy; endoscopy; image alignment; image based rendering; lung; lung cancer; registration; surgical guidance navigation; virtual bronchoscopy; virtual endoscopy; virtual augmented reality; X ray imaging and computed tomography

Indexed keywords

BRONCHOSCOPY; IMAGE ALIGNMENT; IMAGE-BASED RENDERING; LUNG; LUNG CANCER; REGISTRATION; SURGICAL GUIDANCE/NAVIGATION; VIRTUAL BRONCHOSCOPY; VIRTUAL ENDOSCOPY; VIRTUAL/AUGMENTED REALITY; XRAY IMAGING;

ALIGNMENT; BIOLOGICAL ORGANS; COMPUTER VISION; COMPUTERIZED TOMOGRAPHY; DISEASES; IMAGE RECONSTRUCTION;

ENDOSCOPY;

ALGORITHM; ARTICLE; BRONCHOSCOPE; BRONCHOSCOPY; CLINICAL ARTICLE; COMPUTER; COMPUTER ASSISTED TOMOGRAPHY; HUMAN; LABORATORY TEST; LUNG CANCER; PHANTOM; PHYSICIAN; THORAX; VIDEORECORDING;

ADOLESCENT; ADULT; AGED; AGED, 80 AND OVER; ALGORITHMS; BRONCHOSCOPY; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LUNG; LUNG NEOPLASMS; MALE; MIDDLE AGED; SURGERY, COMPUTER-ASSISTED; THORACIC SURGERY, VIDEO-ASSISTED; TOMOGRAPHY, X-RAY COMPUTED;

EID: 84881443818     PISSN: 02780062     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMI.2013.2252361     Document Type: Article

References (71)

1. R. Siegel, D. Naishadham, and A. Jemal, "Cancer statistics 2012," CA Cancer J. Clin., vol. 62, no. 1, pp. 10-29, Jan. 2012.
2. A. D. Sihoe and A. P. Yim, "Lung cancer staging," J. Surg. Res., vol. 117, no. 1, pp. 92-106, Mar. 2004. (Pubitemid 38317214)
3. K. P. Wang and A. C. Mehta, Flexible Bronchoscopy, J. F. Turner, Ed., 2nd ed. Cambridge, MA: Blackwell, 2003.
4. J. Ueno, T. Murase, K. Yoneda, T. Tsujikawa, S. Sakiyama, and K. Kondoh, "Three-dimensional imaging of thoracic diseases with multi-detector row CT," J. Med. Invest., vol. 51, no. 3-4, pp. 163-170, Aug. 2004. (Pubitemid 39610910)
5. M. Y. Dolina, D. C. Cornish, S. A. Merritt, L. Rai, R. Mahraj, W. E. Higgins, and R. Bascom, "Interbronchoscopist variability in endo-bronchial path selection: A simulation study," Chest, vol. 133, no. 4, pp. 897-905, Apr. 2008. (Pubitemid 351536864)
6. F. Asano, "Virtual bronchoscopic navigation," Clin. Chest Med., vol. 31, no. 1, pp. 75-85, Mar. 2010.
7. H. Minami, Y. Ando, F. Nomura, S. Sakai, and K. Shimokata, "Inter-bronchoscopist variability in the diagnosis of lung cancer by flexible bronchoscopy," Chest, vol. 105, no. 2, pp. 1658-1662, Jun. 1994. (Pubitemid 24180646)
8. J. P. Helferty, A. J. Sherbondy, A. P. Kiraly, and W. E. Higgins, "Computer-based system for the virtual-endoscopic guidance of bronchoscopy," Comput. Vis. Image Understand., vol. 108, no. 1-2, pp. 171-187, Oct.-Nov. 2007. (Pubitemid 47333384)
9. S. A. Merritt, J. D. Gibbs, K. C. Yu, V. Patel, L. Rai, D. C. Cornish, R. Bascom, and W. E. Higgins, "Real-time image-guided bronchoscopy for peripheral lung lesions: A phantom study," Chest, vol. 134, no. 5, pp. 1017-1026, Nov. 2008.
10. K. Tsushima, S. Sone, T. Hanaoka, F. Takayama, T. Honda, and K. Kubo, "Comparison of bronchoscopic diagnosis for peripheral pulmonary nodule under fluoroscopic guidance with CT guidance," Respir. Med., vol. 100, no. 4, pp. 737-745, Apr. 2006. (Pubitemid 43285763)
11. K. Yasufuku et al., "Endobronchial ultrasound guided transbronchial needle aspiration for staging of lung cancer," Lung Cancer, vol. 50, no. 3, pp. 347-354, Dec. 2005.
12. P. R. Geraghty, S. T. Kee, G. McFarlane, M. K. Razavi, D. Y. Sze, and M. D. Dake, "CT-guided transthoracic needle aspiration biopsy of pulmonary nodules: Needle size and pneumothorax rate," Radiology, vol. 229, no. 2, pp. 475-481, Nov. 2003. (Pubitemid 37310835)
13. D. J. Vining, K. Liu, R. H. Choplin, and E. F. Haponik, "Virtual bron-choscopy: Relationships of virtual reality endobronchial simulations to actual bronchoscopic findings," Chest, vol. 109, no. 2, pp. 549-553, Feb. 1996. (Pubitemid 26058617)
14. R. M. Summers, "Navigational aids for real-time virtual bron-choscopy," Am. J. Roentgen., vol. 168, no. 5, pp. 1165-1170, May 1997. (Pubitemid 27183787)
15. U. Rapp-Bernhardt, T. Welte, M. Budinger, and T. M. Bernhardt, "Comparison of three-dimensional virtual endoscopy with bron-choscopy in patients with oesophageal carcinoma infiltrating the tracheobronchial tree," Br. J. Radiol., vol. 71, pp. 1271-1278, Dec. 1998.
16. W. E. Higgins, K. Ramaswamy, R. Swift, G. McLennan, and E. A. Hoffman, "Virtual bronchoscopy for 3D pulmonary image assessment: State of the art and future needs," Radiographics, vol. 18, no. 3, pp. 761-778, May-Jun. 1998. (Pubitemid 128467058)
17. H. P. McAdams, P. C. Goodman, and P. Kussin, "Virtual bronchoscopy for directing transbronchial needle aspiration of hilar and mediastinal lymph nodes: A pilot study," Am. J. Roentgenol., vol. 170, no. 5, pp. 1361-1364, May 1998. (Pubitemid 28210959)
18. E. F. Haponik, S. L. Aquino, and D. J. Vining, "Virtual bronchoscopy," Clin. Chest Med., vol. 20, no. 1, pp. 201-217, Mar. 1999. (Pubitemid 29161860)
19. K. Hopper, T. Lucas, K. Gleeson, J. Stauffer, R. Bascom, D. Mauger, and R. Mahraj, "Transbronchial biopsy with virtual CT bronchoscopy and nodal highlighting," Radiology, vol. 221, no. 2, pp. 531-536, Nov. 2001. (Pubitemid 33043441)
20. N. Shinagawa, K. Yamazaki, Y. Onodera, K. Miyasaka, E. Kikuchi, H. Dosaka-Akita, and M. Nishimura, "CT-guided transbronchial biopsy using an ultrathin bronchoscope with virtual bronchoscopic navigation," Chest, vol. 125, no. 3, pp. 1138-1143, Mar. 2004. (Pubitemid 38367311)
21. G. M. Weiner, K. Schulze, B. Geiger, H. Ebhardt, K. Wolf, and T. Albrecht, "CT bronchoscopic simulation for guiding transbronchial needle aspiration of extramural mediastinal and hilar lesions: Initial clinical results," Radiology, vol. 250, no. 3, pp. 923-931, Mar. 2009.
22. G. D. Rubin, C. F. Beaulieu, V. Argiro, H. Ringl, A. M. Norbash, J. F. Feller, M. D. Dake, S. Napel, R. B. Jeffrey, and S. Napel, "Perspective volume rendering of CT and MR images: Applications for endoscopic imaging," Radiology, vol. 199, no. 2, pp. 321-330, May 1996. (Pubitemid 26121974)
23. P. Rogalla, J. Van Scheltinga, and B. Hamm, Virtual Endoscopy and Related 3D Techniques. Berlin, Germany: Springer-Verlag, 2002.
24. F. Deligianni, A. J. Chung, and G. Yang, "Nonrigid 2-D/3-D registration for patient specific bronchoscopy simulation with statistical shape modeling: Phantom validation," IEEE Trans. Med. Imag., vol. 25, no. 11, pp. 1462-1471, Nov. 2006. (Pubitemid 44711694)
25. S. B. Solomon, P. White Jr., C. M. Wiener, J. B. Orens, and K. P. Wang, "Three-dimensionsal CT-guided bronchoscopy with a real-time electromagnetic position sensor: A comparison of two image registration methods," Chest, vol. 118, no. 6, pp. 1783-1787, Dec. 2000. (Pubitemid 32001810)
26. T. R. Gildea, P. J. Mazzone, D. Karnak, M. Meziane, and A. C. Mehta, "Electromagnetic navigation diagnostic bronchoscopy: A prospective study," Am. J. Resp. Crit. Care Med., vol. 174, no. 9, pp. 982-989, Nov. 2006. (Pubitemid 44684446)
27. Y. Schwarz, J. Greif, H. D. Becker, A. Ernst, and A. Mehta, "Real-time electromagnetic navigation bronchoscopy to peripheral lung lesions using overlaid CT images: The first human study," Chest, vol. 129, no. 4, pp. 988-994, Apr. 2006.
28. R. Eberhardt, D. Anantham, A. Ernst, D. Feller-Kopman, and F. Herth, "Multimodality bronchoscopic diagnosis of peripheral lung lesions: A randomized controlled trial," Am. J. Respir. Crit. Care Med., vol. 176, no. 1, pp. 36-41, Jul. 2007. (Pubitemid 47048052)
29. I. Gergel, T. R. dos Santos, R. Tetzlaff, L. Maier-Hein, H.-P. Meinzer, and I. Wegner, "Particle filtering for respiratory motion compensation during navigated bronchoscopy," in SPIE Medical Imaging 2010: Visualization, Image-Guided Procedures, and Modeling, K. H. Wong and M. I. Miga, Eds., 2010, vol. 7625, pp. 76250W1-76250W8.
30. X. Luo, M. Feuerstein, T. Sugiura, T. Kitasaka, K. Imaizumi, Y. Hasegawa, and K. Mori, "Towards hybrid bronchoscope tracking under respiratory motion-Evaluation on a dynamic motion phantom," in SPIE Medical Imaging 2010: Visualization, Image-Guided Procedures, and Modeling, K. H. Wong and M. I. Miga, Eds., 2010, vol. 7625, pp. 76251B1-76251B12.
31. T. D. Soper, D. R. Haynor, R. W. Glenny, and E. J. Seibel, "In vivo validation of a hybrid tracking system for navigation of an ultrathin bronchoscope within peripheral airways," IEEE Trans. Biomed. Eng., vol. 57, no. 3, pp. 736-745, Mar. 2010.
32. W. E. Grimson, G. J. Ettinger, S. J. White, T. Lozano-Perez, W. E. Wells, III, and R. Kikinis, "An automatic registration method for frame-less stereotaxy, image guided surgery, and enhanced reality visualization," IEEE Trans. Med. Imag., vol. 15, no. 2, pp. 129-140, Apr. 1996. (Pubitemid 126780306)
33. C. R. Maurer Jr., J. M. Fitzpatrick, M. Y. Wang, R. L. Galloway Jr., R. J. Maciunas, and G. S. Allen, "Registration of head volume images using implantable fiducial markers," IEEE Trans. Med. Imag., vol. 16, no. 4, pp. 447-462, Aug. 1997. (Pubitemid 127770429)
34. J. D. Stefansic, A. J. Herline, Y. Shyr, W. C. Chapman, J. M. Fitzpatrick, B. M. Dawant, and R. L. Galloway, Jr., "Registration of physical space to laparoscopic image space for use in minimally invasive hepatic surgery," IEEE Trans. Med. Imag., vol. 19, no. 10, pp. 1012-1023, Oct. 2000. (Pubitemid 32075869)
35. I. Bricault, G. Ferretti, and P. Cinquin, "Registration of real and CT-de-rived virtual bronchoscopic images to assist transbronchial biopsy," IEEE Trans. Med.Imag., vol. 17, no. 5, pp. 703-714, Oct. 1998. (Pubitemid 128747726)
36. J. P. Helferty, C. Zhang, G. McLennan, and W. E. Higgins, "Videoen-doscopic distortion correction and its application to virtual guidance of endoscopy," IEEE Trans. Med. Imag., vol. 20, no. 7, pp. 605-617, Jul. 2001. (Pubitemid 32697979)
37. W. E. Higgins, J. P. Helferty, K. Lu, S. A. Merritt, L. Rai, and K. C. Yu, "3D CT-video fusion for image-guided bronchoscopy," Comput. Med. Imag. Graph., vol. 32, no. 3, pp. 159-173, Apr. 2008.
38. K. Mori, D. Deguchi, J. Sugiyama, Y. Suenaga, J. Toriwaki, C. R. Maurer, H. Takabatake, and H. Natori, "Tracking of bronchoscope using epipolar geometry analysis and intensity-based image registration of real and virtual endoscopic images," Med. Image Anal., vol. 6, no. 3, pp. 321-336, Sep. 2002. (Pubitemid 35282837)
39. D. Deguchi, K. Mori, M. Feuerstein, T. Kitasaka, C. R. Maurer, Y. Sue-naga, H. Takabatake, M. Mori, and H. Natori, "Selective image similarity measure for bronchoscope tracking based on image registration," Med. Image Anal., vol. 13, no. 1, pp. 621-633, Jul. 2009.
40. X. Luo, M. Feuerstein, T. Reichl, T. Kitasaka, and K. Mori, P. J. Edwards, H. Liao, and X. Pan, Eds., "An application driven comparison of several feature extraction algorithms in bronchoscope tracking during navigated bronchoscopy," in 5th Int. Workshop Med. Imag. Augmented Reality, 2010, vol. LNCS 6326, pp. 475-484.
41. L. Rai, J. P. Helferty, and W. E. Higgins, "Combined video tracking and image-video registration for continuous bronchoscopic guidance," Int. J. Comput. Assist. Radiol. Surg., vol. 3, no. 3-4, pp. 315-329, Sep. 2008.
42. L. Rai, S. A. Merritt, and W. E. Higgins, "Real-time image-based guidance method for lung-cancer assessment," in IEEE Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2437-2444. (Pubitemid 44931615)
43. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
44. R. Szeliski, Computer Vision-Algorithms and Applications. New York, NY: Springer, 2011.
45. J. D. Gibbs, M. W. Graham, and W. E. Higgins, "3D MDCT-based system for planning peripheral bronchoscopic procedures," Comput. Biol. Med., vol. 39, no. 3, pp. 266-279, Mar. 2009.
46. M. W. Graham, J. D. Gibbs, D. C. Cornish, and W. E. Higgins, "Robust 3D airway-tree segmentation for image-guided Peripheral Bron-choscopy," IEEE Trans. Med. Imag., vol. 29, no. 4, pp. 982-997, Apr. 2010.
47. A. P. Kiraly, E. A. Hoffman, G. McLennan, W. E. Higgins, and J. M. Reinhardt, "3D human airway segmentation methods for clinical virtual bronchoscopy," Acad. Radiol., vol. 9, no. 10, pp. 1153-1168, Oct. 2002.
48. A. P. Kiraly, J. P. Helferty, E. A. Hoffman, G. McLennan, and W. E. Higgins, "3D path planning for virtual bronchoscopy," IEEE Trans. Med. Imag., vol. 23, no. 11, pp. 1365-1379, Nov. 2004.
49. K. C. Yu, J. D. Gibbs, M. W. Graham, and W. E. Higgins, "Image-based reporting for bronchoscopy," J. Digital Imag., vol. 23, no. 1, pp. 39-50, Feb. 2010.
50. M. W. Graham, J. D. Gibbs, and W. E. Higgins, "Computer-based route-definition system for peripheral bronchoscopy," J. Digital Imag., vol. 25, no. 2, pp. 307-317, Apr. 2012.
51. J. D. Gibbs, M. W. Graham, R. Bascom, D. C. Cornish, R. Khare, and W. E. Higgins, "Computer-based planning and guidance system for image-guided bronchoscopy," IEEE Trans. Biomed. Eng., submitted for publication.
52. B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artif. Intell., 1981, pp. 674-679. (Pubitemid 14478744)
53. G. D. Hager and P. N. Belhumeur, "Efficient region tracking with parametric models of geometry and illumination," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 10, pp. 1025-1039, Oct. 1998. (Pubitemid 128741363)
54. H.-Y. Shum and R. Szeliski, "Construction of panoramic image mosaics with global and local alignment," Int. J. Comput. Vis., vol. 16, no. 1, pp. 63-84, Feb. 2000.
55. S. Baker and I. Matthews, "Equivalence and efficiency of image alignment algorithms," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2001, pp. 1090-1097.
56. S. Baker and I. Matthews, "Lucas-Kanade 20 years on: A unifying framework," Int. J. Comput. Vis., vol. 56, no. 3, pp. 221-255, Feb. 2004.
57. G. D. Evangelidis and E. Z. Psarakis, "Parametric image alignment using enhanced correlation coefficient maximization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 10, pp. 1858-1865, Oct. 2008.
58. E. Izquierdo and J.-R. Ohm, "Image-based rendering and 3D modeling: A complete framework," Signal Process.: Image Commun., vol. 15, no. 10, pp. 817-858, Aug. 2000.
59. E. Trucco and A. Verri, Introductory Techniques for 3-D Computer Vision. Upper Saddle River, NJ: Prentice-Hall, 1998.
60. Y. Ma, J. Kosecka, S. Soatto, and S. Sastry, An Invitation to 3-D Vi-sion-From Images to Models. New York: Springer, 2001.
61. T. Okatani and K. Deguchi, "Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center," Comput. Vis. Image Understand., vol. 66, no. 2, pp. 119-131, May 1997. (Pubitemid 127435276)
62. S. Baker and I. Matthews, Lucas-Kanade 20 years on: Part 5 Carnegie Mellon Univ., Tech. Rep. CMU-RI-TR-04-64, 2004, Technical Report CMU-RI-TR-04-64.
63. S. Romdhani and T. Vetter, "Efficient, robust and accurate fitting of a 3D morphable model," in Proc. 9th IEEE Int. Conf. Comput. Vis., 2003, vol. 1, pp. 59-66.
64. S. Baker and T. Kanade, "Limits on super-resolution and how to break them," in IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 15, 2000, vol. 2, pp. 372-379.
65. B. Flood, L. Rai, and W. E. Higgins, "System for robust bronchoscopic video distortion correction," in SPIE Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, K. H. Wong and D. R. Holmes, Eds., 2011, vol. 7964, pp. 79641L-1-79641L-16.
66. D. Hearn and M. P. Baker, Computer Graphics with OpenGL, 3rd ed. Upper Saddle River, NJ: Pearson Prentice Hall, 2004.
67. S. A. Merritt, L. Rai, and W. E. Higgins, "Real-time CT-video registration for continuous endoscopic guidance," in SPIE Medical Imaging 2006: Physiology, Function, and Structure from Medical Images, A. Manduca and A. A. Amini, Eds., 2006, pp. 370-384.
68. W. E. Higgins, R. Bascom, M. W. Graham, J. D. Gibbs, D. C. Cornish, M. Khan, and R. Khare, "Image-guided bronchoscopic sampling of peripheral lesions: A human study," Am. J. Respirat. Crit. Care Med., vol. 181, pp. A5171-A5171, May 2010.
69. N. Shinagawa et al., "Virtual bronchoscopic navigation system shortens the examination time: Feasibility study of virtual broncho-scopic navigation system," Lung Cancer, vol. 56, no. 2, pp. 201-206, May 2007. (Pubitemid 46590483)
70. R. M. Summers, A. Hara, W. Luboldt, and C. Johnson, "Computed tomographic and magnetic resonance colonography: Sum summary of progress from 1995 to 2000," Current Problems Diagnostic Radiol., vol. 30, no. 5, pp. 141-168, Sep.-Oct. 2001.
71. J. Liu, K. R. Subramanian, and T. S. Yoo, "Region flow: A multi-stage method for colonoscopy tracking," in Proc. 13th Int. Conf. Med. Image Comput. Comput.-Assist. Intervent.: Part II, 2010, pp. 505-513.