Stereoscopic visualization and 3-D technologies in medical endoscopic teleoperation

IEEE Transactions on Industrial Electronics

Volumn 62, Issue 1, 2015, Pages 525-535

  Livatino, Salvatore ^a   De Paolis, Lucio T ^b   D'Agostino, Michele ^c   Zocco, Alessandro ^b,d   Agrimi, Alessio ^e   De Santis, Alberto ^f   Bruno, Luisa Vittoria ^b   Lapresa, Michele ^g  

^a UNIVERSITY OF HERTFORDSHIRE   (United Kingdom)

^b UNIVERSITY OF SALENTO   (Italy)

^c Optulink Inc   (United States)

^d Elettronica SpA   (Italy)

^e Capgemini   (Italy)

^f Logispin Limited   (Gibraltar)

^g Softkinetic   (Belgium)

Author keywords

3 D displays.; Endoscopic intervention; stereo vision; stereoscopic 3 D visualization (S3D); teleoperation; virtual reality(VR)

Indexed keywords

CATHODES; CRYSTAL FILTERS; DECISION MAKING; ENDOSCOPY; HELMET MOUNTED DISPLAYS; LIQUID CRYSTAL DISPLAYS; REMOTE CONTROL; STEREO IMAGE PROCESSING; STEREO VISION; THREE DIMENSIONAL COMPUTER GRAPHICS; VIRTUAL REALITY; VISUALIZATION;

3-D DISPLAYS; 3D VISUALIZATION; DIGITAL LIGHT PROCESSING; DISPLAY TECHNOLOGIES; ENDOSCOPIC INTERVENTION; FLEXIBLE ENDOSCOPES; HEAD MOUNTED DISPLAYS; STEREOSCOPIC VISUALIZATION;

LIGHT EMITTING DIODES;

EID: 84919941356     PISSN: 02780046     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIE.2014.2334675     Document Type: Article

References (61)

1. P. Gomes, "Surgical robotics: Reviewing the past, analysing the present, imagining the future," Robot. Comput.-Integr. Manuf., vol. 27, no. 2, pp. 261-266, Apr. 2011.
2. da Vinci Surgical System 2013. [Online]. Available: http://www. intuitivesurgical.com/products/davinci-surgical-system
3. H. Tanaka et al., "Implementation of bilateral control system based on acceleration control using FPGA for multi-DOF haptic endoscopic surgery robot," IEEE Trans. Ind. Electron., vol. 56, no. 3, pp. 618-627, Mar. 2009.
4. W. Motooka et al., "Development of 16-DOF telesurgical forceps master/slave robot with haptics," in Proc. IEEE IECON, 2010, pp. 2081-2086.
5. G. D. Hager et al., "Surgical and interventional robotics: Part III surgical assistance systems," IEEE Robot. Autom. Mag., vol. 15, no. 4, pp. 2081-2086, Dec. 2008.
6. P. Charland and T. Peters, "Optimal display conditions for quantitative analysis of stereoscopic cerebral angiograms," IEEE Trans. Med. Imag., vol. 15, no. 5, pp. 648-656, Oct. 1996.
7. A. S. Talukdar and D. L. Wilson, "Modeling and optimization of rotational C-Arm stereoscopic X-ray angiography," IEEE Trans. Med. Imag., vol. 18, no. 7, pp. 604-616, Jul. 1999.
8. D. Dey, D. G. Gobbi, P. J. Slomka, K. J. M. Surry, and T. M. Peters, "Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: Creating stereoscopic panoramas," IEEE Trans. Med. Imag., vol. 21, no. 1, pp. 23-30, Jan. 2002.
9. P. van Bergen, W. Kunert, J. Bessell, and G. F. Buess, "Comparative study of two-dimensional and three-dimensional vision systems for minimally invasive surgery," Surg. Endoscopy, vol. 12, no. 7, pp. 948-954, Jul. 1998.
10. I. C. Jourdan et al., "Stereoscopic vision provides a significant advantage for precision robotic laparoscopy," British J. Surg., vol. 91, no. 7, pp. 879-885, Jul. 2004.
11. S. B. Bhayani and G. L. Andriole, "Three-Dimensional (3D) Vision: Does it improve laparoscopic skills? An assessment of a 3D head-mounted visualization system," Rev. Urology, vol. 7, no. 4, pp. 211-214, 2005.
12. J. G. Bittner, C. A. Hathaway, and J. A. Brown, "Three-dimensional visualisation and articulating instrumentation: Impact on simulated laparoscopic tasks," J. Min. Access Surg., vol. 4, no. 2, pp. 31-38, Apr.-Jun. 2008.
13. S. K. Maithel, L. Villegas, N. Stylopoulos, S. Dawson, and D. B. Jones, "Simulated laparoscopy using a head-mounted display vs traditional video monitor: An assessment of performance and muscle fatigue," Surg. Endoscopy, vol. 19, no. 3, pp. 406-411, Mar. 2005.
14. J. C. Byrn et al., "Three-dimensional imaging improves surgical performance for both novice and experienced operators using the da Vinci Robot System," Amer. J. Surg., vol. 193, no. 4, pp. 519-522, Apr. 2007.
15. J. Haase, "How to develop the surgical dexterity needed for endoscope neurosurgery," Pan Arab. J. Neurosurg., vol. 13, no. 2, pp. 1-8, 2009.
16. S. Livatino, G. Muscato, S. Sessa, and V. Neri, "Depth-enhanced mobile robot teleguide based on laser images," Mechatronics, vol. 20, no. 7, pp. 739-750, Oct. 2010.
17. S. Livatino, G. Muscato, D. De Tommaso, and M. Macaluso, "Augmented reality stereoscopic visualization for intuitive robot teleguide," in Proc. IEEE ISIE, 2010, pp. 2828-2833.
18. University of Meryland Medical Center Encycolpedia, 2012. [Online]. Available: http://www.umm.edu/imagepages/9138.htm
19. Falling Pixel Inc, Pulmonary System 3D Model 2013. [Online]\ble: https://www.fallingpixel.com/3d-models/5105
20. D. Feller-Kopman and J. A. Gorden, Endobronchial Ultrasound, A. Ernst and F. J. F. Herth, Eds. New York, NY, USA: Springer-Verlag, 2009, pp. 1-24.
21. D. Feller-Kopman, "Bronchial Tree: Segmental Anatomy from an Endobronchial View," Youtube.com 2013.
22. E. Patrick et al., "Using a large projection screen as an alternative to headmounted displays for virtual environments," in Proc. Int. Conf. CHI, 2000, vol. 2, pp. 478-485.
23. F. Tyndiuk, G. Thomas, V. Lespinet-Najib, and C. Schlick, "Cognitive comparison of 3d interaction in front of Large vs. Small displays," in Proc. ACM Int. Conf. VRST, 2003, pp. 117-123.
24. C. Demiralp, C. Jackson, D. Karelitz, S. Zhang, and D. Laidlaw, "Cave and fishtank virtual-reality displays: A qualitative and quantitative comparison," IEEE Trans. Vis. Comput. Graph., vol. 12, no. 3, pp. 323-330, 2006.
25. D. Tan, D. Gergle, P. Scupelli, and R. Pausch, "With similar visual angles, larger displays improve spatial performance," in Proc. ACM Int. Conf. VRST, 2005, pp. 46-55.
26. B. Riecke, J. Schulte-Pelkum, F. Caniard, and H. Blthoff, "Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality," in Proc. Int. Work. Presence, 2005, pp. 49-57.
27. J. H. Cho, H. I. Son, D. G. Lee, T. Bhattacharjee, and D. Y. Lee, "Gain-scheduling control of teleoperation systems interacting with soft tissues," IEEE Trans. Ind. Electron., vol. 60, no. 3, pp. 946-957, Mar. 2013.
28. K. Sugawara, Y. Suzuki, D. Yashiro, and K. Ohnishi, "Performance evaluation of thrust wires for application to endoscopic forceps," in Proc. IEEE ISIE, 2011, pp. 2201-2206.
29. T. Ni et al., "A survey of high-resolution display technologies, techniques, and applications," in Proc. IEEE Int. Conf. VR, 2006, pp. 223-236.
30. M. McKenna and D. Zeltzer, "Three dimensional visual display systems for virtual environments," Presence, Teleoper. Virtual Environ., vol. 1, no. 4, pp. 421-458, 1992.
31. C. D. Liao and J. C. Tsai, "The evolution ofMEMS displays," IEEE Trans. Ind. Electron., vol. 56, no. 4, pp. 1057-1065, Apr. 2009.
32. C. H. Yeh et al., "Fast mode decision algorithm through inter-view ratedistortion prediction for multiview video coding system," IEEE Trans. Ind. Informat., vol. 10, no. 1, pp. 594-603, Feb. 2013.
33. C. Chen and Y. F. Zheng, "Passive and active stereo vision for smooth surface detection of deformed plates," IEEE Trans. Ind. Electron., vol. 42, no. 3, pp. 300-306, Jun. 1995.
34. I. P. Howard and B. J. Rogers, Binocular Vision and Stereopsis. London, U.K.: Oxford Univ. Press, 1995.
35. J. I. Cutting and P. M. Vishton, "Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth," in Handbook of Perception and Cognition: Perception of Space and Motion, vol. 5, E. Rogers, Ed. San Diego, CA, USA: Academic, 1995, pp. 69-117.
36. D. Drascic, "Skill acquisition and task performance in teleoperation using monoscopic and stereoscopic video remote viewing," in Proc. 35th Human Factors Soc., 1991, pp. 1367-1371.
37. U. Naepflin and M. Menozzi, "Can movement parallax compensate lacking stereopsis in spatial explorative tasks?" Displays, vol. 22, no. 5, pp. 157-164, Nov. 2001.
38. T. Muraoka and H. Ikeda, "Perception of pseudo-stereoscopic image depths of three-dimensional objects displayed on VDT screen," in Proc. IEEE Int. Conf. Ind. Informat., 2007, pp. 681-685.
39. M. Bocker, D. Runde, and L. Muhlback, "On the reproduction of motion parallax in videocommunications," in Proc. 39th Human Factors Soc., 1995, pp. 198-202.
40. G. Jones, D. Lee, N. Holliman, and D. Ezra, "Controlling perceived depth in stereoscopic images," in Proc. SPIE Stereoscopic Displays VR Syst. VIII, 2001, vol. 4297, pp. 42-54.
41. I. Sexton and P. Surman, "Stereoscopic and autostereoscopic display systems," IEEE Signal Process. Mag., vol. 16, no. 3, pp. 85-89, May 1999.
42. L. Leroy, P. Fuchs, and G. Moreau, "Visual fatigue reduction for immersive stereoscopic displays by disparity, content, and focus-point adapted blur," IEEE Trans. Ind. Electron., vol. 59, no. 10, pp. 3998-4004, Oct. 2012.
43. L. Leroy, P. Fuchs, and G. Moreau, "Real time comfort enhancement in stereoscopic displays by disparity and content-adapted blur," in Proc. IEEE ISIE, 2010, pp. 3595-3600.
44. W. Birkfellne et al., "Computer-enhanced stereoscopic vision in a headmounted operating binocular," Phys. Med. Biol., vol. 48, no. 3, pp. N49-N57, Feb. 2003.
45. M. Ferre, R. Aracil, and M. A. Sanchez-Uran, "Stereoscopic human interfaces," IEEE Robot. Autom. Mag., vol. 15, no. 4, pp. 50-57, Dec. 2008.
46. Y. Y. Yed and L. D. Silverstein, "Limits of fusion and depth judgment in stereoscopic color displays," Human Factors, vol. 32, no. 1, pp. 45-60, Feb. 1990.
47. C. Paloc et al., "Computer-aided surgery based on auto-stereoscopic augmented reality," in Proc. Int. Conf. Inf. Vis., 2004, pp. 189-193.
48. D. Maupu, M. H. Van Horn, S. Weeks, and E. Bullitt, "3D stereo interactive medical visualization," IEEE Comput. Graph. Appl., vol. 25, no. 5, pp. 67-71, Sep./Oct. 2005.
49. P. M. Novotny et al., "Stereo display of 3D ultrasound images for surgical robot guidance," in Proc. IEEE Int. Conf. Eng. Med. Biol. Soc., 2006, pp. 1509-1512.
50. Y. Zhao and Y. Lou, "A vision based navigation for capsule endoscopies," in Proc. IEEE ICIEA, 2013, pp. 723-728.
51. J. Rubin, Handbook of Usability Testing: How to Plan, Design and Conduct Effective Tests. Hoboken, NJ, USA: Wiley, 1993.
52. J. Nielson, Usability Engineering. San Mateo, CA, USA: Morgan Kaufmann, 1993.
53. D. Kasik, J. Troy, S. Amorosi, M. Murray, and S. Swamy, "Evaluating graphics displays for complex 3D models," IEEE Comput. Graph. Appl., vol. 22, no. 3, pp. 56-64, May/Jun. 2002.
54. L. Lipton, The Stereographics Developers' Handbook. San Rafael, CA, USA: StereoGraphics Corp., 1997.
55. I. Rivas-Blanco, P. del Saz-Orozco, I. Garcia-Morales, and V. Mufioz, "Robotic system for single incision laparoscopic surgery," in Proc. IEEE IECON, 2012, pp. 2774-2779.
56. S. Livatino, F. Banno, and G. Muscato, "3-D Integration of robot vision and laser data with semiautomatic calibration in augmented reality stereoscopic visual interface," IEEE Trans. Ind. Informat, vol. 8, no. 1, pp. 69-77, Feb. 2012.
57. F. Devemay, F. Mourgues, and E. Coste-Maniere, "Towards endoscopic augmented reality for robotically assisted minimally invasive cardiac surgery," in Proc. IEEE Workshop Med. Imag. AR, 2001, pp. 16-20.
58. W. Hu, G. P. Liu, and H. Zhou, "Web-based 3-D control laboratory for remote real-time experimentation," IEEE Trans. Ind. Electron., vol. 60, no. 10, pp. 4673-4682, Oct. 2013.
59. H. S. Park, T. Shiratori, I. Matthews, and Y. Sheikh, "3D reconstruction of a moving point from a series of 2D projections," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 40203-4023.
60. Y. Gao, M. Wang, D. Tao, R. Ji, and Q. Dai, "3-D Object retrieval and recognition with hypergraph analysis," IEEE Trans. Imag. Process., vol. 21, no. 9, pp. 4290-4303, Sep. 2012.
61. Y. Gao, M. Wang, R. Ji, X. Wu, and Q. Dai, "3-D Object retrieval with Hausdorff distance learning," IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 2088-2098, Apr. 2014.