Calibration of time-of-flight cameras for accurate intraoperative surface reconstruction

Medical Physics

Volumn 40, Issue 8, 2013, Pages

  Mersmann, Sven ^a   Seitel, Alexander ^a   Erz, Michael ^b   Jähne, Bernd ^b   Nickel, Felix ^b   Mieth, Markus ^b   Mehrabi, Arianeb ^b   Maier Hein, Lena ^a  

^a GERMAN CANCER RESEARCH CENTER DKFZ   (Germany)

^b UNIVERSITY OF HEIDELBERG   (Germany)

Author keywords

calibration; intraoperative surface reconstruction; time of flight cameras

Indexed keywords

CALIBRATION; CAMERAS; IMAGE ACQUISITION; MANUFACTURE; MEDICAL APPLICATIONS; MEDICAL IMAGING; MORPHOLOGY; MOTION TRACKING; SURFACE RECONSTRUCTION; SURGERY; SYSTEMATIC ERRORS;

CALIBRATION PROCEDURE; INTRA-OPERATIVE; MULTIMODAL REGISTRATION; RESPIRATORY MOTION TRACKING; SURFACE-BASED REGISTRATION; TARGET REGISTRATION ERRORS; TIME OF FLIGHT (TOF) CAMERAS; TIME-OF-FLIGHT CAMERAS;

COMPUTERIZED TOMOGRAPHY;

ARTICLE; CALIBRATION; CAMERA; IMAGE GUIDED SURGERY; MEASUREMENT ACCURACY; MEASUREMENT ERROR; PRIORITY JOURNAL; SENSOR; SURGICAL TECHNIQUE; SYSTEMATIC ERROR; TIME OF FLIGHT CAMERA; COMPUTER ASSISTED SURGERY; COMPUTER ASSISTED TOMOGRAPHY; EQUIPMENT; HUMAN; IMAGE PROCESSING; INTRAOPERATIVE PERIOD; LIVER; RADIOGRAPHY; TIME; DEVICES; SURGERY;

CALIBRATION; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; INTRAOPERATIVE PERIOD; LIVER; SURGERY, COMPUTER-ASSISTED; TIME FACTORS; TOMOGRAPHY, X-RAY COMPUTED;

EID: 84881632326     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.4812889     Document Type: Article

References (56)

1. L.-M. Su, B. P. Vagvolgyi, R. Agarwal, C. E. Reiley, R. H. Taylor, and G. D. Hager, " Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration.," Urology 73, 896-900 (2009). 10.1016/j.urology.2008.11.040
2. T. Langø, G. A. Tangen, R. Måarvik, B. Ystgaard, Y. Yavuz, J. H. Kaspersen, O. V. Solberg, and T. A. N. Hernes, " Navigation in laparoscopy-prototype research platform for improved image-guided surgery.," Minimally Invasive Ther. Allied Technol. 17, 17-33 (2008). 10.1080/13645700701797879
3. J.-H. Kim, A. Bartoli, T. Collins, and R. Hartley, " Tracking by detection for interactive image augmentation in laparoscopy.," in Biomedical Image Registration, Lecture Notes in Computer Science Vol. 7359, edited by, B. Dawant, G. Christensen, J. Fitzpatrick, and, D. Rueckert, (Springer, Berlin, 2012), pp. 246-255. 10.1007/978-3-642-31340-0-26
4. D. M. Cash, T. K. Sinha, W. C. Chapman, H. Terawaki, B. M. Dawant, R. L. Galloway, and M. I. Miga, " Incorporation of a laser range scanner into image-guided liver surgery: Surface acquisition, registration, and tracking.," Med. Phys. 30, 1671-1682 (2003). 10.1118/1.1578911
5. C. Schaller, J. Penne, and J. Hornegger, " Time-of-flight sensor for respiratory motion gating.," Med. Phys. 35, 3090-3093 (2008). 10.1118/1.2938521
6. C. Schaller, A. Adelt, J. Penne, and J. Hornegger, " Time-of-flight sensor for patient positioning.," Proc. SPIE 7261, 726110-726118 (2009). 10.1117/12.812498
7. S. Placht, J. Stancanello, C. Schaller, M. Balda, and E. Angelopoulou, " Fast time-of-flight camera based surface registration for radiotherapy patient positioning.," Med. Phys. 39, 4-17 (2012). 10.1118/1.3664006
8. J. Penne, C. Schaller, J. Hornegger, and T. Kuwert, " Robust real-time 3D respiratory motion detection using time-of-flight cameras.," Int. J. Comp. Assist. Radiol. Surg. 3, 427-431 (2008). 10.1007/s11548-008-0245-2
9. Z. Xu, R. Schwarte, H. Heinol, B. Buxbaum, and T. Ringbeck, " Smart pixel-photonic mixer device (PMD).," in Proceedings of the International Conference on Mechatronics and Machine Vision in Practice (Centre for Intelligent Design, Automation and Manufacturing (CIDAM), Hong Kong, 1998), Vol. M2VIP, pp. 259-264.
10. R. Lange, " 3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology.," Ph.D. thesis (University of Siegen, 2000).
11. M. Albrecht, " Photogate-PMD-Sensoren.," Ph.D. thesis (University of Siegen, 2007).
12. S. Mersmann, M. Müller, A. Seitel, F. Arnegger, R. Tetzlaff, J. Dinkel, M. Baumhauer, B. Schmied, H.-P. Meinzer, and L. Maier-Hein, " Time-of-flight camera technique for augmented reality in computer-assisted interventions.," Proc. SPIE 7964, 79642C-1-79642C-9 (2011). 10.1117/12.878149
13. Evaluation of Measurement Data-Guide to the Expression of Uncertainty in Measurement (GUM), edited by BIPM, IEC, IFCC, ILAC, ISO, IUPAC, IUPAP, and OIML, 2nd ed. (International Organization for Standardization (ISO), Geneva, 2008).
14. S. Foix, G. Alenya, and C. Torras, " Lock-in time-of-flight (ToF) cameras: A survey.," IEEE Sens. J. 11, 1917-1926 (2011). 10.1109/JSEN.2010.2101060
15. H. G. Heinol, " Untersuchung und Entwicklung von Modulationslaufzeitbasierten 3D-Sichtsystemen.," Ph.D. thesis (University of Siegen, 2001).
16. M. Schmidt, " Analysis, modeling and dynamic optimization of 3D time-of-flight imaging systems.," Ph.D. thesis (University of Heidelberg, 2011), http://www.ub.uni-heidelberg.de/archiv/12297.
17. M. Erz, " Charakterisierung von Laufzeitkamerasystemen für Lumineszenzlebensdauermessungen.," Ph.D. thesis (University of Heidelberg, 2011), http://www.ub.uni-heidelberg.de/archiv/11598.
18. T. Kahlmann, F. Remondino, and H. Ingensand, " Calibration for incrased accuracy of the range imaging camera SwissRanger™," in Proceedings of the ISPRS Commission V Symposium 'Image Engineering and Vision Metrology', Dresden, Germany (ISPRS Archives, Beijing, 2006).
19. A. Kolb, E. Barth, R. Koch, and R. Larsen, " Time-of-flight sensors in computer graphics.," in Eurographics 2009-State of the Art Reports, edited by, M. Pauly, and, G. Greiner, (Eurographics Association, Munich, 2009), pp. 119-134.
20. M. Lindner, I. Schiller, A. Kolb, and R. Koch, " Time-of-flight sensor calibration for accurate range sensing.," Comput. Vis. Image Underst. 114, 1318-1328 (2010). 10.1016/j.cviu.2009.11.002
21. J. H. Gisolf, " Die Temperaturabhängigkeit des Widerstandes von Halbleitern.," Ann. Phys. 436, 3-26 (1947). 10.1002/andp.19474360103
22. S. Mersmann, D. Guerrero, H.-P. Meinzer, and L. Maier-Hein, " Effect of active air conditioning in medical intervention rooms on the temperature dependency of time-of-flight distance measurements.," in Bildverarbeitung für die Medizin 2012, edited by, T. Tolxdorff, T. M. Deserno, H. Handels, and, H.-P. Meinzer, (Springer, Berlin, 2012), pp. 398-403.
23. H. Rapp, " Experimental and theoretical investigation of correlating ToF-camera systems.," M.S. thesis (University of Heidelberg, Germany, 2007), http://www.ub.uni-heidelberg.de/archiv/7666.
24. D. D. Lichti, C. Kim, and S. Jamtsho, " An integrated bundle adjustment approach to range camera geometric self-calibration.," ISPRS J. Photogramm. 65, 360-368 (2010). 10.1016/j.isprsjprs.2010.04.002
25. S. Fuchs and S. May, " Calibration and registration for precise surface reconstruction with time of flight cameras.," Int. J. Intell. Syst. Technol. Appl. 5, 274-284 (2008). 10.1504/IJISTA.2008.021290
26. M. Lindner, " Calibration and real-time processing of time-of-flight range data.," Ph.D. thesis (University of Siegen, 2010).
27. M. Lindner and A. Kolb, " Calibration of the intensity-related distance error of the PMD ToF-camera.," Proc. SPIE 6764, 67640W (2007). 10.1117/12.752808
28. ISO, International Vocabulary of Basic and General Terms in Metrology (VIM) (ISO, Geneva, 2004).
29. D. D. Lichti and C. Kim, " A comparison of three geometric self-calibration methods for range cameras.," Remote Sens. 3, 1014-1028 (2011). 10.3390/rs3051014
30. D. D. Lichti, X. Qi, and T. Ahmed, " Range camera self-calibration with scattering compensation.," ISPRS J. Photogramm. 74, 101-109 (2012). 10.1016/j.isprsjprs.2012.09.008
31. P. Westfeld, " Ansätze zur Kalibrierung des Range-Imaging-Sensors SR-3000 unter simultaner Verwendung von Intensitäts- und Entfernungsbildern.," in Photogrammetrie-Laserscanning-Optische 3D-Messtechnik. Beiträge der Oldenburger 3D-Tage 2007, edited by, T. Luthmann, and, C. Müller, (Wichmann Verlag, Heidelberg, 2007).
32. M. Shahbazi, S. Homayouni, M. Saadatseresht, and M. Sattari, " Range camera self-calibration based on integrated bundle adjustment via joint setup with a 2d digital camera.," Sensors 11, 8721-8740 (2011). 10.3390/s110908721
33. O. Steiger, J. Felder, and S. Weiss, " Calibration of time-of-flight range imaging cameras.," in Proceedings of the IEEE International Workshop Imaging Systems Technical, San Diego, CA (IEEE Computer Society, Los Alamitos, 2008), pp. 123-126.
34. T. Kahlmann and H. Ingensand, " Calibration and development for increased accuracy of 3D range imaging cameras.," J. Appl. Geodesy 2, 1-11 (2008). 10.1515/JAG.2008.001
35. S. Fuchs and G. Hirzinger, " Extrinsic and depth calibration of ToF-cameras.," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (IEEE Computer Society, Los Alamitos, 2008), Vol. 5, pp. 1-6.
36. S. Hussmann, P. Huhn, and A. Hermanski, " Systematic distance deviation error compensation for a ToF-camera in the close-up range.," in Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (I2MTC), Graz, Austria (IEEE Computer Society, Los Alamitos, 2012), pp. 1546-1550. 10.1109/I2MTC.2012.6229118
37. I. Schiller, C. Beder, and R. Koch, " Calibration of a PMD-Camera using a planar calibration pattern together with a multi-camera setup.," in Proceedings of the XXI ISPRS Congress, Beijing, China (ISPRS Archives, Beijing, 2008), pp. 297-302.
38. J. P. Godbaz, M. J. Cree, and A. A. Dorrington, " Understanding and ameliorating non-linear phase and amplitude responses in amcw lidar.," Remote Sens. 4, 21-42 (2011). 10.3390/rs4010021
39. W. Karel and N. Pfeifer, " Range camera calibration based on image sequences and dense, comprehensive error statistics.," Proc. SPIE 7239, 72390D (2009). 10.1117/12.807785
40. M. Lindner and A. Kolb, " Lateral and depth calibration of PMD-distance sensors.," Advances in Visual Computing, Lake Tahoe (Springer, Berlin, 2006), Vol. 2, pp. 524-533.
41. W. Karel, " Integrated range camera calibration using image sequences from hand-held operation.," in Proceedings of the XXI ISPRS Congress, Beijing, China (ISPRS Archives, Beijing, 2008), Vol. (Part B5), pp. 945-951.
42. Y. M. Kim, D. Chan, C. Theobalt, and S. Thrun, " Design and calibration of a multi-view ToF sensor fusion system.," in Proceedings of the IEEE CVPR Workshops, Anchorage, Alaska (IEEE Computer Society, Los Alamitos, 2008), Vols. 1-3, pp. 1524-1530.
43. J. Radmer, P. Fuste, H. Schmidt, and J. Kruger, " Incident light related distance error study and calibration of the PMD-range imaging camera.," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (IEEE Computer Society, Los Alamitos, 2008), Vols. 1-3, pp. 1579-1584.
44. F. Chiabrando, R. Chiabrando, D. Piatti, and F. Rinaudo, " Sensors for 3D imaging: metric evaluation and calibration of a CCD/CMOS time-of-flight camera.," Sensors 9, 10080-10096 (2009). 10.3390/s91210080
45. J. Rüeger, Electronic Distance Measurement: An Introduction (Springer-Verlag, Heidelberg, 1990).
46. D. Piatti and F. Rinaudo, " SR-4000 and CamCube 3.0 Time of flight (ToF) cameras: Tests and comparison.," Remote Sens. 4, 1069-1089 (2012). 10.3390/rs4041069
47. F. Mufti and R. Mahony, " Statistical analysis of signal measurement in time-of-flight cameras.," ISPRS J. Photogramm. 66, 720-731 (2011). 10.1016/j.isprsjprs.2011.06.004
48. Z. Zhang, " A flexible new technique for camera calibration.," IEEE Trans. Pattern Anal. 22, 1330-1334 (2000). 10.1109/34.888718
49. I. Wolf, M. Vetter, I. Wegner, T. Böttger, M. Nolden, M. Schöbinger, M. Hastenteufel, T. Kunert, and H.-P. Meinzer, " The medical imaging interaction toolkit.," Med. Image Anal. 9, 594-604 (2005). 10.1016/j.media.2005.04.005
50. C. Tomasi and R. Manduchi, " Bilateral filtering for gray and color images.," in Proceedings of the Sixth International Conference on Computer Vision, Bombay, India (IEEE Computer Society, Los Alamitos, 1998), pp. 839-846. 10.1109/ICCV.1998.710815
51. W. E. Lorensen and H. E. Cline, " Marching cubes: A high resolution 3d surface construction algorithm.," in Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '87, Anaheim (ACM, New York, 1987), pp. 163-169. 10.1145/37401.37422
52. P. J. Besl and N. D. McKay, " A method for registration of 3D shapes.," IEEE Trans. Pattern. Anal. 14, 239-256 (1992). 10.1109/34.121791
53. M. Miga, T. Sinha, D. Cash, R. Galloway, and R. Weil, " Cortical surface registration for image-guided neurosurgery using laser-range scanning.," IEEE Trans. Med. Imaging. 22, 973-985 (2003). 10.1109/TMI.2003.815868
54. L. Maier-Hein, A. Franz, T. dos Santos, M. Schmidt, M. Fangerau, H.-P. Meinzer, and J. M. Fitzpatrick, " Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error.," IEEE Trans. Pattern Anal. 34, 1520-1532 (2012). 10.1109/TPAMI.2011.248
55. M. Erz and B. Jähne, " Radiometric and spectrometric calibrations, and distance noise measurement of ToF cameras.," 3rd Workshop on Dynamic 3-D Imaging (, R. Koch, and, A. Kolb, eds.), Lecture Notes in Computer Science (Springer, Berlin, 2009), Vol. 5742, pp. 28-41.
56. M. Schmidt and B. Jähne, " A physical model of Time-of-Flight 3D imaging systems, including suppression of ambient light.," 3rd Workshop on Dynamic 3-D Imaging (, R. Koch, and, A. Kolb, eds.), Lecture Notes in Computer Science (Springer, Berlin, 2009), Vol. 5742, pp. 1-15.