Mobile markerless augmented reality and its application in forensic medicine

International Journal of Computer Assisted Radiology and Surgery

Volumn 10, Issue 5, 2015, Pages 573-586

  Kilgus, Thomas ^a   Heim, Eric ^a   Haase, Sven ^c   Prüfer, Sabine ^d   Müller, Michael ^a   Seitel, Alexander ^a,b   Fangerau, Markus ^a   Wiebe, Tamara ^a   Iszatt, Justin ^a   Schlemmer, Heinz Peter ^a   Hornegger, Joachim ^c   Yen, Kathrin ^d   Maier Hein, Lena ^a  

^a GERMAN CANCER RESEARCH CENTER DKFZ   (Germany)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^c UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^d UNIVERSITY HOSPITAL HEIDELBERG   (Germany)

Author keywords

Forensic medicine; Iterative closest point algorithm; Kinect; Mobile application; Mobile augmented reality; Range imaging; Surface documentation

Indexed keywords

ACCURACY; ARTICLE; AUTOPSY; CAMERA; COMPUTER ASSISTED TOMOGRAPHY; FEASIBILITY STUDY; FORENSIC MEDICINE; IMAGING PHANTOM; MICROCOMPUTER; MOBILE APPLICATION; MOBILE MARKERLESS; PILOT STUDY; PRIORITY JOURNAL; REGISTRATION; SKIN SURFACE; TRIANGULATION; COMPUTER PROGRAM; FIDUCIAL MARKER; HUMAN; PROCEDURES;

AUTOPSY; FEASIBILITY STUDIES; FIDUCIAL MARKERS; FORENSIC MEDICINE; HUMANS; PILOT PROJECTS; SOFTWARE; TOMOGRAPHY, X-RAY COMPUTED;

EID: 84928523605     PISSN: 18616410     EISSN: 18616429     Source Type: Journal    
DOI: 10.1007/s11548-014-1106-9     Document Type: Article

References (49)

1. Ampanozi G, Zimmermann D, Hatch GM, Ruder TD, Ross S, Flach PM, Thali MJ, Ebert LC (2012) Format preferences of district attorneys for post-mortem medical imaging reports: understandability, cost effectiveness, and suitability for the courtroom: A questionnaire based study. Legal Med 14(3):116–120
2. Batcher KE (1968) Sorting networks and their applications. In: Proceedings of the April 30-May 2, Spring Joint Computer Conference, pp 307–314
3. Bauer S, Seitel A, Hofmann H, Blum T, Wasza J, Balda M, Meinzer HP, Navab N, Hornegger J, Maier-Hein L (2013) Real-time range imaging in health care: a survey. In: Time-of-flight and depth imaging. Sensors, algorithms, and applications. Springer, Berlin, pp 228–254
4. Besl PJ, Neil DM (1992) A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256
5. Bhatia A, Patel S, Pantol G, Wu YY, Plitnikas M, Hancock C (2013) Intra and inter-observer reliability of mobile tablet PACS viewer system versus standard PACS viewing station-diagnosis of acute central nervous system events. Open J Radiol 03(02):91–98
6. Blum T, Kleeberger V, Bichlmeier C, Navab N (2012) mirracle: an augmented reality magic mirror system for anatomy education. In: Virtual reality short papers and posters (VRW), IEEE, pp 115–116
7. Chetverikov D, Svirko D, Stepanov D, Krsek P (2002) The trimmed iterative closest point algorithm. In: Proceedings of the 16th international conference on pattern recognition, vol 3. IEEE, pp 545–548
8. Dabrowski JR, Munson EV (2001) Is 100 milliseconds too fast? In: CHI EA ’01 CHI ’01 Extended abstracts on human factors in computing systems. ACM Press, pp 317–318
9. Danilchenko A, Fitzpatrick JM (2011) General approach to first-order error prediction in rigid point registration. IEEE Trans Med Imaging 30(3):679–693
10. dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer HP, Meinzer HP, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal (accepted subject to minor revisions)
11. Ebert L, Hatch G, Thali M, Ross S (2013) Invisible touch–control of a DICOM viewer with finger gestures using the kinect depth camera. J Forensic Radiol Imaging 1(1):10–14
12. Ebert LC, Hatch G, Ampanozi G, Thali MJ, Ross S (2012) You can’t touch this: touch-free navigation through radiological images. Surg Innov 19(3):301–307
13. Ebert LC, Ptacek W, Breitbeck R, Fürst M, Kronreif G, Martinez RM, Thali M, Flach PM (2014) Virtobot 2.0: the future of automated surface documentation and CT-guided needle placement in forensic medicine. Forensic Sci Med Pathol 10(2):179–186
14. Ebert LC, Ptacek W, Naether S, Fürst M, Ross S, Buck U, Weber S, Thali M (2009) Virtobot: a multi-functional robotic system for 3D surface scanning and automatic post mortem biopsy. Int J Med Robot Comput Assist Surg 6:18–27
15. Glocker B, Izadi S, Shotton J, Criminisi A (2013) Real-time RGB-D camera relocalization. In: IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 173–179
16. Handa A, Newcombe RA, Angeli A, Davison AJ (2012) Real-time camera tracking: when is high frame-rate best? In: Computer vision-ECCV 2012. Springer, pp 222–235
17. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge
18. Horn BKP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am A 4(4):629–642
19. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A, Fritzgibbon A (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM, pp 559–568
20. John S, Poh ACC, Lim TCC, Chan EHY, Chong LR (2012) The iPad tablet computer for mobile on-call radiology diagnosis? Auditing discrepancy in CT and MRI reporting. J Digit Imaging 25(5):628–634
21. Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(12):1437–1454
22. Kilgus T, Franz A, Seitel A, März K, Bartha L, Fangerau M, Mersmann S, Groch A, Meinzer HP, Maier-Hein L (2012) Registration of partially overlapping surfaces for range image based augmented reality on mobile devices. In: SPIE medical imaging. International Society for Optics and Photonics, pp 83160T–83160T
23. Kilgus T, dos Santos TR, Seitel A, Yung K, Franz AM, Groch A, Wolf I, Meinzer HP, Maier-Hein L (2011) Generation of triangle meshes from time-of-flight data for surface registration. In: Bildverarbeitung für die Medizin 2011. Springer, pp 189–193
24. Kirk DB, Wen-mei WH (2012) Programming massively parallel processors: a hands-on approach. Newnes
25. Kramer J, Burrus N, Echtler F, Daniel HC, Parker M (2012) Hacking the Kinect. Springer, New York
26. Li H, Sumner RW, Pauly M (2008) Global correspondence optimization for non-rigid registration of depth scans. In: Eurographics symposium on geometry processing, vol 27. Wiley, pp 1421–1430
27. Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3d surface construction algorithm. In: ACM siggraph computer graphics, vol 21. ACM, pp 163–169
28. Maier-Hein L, Fangerau M, Meinzer HP, Seitel A (2013) Visualization of anatomical data by augmented reality. US Patent App. 13/891,310
29. Maier-Hein L, Franz AM, Fangerau M, Schmidt M, Seitel A, Mersmann S, Kilgus T, Groch A, Yung K, dos Santos TR, Meinzer HP (2011) Towards mobile augmented reality for on-patient visualization of medical images. In: Bildverarbeitung für die Medizin 2011, Informatik Aktuell. Springer, pp 389–393
30. Maier-Hein L, Franz AM, Santos TRD, Schmidt M, Fangerau M, Meinzer HP, Fitzpatrick JM (2012) Convergent iterative closest-point algorithm to accommodate anisotropic and inhomogenous localization error. IEEE Trans Pattern Anal Mach Intell 34(8):1520–1532
31. Mersmann S, Seitel A, Erz M, Jähne B, Nickel F, Mieth M, Mehrabi A, Maier-Hein L (2013) Calibration of time-of-flight cameras for accurate intraoperative surface reconstruction. Med Phys 40(8):082701
32. Müller M, Rassweiler MC, Klein J, Seitel A, Gondan M, Baumhauer M, Teber D, Rassweiler JJ, Meinzer HP, Maier-Hein L (2013) Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. Int J Comput Assist Radiol Surg 8(4):663–675
33. Myronenko A, Song X (2010) Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275
34. Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C (2007) Action- and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput Gr Appl 27(5):10–14
35. Nolden M, Zelzer S, Seitel A, Wald D, Müller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein KH, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances. Int J Comput Assist Radiol Surg 8(4):607–620
36. Qiu D, May S, Nüchter A (2009) GPU-accelerated nearest neighbor search for 3D registration. In: Proceedings of the 7th international conference on computer vision systems. Springer, pp 194–203
37. Roberts IS, Benamore RE, Benbow EW, Lee SH, Harris JN, Jackson A, Mallett S, Patankar T, Peebles C, Roobottom C, Trail ZC (2012) Post-mortem imaging as an alternative to autopsy in the diagnosis of adult deaths: a validation study. Lancet 379(9811):136–142
38. Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: IEEE international conference on robotics and automation. IEEE, pp 3212–3217
39. Seeling P, Fitzek FH, Ertli G, Pulipaka A, Reisslein M (2010) Video network traffic and quality comparison of VP8 and H.264 SVC. In: Proceedings of the 3rd workshop on mobile video delivery. ACM, pp 33–37
40. Seitel A, Engel M, Sommer CM, Radeleff BA, Essert-Villard C, Baegert C, Fangerau M, Fritzsche KH, Yung K, Meinzer HP (2011) Computer-assisted trajectory planning for percutaneous needle insertions. Med Phys 38(6):3246–3259
41. Seitel A, Yung K, Mersmann S, Kilgus T, Groch A, dos Santos TR, Franz AM, Nolden M, Meinzer HP, Maier-Hein L (2012) MITK-ToF: range data within MITK. Int J Comput Assist Radiol Surg 7(1):87–96
42. Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T, Kosugi C, Higuchi R, Watayo Y, Yagawa Y, Uemura S, Tsuchiya H, Azuma T (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepato Biliary Pancreat Sci 17(5):629–636
43. Thali MJ, Braun M, Wirth J, Vock P, Dirnhofer R (2003) 3D surface and body documentation in forensic medicine: 3-D/CAD photogrammetry merged with 3D radiological scanning. J Forensic Sci 48(6):1356–1365
44. Thali MJ, Dirnhofer R, Vock P (2009) The virtopsy approach: 3D optical and radiological scanning and reconstruction in forensic medicine. CRC Press, Boca Raton
45. Urschler M, Höller J, Bornik A, Paul T, Giretzlehner M, Bischof H, Yen K, Scheurer E (2014) Intuitive presentation of clinical forensic data using anonymous and person-specific 3d reference manikins. Forensic Sci Int 241:155–166
46. Valette S, Chassery JM, Prost R (2008) Generic remeshing of 3D triangular meshes with metric-dependent discrete voronoi diagrams. IEEE Trans Vis Comput Gr 14(2):369–381
47. Yang J, Li H, Jia Y (2013) Go-ICP: solving 3D registration efficiently and globally optimally. In: International conference on computer vision. IEEE, pp 1457–1464
48. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334
49. Zhu E, Masiello I, Hadadgar A, Zary N (2014) Augmented reality in healthcare education: an integrative review. PeerJ PrePrints 2:1–22