A machine learning approach for real-time modelling of tissue deformation in image-guided neurosurgery

Artificial Intelligence in Medicine

Volumn 80, Issue , 2017, Pages 39-47

  Tonutti, Michele ^a   Gras, Gauthier ^a   Yang, Guang Zhong ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Artificial neural networks; Biomechanics; Finite element method; Image guided surgery; Machine learning; Soft tissue deformation; Support vector regression

Indexed keywords

ARTIFICIAL INTELLIGENCE; AUGMENTED REALITY; BIOMECHANICS; DEFORMATION; EDUCATION; ERRORS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MESH GENERATION; MESH NETWORKING; NEURAL NETWORKS; NEUROSURGERY; PATIENT REHABILITATION; SURGERY; SURGICAL EQUIPMENT; TISSUE; TISSUE ENGINEERING; TUMORS;

DEFORMATION MODELING; FINITE ELEMENT METHOD SIMULATION; IMAGE GUIDED NEURO SURGERIES; IMAGE GUIDED SURGERY; MACHINE LEARNING APPROACHES; REAL-TIME APPLICATION; SOFT TISSUE DEFORMATION; SUPPORT VECTOR REGRESSION (SVR);

FINITE ELEMENT METHOD;

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; BRAIN TUMOR; HUMAN; IMAGE GUIDED NEUROSURGERY; LEARNING ALGORITHM; MACHINE LEARNING; NEUROSURGERY; PRIORITY JOURNAL; SUPPORT VECTOR MACHINE; TISSUE DEFORMATION; TISSUE INJURY; ALGORITHM; COMPUTER ASSISTED SURGERY; COMPUTER SIMULATION; FINITE ELEMENT ANALYSIS;

ALGORITHMS; COMPUTER SIMULATION; FINITE ELEMENT ANALYSIS; HUMANS; MACHINE LEARNING; NEUROSURGICAL PROCEDURES; SURGERY, COMPUTER-ASSISTED;

EID: 85025439926     PISSN: 09333657     EISSN: 18732860     Source Type: Journal    
DOI: 10.1016/j.artmed.2017.07.004     Document Type: Article

References (70)

1. Payne, C.J., Yang, G.-Z., Hand-held medical robots. Ann Biomed Eng 42:8 (2014), 1594–1605.
2. Payne, C.J., Marcus, H.J., Yang, G.-Z., A smart haptic hand-held device for neurosurgical microdissection. Ann Biomed Eng 43:9 (2015), 2185–2195.
3. Hughes-Hallett, A., Pratt, P., Dilley, J., Vale, J., Darzi, A., Mayer, E., Augmented reality: 3D image-guided surgery. Cancer Imaging, 15(Suppl 1), 2015, O8.
4. Gras, G., Marcus, H.J., Payne, C.J., Pratt, P., Visual force feedback for hand-held microsurgical instruments. Medical image computing and computer-assisted intervention, vol. 9349, 2015, 480–487.
5. Abhari, K., Baxter, J.S.H., Chen, E.S., Khan, A.R., Wedlake, C., Peters, T., et al. The role of augmented reality in training the planning of brain tumor resection. Augment Real Environ Med Imaging Comput Assist Interv, 2013, 241–248.
6. Marcus, H.J., Pratt, P., Hughes-Hallett, A., Cundy, T.P., Marcus, A.P., Yang, G.Z., et al. Comparative effectiveness and safety of image guidance systems in neurosurgery: a preclinical randomized study. J Neurosurg 123:August (2015), 1–7.
7. Carter, T.J., Sermesant, M., Cash, D.M., Barratt, D.C., Tanner, C., Hawkes, D.J., Application of soft tissue modelling to image-guided surgery. Med Eng Phys 27:10 (2005), 893–909.
8. Kyriacou, S.K., Davatzikos, C., A biomechanical model of soft tissue deformation with applications to non-rigid registration of brain images with tumor pathology. Medical image computing and computer assisted interventation MICCAI′98, vol. 1496, 1998, 531–538.
9. Wittek, A., Miller, K., Kikinis, R., Warfield, S.K., Patient-specific model of brain deformation: application to medical image registration. J Biomech 40:4 (2007), 919–929.
10. Nicolau, S., Soler, L., Mutter, D., Marescaux, J., Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:3 (2011), 189–201.
11. Nakamoto, M., Hirayama, H., Sato, Y., Konishi, K., Kakeji, Y., Hashizume, M., et al. Recovery of respiratory motion and deformation of the liver using laparoscopic freehand 3D ultrasound system. Med Image Anal 11:5 (2007), 429–442.
12. Dixon, B.J., Daly, M.J., Chan, H., Vescan, A.D., Witterick, I.J., Irish, J.C., Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc Other Interv Tech 27:2 (2013), 454–461.
13. Courtecuisse, H., Jung, H., Allard, J., Duriez, C., Lee, D.Y., Cotin, S., GPU-based real-time soft tissue deformation with cutting and haptic feedback. Prog Biophys Mol Biol 103:2–3 (2010), 159–168.
14. Choi, H., Cho, B., Masamune, K., Hashizume, M., Hong, J., An effective visualization technique for depth perception in augmented reality-based surgical navigation. Int J Med Robot Comput Assist Surg, 2015.
15. Bichlmeier, C., Wimmer, F., Heining, S.M., Navab, N., Contextual anatomic mimesis: hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. 2007 6th IEEE and ACM international symposium on mixed and augmented reality, ISMAR, 2007, 129–138.
16. Goulette, F., Chen, Z.W., Fast computation of soft tissue deformations in real-time simulation with hyper-elastic mass links. Comput Methods Appl Mech Eng 295 (2015), 18–38.
17. Wittek, A., Joldes, G., Couton, M., Warfield, S.K., Miller, K., Patient-specific non-linear finite element modelling for predicting soft organ deformation in real-time; Application to non-rigid neuroimage registration. Prog Biophys Mol Biol 103:2–3 (2010), 292–303.
18. Miller, K., Wittek, A., Joldes, G., Biomechanics of the brain for computer-integrated surgery. Acta Bioeng Biomech 12:2 (2010), 25–37.
19. Choi, K.-S., Sun, H., Heng, P.-A., An efficient and scalable deformable model for virtual reality-based medical applications. Artif Intell Med 32:1 (2004), 51–69.
20. Basafa, E., Farahmand, F., Real-time simulation of the nonlinear visco-elastic deformations of soft tissues. Int J Comput Assist Radiol Surg 6:3 (2011), 297–307.
21. Halic, T., Kockara, S., Bayrak, C., Rowe, R., Chen, B., Soft tissue deformation and optimized data structures for mass spring methods. Proceedings of the 2009 ninth IEEE international conference on bioinformatics and bioengineering, BIBE ‘09, 2009, IEEE Computer Society, Washington, DC, USA, 45–52.
22. Paloc, C., Bello, F., Kitney, R., Darzi, A., Online multiresolution volumetric mass spring model for real time soft tissue deformation. Proceedings of the 5th international conference on medical image computing and computer-assisted intervention-part II, MICCAI ‘02, London UK, UK, 2002, Springer-Verlag, 219–226.
23. Chunping, G., Finite Element Modelling of The Human Brain and Application in Neurosurgical Procedures (PhD thesis). 2004, National University of Singapore.
24. Wittek, A., Hawkins, T., Miller, K., On the unimportance of constitutive models in computing brain deformation for image-guided surgery. Biomech Model Mechanobiol 8:1 (2009), 77–84.
25. Joldes, G.R., Wittek, A., Miller, K., Real-time nonlinear finite element computations on GPU – application to neurosurgical simulation. Comput Methods Appl Mech Eng 199:49–52 (2010), 3305–3314.
26. Bucki, M., Lobos, C., Payan, Y., Hitschfeld, N., Jacobian-based repair method for finite element meshes after registration. Eng Comput 27:3 (2011), 285–297.
27. Tizzard, A., Horesh, L., Yerworth, R.J., Holder, D.S., Bayford, R.H., Generating accurate finite element meshes for the forward model of the human head in EIT. Physiol Meas 26:2 (2005), S251–S261.
28. Lashkari, M., Frahmand, F., Kangarlou, K., Finite element modeling of the human brain under impact conditions. Int Res J Appl Basic Sci 6:6 (2013), 875–881.
29. Szekely, G., Brechbuhler, C., Hutter, R., Rhomberg, A., Ironmonger, N., Schmid, P., Modelling of soft tissue deformation for laparoscopic surgery simulation. Med Image Anal 4 (2000), 57–66.
30. Marcus, H.J., Cundy, T.P., Hughes-Hallett, A., Yang, G.-Z., Darzi, A., Nandi, D., Endoscopic and keyhole endoscope-assisted neurosurgical approaches: a qualitative survey on technical challenges and technological solutions. Br J Neurosurg 28:August 2013 (2014), 1–5.
31. Gillies, D., Bourmpos, M., Modelling soft tissue deformations using principal component analysis and haptic devices. 2003, Department of Computing, Imperial College of London.
32. Mosbech, T.H., Ersbøoll, B.K., Christensen, L.B., An efficient data-driven tissue deformation model. 2009 IEEE 12th international conference on computer vision workshops, ICCV workshops, 2009, 2009, 1771–1777.
33. Akinbiyi, T., Reiley, C.E., Saha, S., Burschka, D., Hasser, C.J., Yuh, D.D., et al. Dynamic augmented reality for sensory substitution in robot-assisted surgical systems. Annual international conference of the IEEE engineering in medicine and biology – proceedings, 2006, 567–570.
34. Yuen, S.G., Yip, M.C., Vasilyev, N.V., Perrin, D.P., del Nido, P.J., Howe, R.D., Robotic force stabilization for beating heart intracardiac surgery. Medical image computing and computer-assisted intervention – MICCAI 2009, Pt I, proceedings, vol. 5761, 2009, 26–33.
35. Kitagawa, M., Dokko, D., Okamura, A.M., Yuh, D.D., Effect of sensory substitution on suture-manipulation forces for robotic surgical systems. J Thorac Cardiovasc Surg 129:1 (2005), 151–158.
36. Madhavan, S., Zenklusen, J.-C., Kotliarov, Y., Sahni, H., Fine, H.A., Buetow, K., Rembrandt: helping personalized medicine become a reality through integrative translational research. Mol Cancer Res MCR 7:2 (2009), 157–167.
37. Anon, BrainsSuite: Publications. 2016 [Online accessed 10.08.16] http://brainsuite.org/publications/.
38. Miller, K., Chinzei, K., Mechanical properties of brain tissue in tension. J Biomech 35:4 (2002), 483–490.
39. Miller, K., Chinzei, K., Orssengo, G., Bednarz, P., Mechanical properties of brain tissue in vivo: experiment and computer simulation. J Biomech 33 (2000), 1369–1376.
40. Breedlove, E.L., Computational Modelling of Brain Tissue Biomechanics at High Strain Rates (PhD thesis). 2014, Purdue University.
41. Kazemi, K., Noorizadeh, N., Quantitative comparison of SPM, FSL, and brain suite for brain MR image segmentation. J Biomed Phys Eng 4:1 (2014), 13–26.
42. Yushkevich, P.A., Piven, J., Hazlett, H.C., Smith, R.G., Ho, S., Gee, J.C., et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31:3 (2006), 1116–1128.
43. Fang, Q., Boas, D., Tetrahedral mesh generation from volumetric binary and gray scale images. International symposium on biomedical imaging, no. ISBI, 2009, 1563–1569.
44. Maas, S.A., Ellis, B.J., Ateshian, G.A., Weiss, J.A., FEBio: finite elements for biomechanics. J Biomech Eng, 134(1), 2012, 011005.
45. Wu, Y.F., Li, A.Q., Wang, H., Parameter identification of hyperelastic and hyper-viscoelastic models. The 2015 world congress of advances in structural engineering and mechanics, 2015.
46. Mihai, L.A., Chin, L., Janmey, P.A., Goriely, A., A comparison of hyperelastic constitutive models applicable to brain and fat tissues. J R Soc Interface/R Soc, 12(110), 2015, 0486.
47. Franceschini, G., Bigoni, D., Regitnig, P., Holzapfel, G.A., Brain tissue deforms similarly to filled elastomers and follows consolidation theory. J Mech Phys Solids 54:12 (2006), 2592–2620.
48. Bardyn, T., Büchler, P., Patient-specific stress/strain calculations in the tissues, tech. rep., CONTRA CANCRUM – Clinically Oriented Translational Cancer Multilevel Modelling. 2010.
49. Preziosi, L., Vitale, G., Mechanical aspects of tumour growth. New Trends Phys Mech Biol Syst, 2011, 177–228.
50. Twomey, J., Smith, A., Performance measures, consistency, and power for artificial neural network models. Math Comput Model 21:1–2 (1995), 243–258.
51. Lourakis, M.I.a., A brief description of the Levenberg–Marquardt algorithm implemented by levmar. 2005.
52. Burden, F., Winkler, D., Bayesian regularization of neural networks. Methods Mol Biol 458 (2008), 25–44.
53. Foresee, F.D., Hagan, M.T., Gauss–Newton approximation to Bayesian regularization. Proceedings of the 1997 international joint conference on neural networks, 1997, 1930–1935.
54. Smola, A.J., Scholkopf, B., A tutorial on support vector regression. Stat Comput 14:3 (2004), 199–222.
55. Welling, M., Support vector regression. Lecture notes in computer science. 2005, University of California, Irvine.
56. Mollemans, W., Tetrahedral mass spring model for fast soft tissue deformation. Surg Simul Soft Tissue Model, 2003, 145–154.
57. Nguyen, D., Widrow, B., Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights. 1990 IJCNN international joint conference on neural networks, 1990, 1990, IEEE, 21–26.
58. Marescaux, J., Diana, M., Next step in minimally invasive surgery: hybrid image-guided surgery. J Pediatr Surg 50:1 (2014), 30–36.
59. Mahvash, M., Tabrizi, L.B., A novel augmented reality system of image projection for image-guided neurosurgery. Acta Neurochir 155:5 (2013), 943–947.
60. Sielhorst, T., Feuerstein, M., Navab, N., Advanced medical displays: a literature review of augmented reality. IEEE/OSA J Display Technol 4:4 (2008), 451–467.
61. Feng, Y., Okamoto, R.J., Namani, R., Genin, G.M., Bayly, P.V., Measurements of mechanical anisotropy in brain tissue and implications for transversely isotropic material models of white matter. J Mech Behav Biomed Mater 23 (2013), 117–132.
62. Zhang, W., Zhang, R.R., Wu, F., Feng, L.L., Yu, S.-B., Wu, C.W., Differences in the viscoelastic features of white and grey matter in tension. J Biomech 49:16 (2016), 3990–3995.
63. Jin, X., Zhu, F., Mao, H., Shen, M., Yang, K.H., A comprehensive experimental study on material properties of human brain tissue. J Biomech 46:16 (2013), 2795–2801.
64. Nimsky, C., Ganslandt, O., Hastreiter, P., Fahlbusch, R., Intraoperative compensation for brain shift. Surg Neurol 56:6 (2001), 357–364.
65. Chabanas, M., Luboz, V., Payan, Y., Patient specific finite element model of the face soft tissues for computer-assisted maxillofacial surgery. Med Image Anal 7:2 (2003), 131–151.
66. Portnoy, S., Yarnitzky, G., Yizhar, Z., Kristal, A., Oppenheim, U., Siev-Ner, I., et al. Real-time patient-specific finite element analysis of internal stresses in the soft tissues of a residual limb: a new tool for prosthetic fitting. Ann Biomed Eng 35:1 (2007), 120–135.
67. Al-Shareef, A.J., Abbod, M.F., Neural networks initial weights optimisation. UKSim2010 – UKSim 12th international conference on computer modelling and simulation, 2010, 57–61.
68. Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B., Algorithms for hyper-parameter optimization. Advances in neural information processing systems (NIPS), 2011, 2546–2554.
69. Yang, C.-T., Huang, C.-L., Lin, C.-F., Hybrid CUDA, OpenMP, and MPI parallel programming on multicore GPU clusters. Comput Phys Commun 182:1 (2011), 266–269.
70. Neal, M.L., Kerchoffs, R., Current progress in patient-specific modeling. Brief Bioinform. 11:1 (2010), 111–126, 10.1093/bib/bbp049.