A physics-driven Neural Networks-based Simulation System (PhyNNeSS) for multimodal interactive virtual environments involving nonlinear deformable objects

Presence: Teleoperators and Virtual Environments

Volumn 20, Issue 4, 2011, Pages 289-308

  De, Suvranu ^a   Deo, Dhannanjay ^b   Sankaranarayanan, Ganesh ^a   Arikatla, Venkata S ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b KITWARE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL MODELING; DEFORMABLE OBJECT; DEFORMATION FIELD; FINITE ELEMENT MODELS; FORCE FEEDBACK; GENERAL SOLUTIONS; HAPTICS; HIGH RATE; INTERACTION FORCES; INTERACTIVE SIMULATIONS; INTERACTIVE VIRTUAL ENVIRONMENTS; LAPAROSCOPIC SURGERY; MULTI-MODAL; NONLINEAR DEFORMATIONS; NONLINEAR MATERIAL PROPERTIES; OFFLINE; PHYSICS-BASED; PHYSICS-BASED MODELING; PRE-COMPUTATION; PRESCRIBED DISPLACEMENTS; REAL TIME; REAL TIME GRAPHICS; REAL TIME SIMULATIONS; REAL-TIME COMPUTATIONS; REALISTIC SIMULATION; SIMULATION SYSTEMS; SOFTWARE FRAMEWORKS; SURGICAL SIMULATION; TECHNICAL CHALLENGES; TRAINING TOOLS;

COMPUTER SOFTWARE; DEFORMATION; ERROR ANALYSIS; FINITE ELEMENT METHOD; RADIAL BASIS FUNCTION NETWORKS; REAL TIME SYSTEMS; SURGERY; TRANSIENTS; USER INTERFACES; VIRTUAL REALITY;

NEURAL NETWORKS;

EID: 80555136822     PISSN: 10547460     EISSN: 15313263     Source Type: Journal    
DOI: 10.1162/PRES_a_00054     Document Type: Article

References (58)

1. Abe, Y., & Iiguni, Y. (2006). Interpolation capability of the periodic radial basis function network. Proceedings of Vision Image and Signal Processing, 153(6), 785-794.
2. Ahn, B., & Kim, J. (2010). Measurement and characterization of soft tissue behavior with surface deformation and force response under large deformations. Medical Image Analysis, 14(2), 138-148. doi:10.1016/j.media.2009.10.006
3. Allin, S., Matsuoka, Y., & Klatzky, R. (2002). Measuring just noticeable differences for haptic force feedback: Implications for rehabilitation. In Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, HAPTICS 2002 (pp. 299-302).
4. Barbič, J., & James, D. L. (2007). Time-critical distributed contact for 6-DOF haptic rendering of adaptively sampled reduced deformable models. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA'07, 171-180.
5. Basdogan, C., De, S., Kim, J., Muniyandi, M., Kim, H., & Srinivasan, M. (2004). Haptics in minimally invasive surgical simulation and training. IEEE Computer Graphics and Applications, 24(2), 56-64.
6. Bathe, K. (1996). Finite element procedures (Unabridged paperback reprint ed.). Englewood Cliffs, NJ: Prentice Hall.
7. Bianchi, G., Solenthaler, B., Székely, G., & Harders, M. (2004). Simultaneous topology and stiffness identification for mass-spring models based on FEM reference deformations. In Medical Image Computing and Computer-Assisted Intervention, MICCAI, 293-301.
8. Bickel, B., Bächer, M., Otaduy, M. A., Matusik, W., Pfister, H., & Gross, M. (2009). Capture and modeling of nonlinear heterogeneous soft tissue. ACM Transactions on Graphics, 28(3), 1-9.
9. Bonet, J., & Wood, R. D. (1999). Nonlinear continuum mechanics for finite element analysis (Repr.). Cambridge, UK: Cambridge University Press.
10. Bro-Nielsen, M. (1998). Finite element modeling in surgery simulation. Proceedings of the IEEE, 86(3), 490-503.
11. Carr, J. C., Beatson, R. K., Cherrie, J. B., Mitchell, T. J., Fright, W. R., McCallum, B. C., et al. (2001). Reconstruction and representation of 3D objects with radial basis functions. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '01, (pp. 67-76). doi:10.1145/383259.383266
12. Chen, S., Billings, S. A., & Luo, W. (1989). Orthogonal leastsquares methods and their application to non-linear system identification. International Journal of Control, 50(5), 1873-1896.
13. De, S., Lim, Y., Manivannan, M., & Srinivasan, M. A. (2006). Physically realistic virtual surgery using the point-associated finite field (PAFF) approach. Presence: Teleoperators and Virtual Environments, 15(3), 294-308.
14. de Boer, A., van der Schoot, M. S., & Bijl, H. (2007). Mesh deformation based on radial basis function interpolation. Computers & Structures, 85(11-14), 784-795.
15. Delp, S., Loan, P., & Basdogan, C. (1997). Surgical simulation: An emerging technology for training in emergency medicine. Presence: Teleoperators and Virtual Environments, 6(2), 147-159.
16. Deo, D., & De, S. (2009). PhyNNeSS: A Physics-driven neural networks-based surgery simulation system with force feedback. In Proceedings of the Third Joint EuroHaptics Conference and the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, World Haptics 2009, 30-34.
17. Edwards, S., & Tabor, D. (1976). The theory of rubber elasticity (and discussion). Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 351(1666), 397-406.
18. Grzeszczuk, R., Terzopoulos, D., & Hinton, G. (1998). NeuroAnimator: Fast neural network emulation and control of physics-based models. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98).
19. Halic, T., Arikatla, V. S., Sankaranarayanan, G., Lu, Z., Ahn, W., & De, S. (2011). A software framework for multimodal interactive simulations (SoFMIS). Studies in Health Technology and Informatics, 163, 213-217.
20. Hambli, R., Chamekh, A., & Bel Hadj Salah, H. (2006). Realtime deformation of structure using finite element and neural networks in virtual reality applications. Finite Elements in Analysis and Design, 42(11), 985-991.
21. Harders, M., Bajka, M., Spaelter, U., Tuchschmid, S., Bleuler, H., & Szekely, G. (2006). Highly-realistic, immersive training environment for hysteroscopy. Studies in Health Technology and Informatics, 119, 176-181.
22. Ho, C., Basdogan, C., & Srinivasan, M. A. (1999). Efficient point-based rendering techniques for haptic display of virtual objects. Presence: Teleoperators and Virtual Environments, 8(5), 477-491.
23. Höver, R., Kósa, G., Székely, G., & Harders, M. (2009). Data-driven haptic rendering-From viscous fluids to visco-elastic solids. IEEE Transactions on Haptics, 2, 15-27. doi:10.1109/TOH.2009.2
24. Jung, S., & Ghaboussi, J. (2006). Neural network constitutive model for rate-dependent materials. Computers & Structures, 84(15-16), 955-963.
25. Kallassy, A. (2003). A new neural network for response estimation. Computers & Structures, 81(26-27), 2417-2429.
26. Kauer, M., Vuskovic, V., Dual, J., Szekely, G., & Bajka, M. (2001). Inverse finite element characterization of soft tissues. In W. Niessen & M. Viergever (Eds.), Medical Image Computing and Computer-Assisted Intervention-MICCAI 2001, LNCS Computer Science (Vol. 2208, pp. 128-136). Berlin: Springer.
27. Kuehnapfel, U. G., & Neisius, B. (1993). CAD-based graphical computer simulation in endoscopic surgery. Endoscopic Surgery and Allied Technologies, 1(3), 181-184.
28. Kuroe, Y., & Kawakami, H. (2007). Vector field approximation by model inclusive learning of neural networks. In Artificial Neural Networks-ICANN 2007 (pp. 717-726).
29. Lake, G. J., & Thomas, A. G. (1967). The strength of highly elastic materials. Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences, 300(1460), 108-119.
30. Lim, Y., & De, S. (2007). Real time simulation of nonlinear tissue response in virtual surgery using the point collocationbased method of finite spheres. Computer Methods in Applied Mechanics and Engineering, 196(31-32), 3011-3024.
31. Lister, K., & Desai, J. P. (2008). Soft-tissue characterization during monopolar electrocautery procedures. Studies in Health Technology and Informatics, 132, 254-256.
32. Liu, Y., & De, S. (2008). CUDA-based real time surgery simulation. Studies in Health Technology and Informatics, 132, 260-262.
33. Loop, C. (n.d.). Smooth subdivision surfaces based on triangles (Master's thesis). University of Utah.
34. Maciel, A., & De, S. (2008). Physics-based real time laparoscopic electrosurgery simulation. Studies in Health Technology and Informatics, 132, 272-274.
35. Miller, K. (2005). Method of testing very soft biological tissues in compression. Journal of Biomechanics, 38(1), 153-158. doi:10.1016/j.jbiomech.2004.03.004
36. Mullur, A. A., & Messac, A. (2006). Metamodeling using extended radial basis functions: A comparative approach. Engineering with Computers, 21(3), 203-217.
37. Nava, A., Mazza, E., Furrer, M., Villiger, P., & Reinhart, W. (2008). In vivo mechanical characterization of human liver. Medical Image Analysis, 12(2), 203-216.
38. NVIDIA Corporation. (n.d.). CUDA Zone (Vol. 2009). NVIDIA Corporation.
39. Omar, T., Eskandarian, A., & Bedewi, N. (1998). Vehicle crash modelling using recurrent neural networks. Mathematical and Computer Modelling, 28(9), 31-42.
40. Orr, M. J. L. (1995). Regularization in the selection of radial basis function centers. Neural Computation, 7(3), 606-623.
41. Pai, D. K., Doel, K. V. D., James, D. L., Lang, J., Lloyd, J. E., Richmond, J. L., et al. (2001). Scanning physical interaction behavior of 3D objects. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), 87-96.
42. Park, J., & Sandberg, I. W. (1991). Universal approximation using radial-basis-function networks. Neural Computation, 3(2), 246-257.
43. Park, J., & Sandberg, I. W. (1993). Approximation and radialbasis-function networks. Neural Computation, 5(2), 305-316.
44. Peterlik, I., & Matyska, L. (2007). An algorithm of state-space precomputation allowing non-linear haptic deformation modelling using finite element method. In Proceedings of the Second Joint EuroHaptics Conference and the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, World Haptics 2007, 231-236.
45. Picinbono, G., Delingette, H., & Ayache, N. (2003). Nonlinear anisotropic elasticity for real-time surgery simulation. Graphical Models, 65(5), 305-321.
46. Pongrac, H., Färber, B., Hinterseer, P., Kammerl, J., & Steinbach, E. (2006). Limitations of human 3D force discrimination. In Human-Centered Robotics Systems 2006.
47. Radetzky, A., Nürnberger, A., & Pretschner, D. P. (1998). Simulation of elastic tissues in virtual medicine using neurofuzzy systems. Medical Imaging: Image Display. Proceedings of the International Society for Optics and Photonics (SPIE), Vol. 3335, 399-409.
48. Samarasinghe, S. (2006). Neural networks for applied sciences and engineering. Boston: Auerbach Publications.
49. Sankaranarayanan, G., Deo, D., & De, S. (2009). Hybrid network architecture for interactive multi-user surgical simulator with scalable deformable models. Studies in Health Technology and Informatics, 142, 292-294.
50. Saupin, G., Duriez, C., & Cotin, S. (2008). Contact model for haptic medical simulations. In Biomedical Simulation (pp. 157-165).
51. Sherstinsky, A., & Picard, R. W. (1996). On the efficiency of the orthogonal least squares training method for radial basis function networks. IEEE Transactions on Neural Networks, 7(1), 195-200.
52. Székely, G., Brechbühler, C., Hutter, R., Rhomberg, A., Ironmonger, N., & Schmid, P. (2000). Modelling of soft tissue deformation for laparoscopic surgery simulation. Medical Image Analysis, 4(1), 57-66. doi:10.1016/S1361-8415(00)00002-5
53. Tan, H., Radcliffe, J., Ga, B. N., Tan, H. Z., Eberman, B., Srinivasan, M. A., et al. (1994). Human factors for the design of force-reflecting haptic interfaces: Proceedings of the American Society of Mechanical Engineers (ASME) Dynamic Systems and Control, 55(1), 353-359.
54. Valdes, J. L., Biscay, R., & Jimenez, J. C. (1999). Geometric selection of centers for radial basis function approximations involved in intensive computer simulations. Mathematics and Computers in Simulation, 48(3), 295-306.
55. Waszczyszyn, Z., & Ziemianski, L. (2001). Neural networks in mechanics of structures and materials-New results and prospects of applications. Computers & Structures, 79(22-25), 2261-2276.
56. Weiguo, S., & Kui, Y. (2005). Haptic modeling for liver cutting based on fuzzy neural network. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, (pp. 1216-1220).
57. Yu, D. L., Gomm, J. B., & Williams, D. (1997). A recursive orthogonal least squares algorithm for training RBF networks. Neural Processing Letters, 5(3), 167-176.
58. Zhong, Y. M., Shirinzadeh, B., Alici, G., & Smith, J. (2006). A cellular neural network methodology for deformable object simulation. IEEE Transactions on Information Technology in Biomedicine, 10(4), 749-762.