A self-taught artificial agent for multi-physics computational model personalization

Medical Image Analysis

Volumn 34, Issue , 2016, Pages 52-64

  Neumann, Dominik ^a,c   Mansi, Tommaso ^b   Itu, Lucian ^a,d   Georgescu, Bogdan ^b   Kayvanpour, Elham ^e   Sedaghat Hamedani, Farbod ^e   Amr, Ali ^e   Haas, Jan ^e   Katus, Hugo ^e   Meder, Benjamin ^e   Steidl, Stefan ^c   Hornegger, Joachim ^c   Comaniciu, Dorin ^b  

^a SIEMENS AG   (Germany)

^b Medical Imaging Technologies   (United States)

^c UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^d TRANSILVANIA UNIVERSITY OF BRASOV   (Romania)

^e UNIVERSITY HOSPITAL HEIDELBERG   (Germany)

Author keywords

Artificial intelligence; Computational modeling; Model personalization; Reinforcement learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTATION THEORY; COMPUTATIONAL METHODS; COST FUNCTIONS; ELECTROPHYSIOLOGY; HOSPITAL DATA PROCESSING; REINFORCEMENT LEARNING;

ARTIFICIAL AGENTS; CARDIAC ELECTROPHYSIOLOGY; CIRCULATION MODELS; CLINICAL PRACTICES; COMPUTATIONAL MODEL; MODEL INDEPENDENT; OPTIMAL STRATEGIES; PERSONALIZATIONS;

INVERSE PROBLEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; CLINICAL PRACTICE; CONTROLLED STUDY; DECISION MAKING; HEART ELECTROPHYSIOLOGY; HUMAN; MATHEMATICAL MODEL; PATIENT CODING; PERSONALIZATION; PHYSICS; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; COMPUTER SIMULATION; PERSONALIZED MEDICINE; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; PHYSICS; PRECISION MEDICINE; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84964690557     PISSN: 13618415     EISSN: 13618423     Source Type: Journal    
DOI: 10.1016/j.media.2016.04.003     Document Type: Article

References (51)

1. Adam, S., Busoniu, L., Babuska, R., Experience replay for real-time reinforcement learning control. IEEE Sys. Man. Cybern. 42:2 (2012), 201–212.
2. Aguado-Sierra, J., Kerckhoffs, R.C.P., Lionetti, F., Hunt, D., Villongco, C., Gonzales, M., Campbell, S.G., McCulloch, A.D., A computational framework for patient-specific multi-scale cardiac modeling. Kerckhoffs, R.C., (eds.) Patient-Specific Modeling of the Cardiovascular System, 2010, Springer, 203–223.
3. Aguado-Sierra, J., Krishnamurthy, A., Villongco, C., Chuang, J., Howard, E., Gonzales, M.J., Omens, J., Krummen, D.E., Narayan, S., Kerckhoffs, R.C.P., McCulloch, A.D., Patient-specific modeling of dyssynchronous heart failure: a case study. Prog. Biophys. Mol. Bio. 107:1 (2011), 147–155.
4. Arthur, D., Vassilvitskii, S., k-means++: The advantages of careful seeding. ACM-SIAM Symp. Discrete Algorithm., 2007, 1027–1035.
5. Augenstein, K.F., Cowan, B.R., LeGrice, I.J., Nielsen, P.M., Young, A.A., Method and apparatus for soft tissue material parameter estimation using tissue tagged magnetic resonance imaging. J. Biomech. Eng. 127:1 (2005), 148–157.
6. Barreto, A., Precup, D., Pineau, J., Practical kernel-based reinforcement learning. arXiv preprint arXiv:1407.5358, 2014.
7. Bellman, R., Dynamic Programming, 1957, Princeton University Press, Princeton, New Jersey.
8. Bishop, C.M., Pattern Recognition and Machine Learning, 4, 2006, Springer, New York.
9. Chabiniok, R., Moireau, P., Lesault, P.-F., Rahmouni, A., Deux, J.-F., Chapelle, D., Estimation of tissue contractility from cardiac cine-mri using a biomechanical heart model. Biomech. Model. Mech. 11:5 (2012), 609–630.
10. Clayton, R., Bernus, O., Cherry, E., Dierckx, H., Fenton, F., Mirabella, L., Panfilov, A., Sachse, F., Seemann, G., Zhang, H., Models of cardiac tissue electrophysiology: progress, challenges and open questions. Prog. Biophys. Mol. Bio. 104:1 (2011), 22–48.
11. Delingette, H., Billet, F., Wong, K.C.L., Sermesant, M., Rhode, K., Ginks, M., Rinaldi, C.A., Razavi, R., Ayache, N., Personalization of cardiac motion and contractility from images using variational data assimilation. IEEE T. Biomed. Eng. 59:1 (2012), 20–24.
12. Frangi, A.F., Niessen, W.J., Viergever, M., Three-dimensional modeling for functional analysis of cardiac images, a review. IEEE T. Med. Imaging 20:1 (2001), 2–5.
13. Hunter, P.J., Borg, T.K., Integration from proteins to organs: the physiome project. Nat. Rev. Mol. Cell Bio. 4:3 (2003), 237–243.
14. Itu, L., Sharma, P., Georgescu, B., Kamen, A., Suciu, C., Comaniciu, D., Model based non-invasive estimation of PV loop from echocardiography. Proceeding of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2014, Chicago, IL, 6774–6777 http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6945183.
15. Kaelbling, L.P., Littman, M.L., Moore, A.W., Reinforcement learning: A survey. J. Artif. Intell. Res. 4 (1996), 237–285.
16. Kayvanpour, E., Mansi, T., Sedaghat-Hamedani, F., Amr, A., Neumann, D., Georgescu, B., Seegerer, P., Kamen, A., Haas, J., Frese, K.S., Irawati, M., Wirsz, E., King, V., Buss, S., Mereles, D., Zitron, E., Keller, A., Katus, H.A., Comaniciu, D., Meder, B., Towards personalized cardiology: Multi-scale modeling of the failing heart. PLoS ONE, 10(7), 2015, e0134869.
17. Kerckhoffs, R.C.P., Lumens, J., Vernooy, K., Omens, J., Mulligan, L., Delhaas, T., Arts, T., McCulloch, A., Prinzen, F., Cardiac resynchronization: insight from experimental and computational models. Prog. Biophys. Mol. Bio. 97:2 (2008), 543–561.
18. Konukoglu, E., Relan, J., Cilingir, U., Menze, B.H., Chinchapatnam, P., Jadidi, A., Cochet, H., Hocini, M., Delingette, H., Jaïs, P., Haïssaguerre, M., Ayache, N., Sermesant, M., Efficient probabilistic model personalization integrating uncertainty on data and parameters: Application to eikonal-diffusion models in cardiac electrophysiology. Prog. Biophys. Mol. Bio. 107:1 (2011), 134–146.
19. Krishnamurthy, A., Villongco, C.T., Chuang, J., Frank, L.R., Nigam, V., Belezzuoli, E., Stark, P., Krummen, D.E., Narayan, S., Omens, J.H., McCulloch, A.D., Kerckhoffs, R.C.P., Patient-specific models of cardiac biomechanics. J. Comput. Phys. 244 (2013), 4–21.
20. Kristan, M., Leonardis, A., Skočaj, D., Multivariate online kernel density estimation with gaussian kernels. Pattern Recognit. 44:10 (2011), 2630–2642.
21. Kuijpers, N.H., Hermeling, E., Bovendeerd, P.H., Delhaas, T., Prinzen, F.W., Modeling cardiac electromechanics and mechanoelectrical coupling in dyssynchronous and failing hearts. J. Cardiovasc. Transl. Res. 5:2 (2012), 159–169.
22. Kveton, B., Theocharous, G., Kernel-based reinforcement learning on representative states. Association for the Advancement of Artificial Intelligence, 2012, 977–983.
23. Lagarias, J.C., Reeds, J.A., Wright, M.H., Wright, P.E., Convergence properties of the Nelder-Mead simplex method in low dimensions. SIAM J. Optimiz. 9:1 (1998), 112–147.
24. Le Folgoc, L., Delingette, H., Criminisi, A., Ayache, N., Current-based 4D shape analysis for the mechanical personalization of heart models. Medical Computer Vision. Recognition Techniques and Applications in Medical Imaging LNCS, 7766, 2013, Springer, 283–292.
25. Lin, L.-J., Reinforcement learning for robots using neural networks. Technical Report, 1993, DTIC Document.
26. Marchesseau, S., Delingette, H., Sermesant, M., Cabrera-Lozoya, R., Tobon-Gomez, C., Moireau, P., Figueras i Ventura, R.M., Lekadir, K., Hernandez, A., Garreau, M., Donal, E., Leclercq, C., Duckett, S.G., Rhode, K., Rinaldi, C.A., Frangi, A.F., Razavi, R., Chapelle, D., Ayache, N., Personalization of a cardiac electromechanical model using reduced order unscented kalman filtering from regional volumes. Med. Image Anal. 17:7 (2013), 816–829.
27. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumara, D., Wierstra, D., Legg, S., Hassabis, D., Human-level control through deep reinforcement learning. Nature 518:7540 (2015), 529–533.
28. Mülling, K., Kober, J., Kroemer, O., Peters, J., Learning to select and generalize striking movements in robot table tennis. Int. J. Robot. Res. 32:3 (2013), 263–279.
29. Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., Panneershelvam, V., Suleyman, M., Beattie, C., Petersen, S., Legg, S., Mnih, V., Kavukcuoglu, K., Silver, D., 2015. Massively parallel methods for deep reinforcement learning. arXiv: 1507.04296.
30. Neumann, D., Mansi, T., Georgescu, B., Kamen, A., Kayvanpour, E., Amr, A., Sedaghat-Hamedani, F., Haas, J., Katus, H., Meder, B., Hornegger, J., Comaniciu, D., Robust image-based estimation of cardiac tissue parameters and their uncertainty from noisy data. MICCAI LNCS, 8674, 2014, Springer, 9–16.
31. Neumann, D., Mansi, T., Grbic, S., Voigt, I., Georgescu, B., Kayvanpour, E., Amr, A., Sedaghat-Hamedani, F., Haas, J., Katus, H., Meder, B., Hornegger, J., Kamen, A., Comaniciu, D., Automatic image-to-model framework for patient-specific electromechanical modeling of the heart. IEEE Int. Symp. Biomed. Imaging, 2014, 935–938.
32. Neumann, D., Mansi, T., Itu, L., Georgescu, B., Kayvanpour, E., Sedaghat-Hamedani, F., Haas, J., Katus, H., Meder, B., Steidl, S., Hornegger, J., Comaniciu, D., Vito – a generic agent for multi-physics model personalization: Application to heart modeling. MICCAI LNCS, 9350, 2015, Springer, 442–449.
33. Nguyen-Tuong, D., Peters, J., Model learning for robot control: a survey. Cogn. Process. 12:4 (2011), 319–340.
34. Noble, D., Modeling the heart – from genes to cells to the whole organ. Science 295:5560 (2002), 1678–1682.
35. Powell, M.J., The BOBYQA algorithm for bound constrained optimization without derivatives. NA Report NA2009/06, 2009, Cambridge.
36. Prakosa, A., Sermesant, M., Allain, P., Villain, N., Rinaldi, C., Rhode, K., Razavi, R., Delingette, H., Ayache, N., Cardiac electrophysiological activation pattern estimation from images using a patient-specific database of synthetic image sequences. IEEE T. Biomed. Eng, 2013.
37. Rosenbrock, H., An automatic method for finding the greatest or least value of a function. Comput. J. 3:3 (1960), 175–184.
38. Schmid, H., Nash, M., Young, A., Hunter, P., Myocardial material parameter estimation – a comparative study for simple shear. J. Biomech. Eng. 128:5 (2006), 742–750.
39. Seegerer, P., Mansi, T., Jolly, M.-P., Neumann, D., Georgescu, B., Kamen, A., Kayvanpour, E., Amr, A., Sedaghat-Hamedani, F., Haas, J., Katus, H., Meder, B., Comaniciu, D., Estimation of regional electrical properties of the heart from 12-lead ECG and images. Statistical Atlases and Computational Models of the Heart – Imaging and Modelling Challenges LNCS, 8896, 2015, Springer, 204–212.
40. Sermesant, M., Billet, F., Chabiniok, R., Mansi, T., Chinchapatnam, P., Moireau, P., Peyrat, J.-M., Rhode, K., Ginks, M., Lambiase, P., Arridge, S., Delingette, H., Sorine, M., Rinaldi, C.A., Chapelle, D., Razavi, R., Ayache, N., Personalised electromechanical model of the heart for the prediction of the acute effects of cardiac resynchronisation therapy. Functional Imaging and Modeling of the Heart LNCS, 5528, 2009, Springer, 239–248.
41. Sutton, R.S., Barto, A.G., Reinforcement Learning: An Introduction, 1, 1998, MIT press Cambridge, London, England.
42. Tesauro, G., Td-gammon, a self-teaching Backgammon program, achieves master-level play. Neural Comput. 6:2 (1994), 215–219.
43. Wallman, M., Smith, N.P., Rodriguez, B., A comparative study of graph-based, eikonal, and monodomain simulations for the estimation of cardiac activation times. IEEE T. Biomed. Eng. 59:6 (2012), 1739–1748.
44. Wallman, M., Smith, N.P., Rodriguez, B., Computational methods to reduce uncertainty in the estimation of cardiac conduction properties from electroanatomical recordings. Med. Image Anal. 18:1 (2014), 228–240.
45. Wang, V.Y., Lam, H., Ennis, D.B., Cowan, B.R., Young, A.A., Nash, M.P., Modelling passive diastolic mechanics with quantitative mri of cardiac structure and function. Med. Image. Anal. 13:5 (2009), 773–784.
46. Wang, Y., Georgescu, B., Chen, T., Wu, W., Wang, P., Lu, X., Ionasec, R., Zheng, Y., Comaniciu, D., Learning-based detection and tracking in medical imaging: a probabilistic approach. Deformation Models LNCVB, 7, 2013, Springer, 209–235.
47. Westerhof, N., Elzinga, G., Sipkema, P., An artificial arterial system for pumping hearts. J. Appl. Physiol. 31:5 (1971), 776–781.
48. Wong, K.C., Sermesant, M., Rhode, K., Ginks, M., Rinaldi, C.A., Razavi, R., Delingette, H., Ayache, N., Velocity-based cardiac contractility personalization from images using derivative-free optimization. J. Mech. Behav. Biomed. 43 (2015), 35–52.
49. Xi, J., Lamata, P., Niederer, S., Land, S., Shi, W., Zhuang, X., Ourselin, S., Duckett, S.G., Shetty, A.K., Rinaldi, C.A., Rueckert, D., Razavi, R., Smith, N.P., The estimation of patient-specific cardiac diastolic functions from clinical measurements. Med. Image. Anal. 17:2 (2013), 133–146.
50. Zettinig, O., Mansi, T., Neumann, D., Georgescu, B., Rapaka, S., Seegerer, P., Kayvanpour, E., Sedaghat-Hamedani, F., Amr, A., Haas, J., Steen, H., Katus, H., Meder, B., Navab, N., Kamen, A., Comaniciu, D., Data-driven estimation of cardiac electrical diffusivity from 12-lead ECG signals. Med. Image Anal., 2014, 1361–1376.
51. Zheng, Y., Barbu, A., Georgescu, B., Scheuering, M., Comaniciu, D., Four-chamber heart modeling and automatic segmentation for 3-D cardiac CT volumes using marginal space learning and steerable features. IEEE T. Med. Imaging 27:11 (2008), 1668–1681.