Chaste: A case study of parallelisation of an open source finite-element solver with applications to computational cardiac electrophysiology simulation

International Journal of High Performance Computing Applications

Volumn 28, Issue 1, 2014, Pages 13-32

  Bernabeu, Miguel O ^a   Southern, James ^b   Wilson, Nicholas ^b   Strazdins, Peter ^c   Cooper, Jonathan ^d   Pitt Francis, Joe ^d  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b Fujitsu Laboratories of Europe Ltd   (United Kingdom)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

bidomain equations; Cardiac electrophysiology; finite element method; hybrid linear solver; parallelisation; scaling analysis

Indexed keywords

BIDOMAIN EQUATION; CARDIAC ELECTROPHYSIOLOGY; LINEAR SOLVER; PARALLELISATION; SCALING ANALYSIS;

ELECTROPHYSIOLOGY; HEART; LINEAR SYSTEMS; MEDICAL IMAGING; NEUROLOGY; OPEN SYSTEMS; PATIENT TREATMENT;

FINITE ELEMENT METHOD;

EID: 84892934034     PISSN: 10943420     EISSN: 17412846     Source Type: Journal    
DOI: 10.1177/1094342012474997     Document Type: Article

References (67)

1. Amdahl GM spring joint computer conference (AFIPS '67). New York: ACM Press ; 1967: 483.
2. Balay S, Brown J, Buschelman K, Eijkhout V, Gropp WD, Kaushik D, et al USA: Argonne National Laboratory ; 2010:
3. Barrett R, Berry M, Chan TF, Demmel J, Donato J, Dongarra J, et al Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. Philadelphia: SIAM ; 1994:
4. Bekas C, Curioni A, Arbenz P, Flaig C, Van Lenthe GH, Müller R, et al. Extreme scalability challenges in micro-finite element simulations of human bone. Concurrency and Computation: Practice and Experience. 2010 ; 22 (16). 2282-2296
5. Bernabeu MO, Bishop MJ, Pitt-Francis J, Gavaghan D, Grau V, Rodriguez B computers in cardiology. Piscataway: IEEE Press ; 2008: 721-724.
6. Bernabeu MO, Corrias A, Pitt-Francis J, Rodriguez B, Bethwaite B, Enticott C, et al computers in cardiology. Piscataway: IEEE Press ; 2009: 213-216.
7. Bernabeu MO, Kay D. Scalable parallel preconditioners for an open source cardiac electrophysiology simulation package. Procedia Computer Science. 2011 ; 4: 821-830
8. Bernabeu MO, Pathmanathan P, Pitt-Francis J, Kay D. Stimulus protocol determines the most computationally efficient preconditioner for the bidomain equations. IEEE Transactions on Biomedical Engineering. 2010a ; 57 (12). 2806-2815
9. Bernabeu MO, Wallman M, Rodriguez B IEEE annual international conference of the engineering in medicine and biology society (EMBC). Piscataway: IEEE Press ; 2010 b: 760-763.
10. Bernus O, Wilders R, Zemlin CW, Verschelde H, Panfilov AV. A computationally efficient electrophysiological model of human ventricular cells. American Journal of Physiology: Heart and Circulatory Physiology. 2002 ; 282 (6). 2296-2308
11. Bishop MJ, Plank G. Representing cardiac bidomain bath-loading effects by an augmented monodomain approach: Application to complex ventricular models. IEEE Transactions on Biomedical Engineering. 2011 ; 58 (4). 1066-1075
12. Bishop MJ, Plank G, Burton RA, Schneider JE, Gavaghan DJ, Grau V, et al. Development of an anatomically-detailed MRI-derived rabbit ventricular model and assessment of its impact on simulation of electrophysiological function. American Journal of Physiology: Heart and Circulatory Physiology. 2009 ; 298 (2). 699-718
13. Bordas R, Carpentieri B, Fotia G, Maggio F, Nobes R, Pitt-Francis J, et al. Simulation of cardiac electrophysiology on next-generation high-performance computers. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2009 ; 367 (1895). 1951-1969
14. Burton R, Plank G, Schneider J, Grau V, Ahammer H. Keeling S et al. (2006) 3-D models of individual cardiac histo-anatomy: tools and challenges. Annals of the New York Academy of Sciences. 1380 ;: 301-319
15. Cai X, Lines GT 6th international conference on applied parallel computing advanced scientific computing (PARA '02). London: Springer-Verlag ; 2002: 3-17.
16. Calvetti D, Golub G, Reichel L. An adaptive Chebyshev iterative method for nonsymmetric linear systems based on modified moments. Numerische Mathematik. 1994 ; 67: 21-40
17. Calvetti D, Reichel L. A hybrid iterative method for symmetric positive definite linear systems. Numerical Algorithms. 1996 ; 11: 79-98
18. Clayton R, Bernus O, Cherry E, Dierckx H, Fenton F, Mirabella L, et al. Models of cardiac tissue electrophysiology: Progress, challenges and open questions. Progress in Biophysics and Molecular Biology. 2011 ; 104 ((1-3)). 22-48
19. Colli-Franzone P, Pavarino LF. A parallel solver for reaction-diffusion systems in computational electrocardiology. Mathematical Models and Methods in Applied Sciences. 2004 ; 14 (6). 883-912
20. Cooper J Automatic validation and optimisation of biological models. University of Oxford, UK: ; 2009:
21. Devine KD, Boman EG, Heaphy RT, Hendrickson BA, Teresco JD, Faik J, et al. New challenges in dynamic load balancing. Applied Numerical Mathematics. 2005 ; 52 ((2-3)). 133-152
22. Devine KD, Boman EG, Karypis G Frontiers of Scientific Computing. Heroux M Raghavan A Simon H, ed. Philadelphia: SIAM ; 2006:
23. dos Santos R, Plank G, Bauer S, Vigmond E. Parallel multigrid preconditioner for the cardiac bidomain model. IEEE Transactions on Biomedical Engineering. 2004 ; 51 (11). 1960-1968
24. Fishler MG, Thakor NV. A massively parallel computer model of propagation through a two-dimensional cardiac syncytium. Pacing and Clinical Electrophysiology. 1991 ; 14 (11). 1694-1699
25. FitzHugh R. Impulses and physiological states in theoretical models of nerve membrane. Biophysical Journal. 1961 ; 1 (6). 445-466
26. Golub GH, Van Loan CF Matrix Computations. Baltimore: Johns Hopkins University Press ; 1996:
27. Gutknecht MH, Röllin S. The Chebyshev iteration revisited. Journal of Parallel Computing. 2002 ; 28 (2). 263-283
28. Iyer V, Hajjar R, Armoundas A. Mechanisms of abnormal calcium homeostasis in mutations responsible for catecholaminergic polymorphic ventricular tachycardia. Circulation Research. 2007 ; 100 (2). e22-31
29. Keener J, Sneyd J Mathematical Physiology. New York: Springer ; 1998:
30. Keener JP, Bogar K. A numerical method for the solution of the bidomain equations in cardiac tissue. Chaos. 1998 ; 8 (1). 234
31. Linge S, Sundnes J, Hanslien M, Lines G, Tveito A. Numerical solution of the bidomain equations. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2009 ; 367 (1895). 1931-1950
32. Circulation Research.:
33. Manteuffel TA. The Tchebychev iteration for nonsymmetric linear systems. Numerische Mathematik. 1977 ; 28: 307-327
34. Murillo M, Cai XC. A fully implicit parallel algorithm for simulating the non-linear electrical activity of the heart. Numerical Linear Algebra with Applications. 2004 ; 11 ((2-3)). 261-277
35. Ng KT, Hutchinson SA, Gao S. Numerical analysis of electrical defibrillation: The parallel approach. Journal of Electrocardiology. 1995 ; 28 (1). 15-20
36. Niederer S, Mitchell L, Smith N, Plank G. Simulating human cardiac electrophysiology on clinical time-scales. Frontiers in Physiology. 2011 ; 2: 14
37. Pathmanathan P, Bernabeu MO, Bordas R, Cooper J, Garny A, Pitt-Francis JM, et al. A numerical guide to the solution of the bidomain equations of cardiac electrophysiology. Progress in Biophysics and Molecular Biology. 2010 ; 102 ((2-3)). 136-155
38. Pavarino LF, Scacchi S. Multilevel additive schwarz preconditioners for the bidomain reaction-diffusion system. SIAM Journal on Scientific Computing. 2008 ; 31 (1). 420-443
39. Pennacchio M, Simoncini V. Algebraic multigrid preconditioners for the bidomain reaction-diffusion system. Applied Numerical Mathematics. 2009 ; 59 (12). 3033-3050
40. Piggott MD, Gorman GJ, Pain CC, Allison PA, Candy AS, Martin BT, et al. A new computational framework for multi-scale ocean modelling based on adapting unstructured meshes. International Journal for Numerical Methods in Fluids. 2008 ; 56 (8). 1003-1015
41. Pitt-Francis J, Pathmanathan P, Bernabeu MO, Bordas R, Cooper J, Fletcher AG, et al. Chaste: A test-driven approach to software development for biological modelling. Computer Physics Communications. 2009 ; 180 (12). 2452-2471
42. Plank G, Liebmann M, dos Santos RW, Vigmond EJ, Haase G. Algebraic multigrid preconditioner for the cardiac bidomain model. IEEE Transactions on Biomedical Engineering. 2007 ; 54: 585-596
43. Pollard A, Barr R. Computer simulations of activation in an anatomically based model of the human ventricular conduction system. IEEE Transactions on Biomedical Engineering. 1991 ; 38 (10). 982-996
44. Pope B, Fitch B, Pitman M, Rice J, Reumann M. Performance of hybrid programming models for multiscale cardiac simulations: Preparing for petascale computation. IEEE Transactions on Biomedical Engineering. 2011 ; 58 (10). 2965-2969
45. Porras D, Rogers J, Smith W, Pollard A. Distributed computing for membrane-based modeling of action potential propagation. IEEE Transactions on Biomedical Engineering. 2000 ; 47 (8). 1051-1057
46. Potse M, Dubé B, Richer J, Vinet A, Gulrajani RM. A comparison of monodomain and bidomain reaction-diffusion models for action potential propagation in the human heart. IEEE Transactions on Biomedical Engineering. 2006 ; 53 (12). 2425-2435
47. Quan W, Evans S, Hastings H. Efficient integration of a realistic two-dimensional cardiac tissue model by domain decomposition. IEEE Transactions on Biomedical Engineering. 1998 ; 45 (3). 372-385
48. Reumann M, Fitch B, Rayshubskiy A, Keller D, Seemann G, Dossel O, et al IEEE annual international conference of the engineering in medicine and biology society (EMBC). Piscataway: IEEE Press ; 2009: 2795-2798.
49. Sahni O, Zhou M, Shephard MS, Jansen KE 22nd conference on high performance computing networking, storage and analysis (SC '09). New York ACM Press ; 2009:
50. Saleheen H, Claessen P, Ng K. Three-dimensional finite-difference bidomain modeling of homogeneous cardiac tissue on a data-parallel computer. IEEE Transactions on Biomedical Engineering. 1997 ; 44 (2). 200-204
51. Schloegel K, Karypis G, Kumar V. Parallel static and dynamic multi-constraint graph partitioning. Concurrency and Computation: Practice and Experience. 2002 ; 14 (3). 219-240
52. Shadid J, Pawlowski R, Banks J, Chacón L, Lin P, Tuminaro R. Towards a scalable fully-implicit fully-coupled resistive MHD formulation with stabilized FE methods. Journal of Computational Physics. 2010 ; 229 (20). 7649-7671
53. Southern J, Bernabeu MO, Cooper J, Wilson N, Pitt-Francis J international supercomputing conference (ISC'11). Germany: Hamburg ; 2011:
54. Tallent NR, Mellor-Crummey JM, Adhianto L, Fagan MW, Krentel M 22nd conference on high performance computing networking, storage and analysis (SC '09). New York: ACM Press ; 2009:
55. ten Tusscher KH, Hren R, Panfilov AV. Organization of ventricular fibrillation in the human heart. Circulation Research. 2007 ; 100 (12). e87 - e101
56. ten Tusscher KHWJ, Noble D, Noble PJ, Panfilov AV. A model for human ventricular tissue. American Journal of Physiology: Heart and Circulatory Physiology. 2004 ; 286 (4). 1573-1589
57. ten Tusscher KHWJ, Panfilov AV. Alternans and spiral breakup in a human ventricular tissue model. American Journal of Physiology: Heart and Circulatory Physiology. 2006 ; 291 (3). 1088-1100
58. Teresco JD, Devine KD, Flaherty JE Numerical Solution of Partial Differential Equations on Parallel Computers. Bruaset AM Tveito A Barth TJ Griebel M Keyes DE Nieminen RM Roose D Schlick T, ed. Berlin: Springer ; 2006: 55-88.
59. Vázquez M, Arís R, Houzeaux G, Aubry R, Villar P, Garcia-Barnés J, et al. A massively parallel computational electrophysiology model of the heart. International Journal for Numerical Methods in Biomedical Engineering. 2011 ; 27 (12). 1911-1929
60. Vetter FJ, McCulloch AD. Three-dimensional analysis of regional cardiac function: a model of rabbit ventricular anatomy. Progress in Biophysics and Molecular Biology. 1998 ; 69 ((2-3)). 157-183
61. Vigmond EJ, Aguel F, Trayanova NA. Computational techniques for solving the bidomain equations in three dimensions. IEEE Transactions on Biomedical Engineering. 2002 ; 49 (11). 1260-1269
62. Vigmond EJ, Hughes M, Plank G, Leon LJ. Computational tools for modeling electrical activity in cardiac tissue. Journal of Electrocardiology. 2003 ; 36 (1). 69-74
63. Wathen AJ. An analysis of some element-by-element techniques. Computer Methods in Applied Mechanics and Engineering. 1989 ; 74 (3). 271-287
64. Whiteley J. An efficient numerical technique for the solution of the monodomain and bidomain equations. IEEE Transactions on Biomedical Engineering. 2006 ; 53: 2139-2147
65. Winslow RL, Kimball AL, Varghese A, Noble D. Simulating cardiac sinus and atrial network dynamics on the connection machine. Physica D: Nonlinear Phenomena. 1993 ; 64 ((1-3)). 281-298
66. Zemzemi N, Bernabeu MO, Saiz J, Rodriguez B Functional Imaging and Modeling of the Heart. Metaxas DN Axel L, ed. New York: Springer ; 2011: 259-266.
67. Zhou M, Sahni O, Xie T, Shephard M, Jansen K. Unstructured mesh partition improvement for implicit finite element at extreme scale. The Journal of Supercomputing. 2012 ; 59: 1218-1228