Semi-automatic porting of a large-scale CFD code to multi-graphics processing unit clusters

International Journal for Numerical Methods in Fluids

Volumn 69, Issue 11, 2012, Pages 1786-1796

  Corrigan, Andrew ^a   Löhner, Rainald ^a  

^a GEORGE MASON UNIVERSITY   (United States)

Author keywords

CFD; GPUs; Supercomputing hardware

Indexed keywords

2 PERFORMANCE; CFD CODES; COARSE-GRAINED; EDGE-BASED; FINE-GRAINED PARALLELISM; FINITE ELEMENT FORMULATIONS; GPUS; GRAPHICS HARDWARE; GRAPHICS PROCESSING UNIT; INTERFACE CODE; LEGACY CODE; MANY-CORE; MEMORY ACCESS; MEMORY LAYOUT; PROCESSING UNITS; SANTA CLARA; SEMI-AUTOMATED; SEMI-AUTOMATICS; TIME-STEPPING;

AUTOMATION; BENCHMARKING; CELLULAR ARRAYS; COMPUTATIONAL FLUID DYNAMICS; COMPUTER GRAPHICS; DATA TRANSFER; EQUIPMENT TESTING; HARDWARE; INCOMPRESSIBLE FLOW; MESSAGE PASSING; PARALLEL ARCHITECTURES; PROGRAM PROCESSORS;

COMPUTER GRAPHICS EQUIPMENT;

EID: 84859938922     PISSN: 02712091     EISSN: 10970363     Source Type: Journal    
DOI: 10.1002/fld.2664     Document Type: Article

References (48)

1. NVIDIA Corporation. NVIDIA CUDA 3.2 programming guide, 2010.
2. NVIDIA Corporation. Fermi compute architecture white paper, 2009.
3. Khronos OpenCL Working Group. The OpenCL specification: version 1.0 rev. 48, 2009.
4. Owens JD, Luebke D, Govindaraju N, Harris M, Krüger J, Lefohn AE, Purcell TJ. A survey of general-purpose computation on graphics hardware. Computer Graphics Forum 2007; 26:80-113.
5. Brandvik T, Pullan G. Acceleration of a two-dimensional Euler flow solver using commodity graphics hardware. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 2007; 221:1745-1748.
6. Brandvik T, Pullan G. Acceleration of a 3D Euler solver using commodity graphics hardware. 46th AIAA Aerospace Sciences Meeting and Exhibit, number AIAA-2008-607, January 2008.
7. Brandvik T, Pullan G. An accelerated 3D Navier-Stokes solver for flows in turbomachines. Proceedings of GT2009 ASME Turbo Expo 2009: Power for Land, Sea and Air, June 2009.
8. Göddeke D, Buijssen SHM, Wobker H, Turek S. GPU acceleration of an unmodified parallel finite element Navier-Stokes solver. High Performance Computing & Simulation, 2009; 12-21.
9. LeGresley P, Elsen E, Darve E. Large calculation of the flow over a hypersonic vehicle using a GPU. Journal of Computational Physics 2008; 227:10148-10161.
10. Cohen JM, Molemaker MJ.A fast double precision CFD code using CUDA. Parallel Computations Fluid dynamics (ParCFD), Moffett Field, CA, 2009, May 18-22.
11. Phillips EH, Zhang Y, Davis RL, Owens JD. Rapid aerodynamic performance prediction on a cluster of graphics processing units. 47th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA 2009-565, January 2009.
12. Thibault J, Senocak I. CUDA implementation of a Navier-Stokes solver on multi-GPU desktop platforms for incompressible flows. 47th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA 2009-758, January 2009.
13. Jacobsen D, Thibault J, Senocak I. An MPI-CUDA implementation for massively parallel incompressible flow computations on multi-GPU clusters. 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA-2010-522, January 2010.
14. Antoniou AS, Karantasis KI, Polychronopoulos ED, Ekaterinaris JA. Acceleration of a finite-difference WENO scheme for large-scale simulations on many-core architectures. 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA-2010-0525, January 2010.
15. Jespersen DC. Acceleration of a CFD code with a GPU. Technical Report NAS-09-003, NAS, November 2009.
16. Patnaik G, Obenschain KS. Using GPUs on HPC applications to satisfy low-power computational requirements. 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA-2010-524, January 2010.
17. Griebel M, Zaspel P.A multi-GPU accelerated solver for the three-dimensional two-phase incompressible Navier-Stokes equations. Computer Science-Research and Development May 2010; 25(1-2):65-73.
18. Klöckner A, Warburton T, Bridge J, Hesthaven JS. Nodal discontinuous Galerkin methods on graphics processors. Journal of Computational Physics 2009; 228:7863-7882.
19. Markall GR, Ham DA, Kelly PHJ. Towards generating optimised finite element solvers for GPUs from high-level specifications. Proceedings of the 10th International Conference on Computational Science, June 2010.
20. Corrigan A, Camelli FE, Löhner R, Wallin J. Running unstructured grid-based CFD solvers on modern graphics hardware. International Journal for Numerical Methods in Fluids 2010; 66:221-229.
21. Dahm JPS, Fidkowski KJ. Employing coprocessors to accelerate numerical solutions to the Euler equations, 2009. Available from:.
22. Asouti VG, Trompoukis XS, Kampolis IC, Giannakoglou KC. Unsteady CFD computations using vertex-centered finite volumes for unstructured grids on graphics processing units. International Journal for Numerical Methods in Fluids 19 May 2010, DOI: 10.1002/fld.2352.
23. Kampolis IC, Trompoukis XS, Asouti VG, Giannakoglou KC.CFD-based analysis and two-level aerodynamic optimization on graphics processing units. Computer Methods in Applied Mechanics and Engineering 2010; 199(9-12):712-722.
24. Corrigan A, Camelli F, Löhner R, Mut F. Porting of an edge-based CFD solver to GPUs. 48th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, number AIAA-2010-522, January 2010.
25. Corrigan A, Camelli F, Löhner R, Mut F. Porting of FEFLO to GPUs. ECCOMAS CFD 2010: Fifth European Conference on Computational Fluid Dynamics, June 2010.
26. Corrigan A, Camelli F, Löhner R, Mut F. Semi-automatic porting of a large-scale FORTRAN CFD code to GPUs. International Journal for Numerical Methods in Fluids 2 May 2011. DOI: 10.1002/fld.2560.
27. Khronos OpenCL Working Group. The OpenCL specification: version 1.1 rev. 36, 2010.
28. Peterson P.F2PY: a tool for connecting Fortran and Python programs. International Journal of Computational Science and Engineering 2009; 4:296-305.
29. Hoberock J, Bell N. Thrust: a parallel template library, version 1.3, 2010.
30. Liu J, Kailasanath K, Ramamurti R, Gutmark E, Munday D, Löhner R. Large-eddy simulations of a supersonic jet and its near-field acoustic properties. AIAA Journal 2009; 8(47):1849-1864.
31. Baum JD, Luo H, Löhner R, Goldberg E, Feldhun A. Application of unstructured adaptive moving body methodology to the simulation of fuel tank separation from an F-16 C/D fighter. AIAA Aerospace Sciences Meeting, number AIAA-1997-0166, January 1997.
32. Baum JD, Luo H, Löhner R, Yang C, Pelessone D, Charman C.A coupled fluid/structure modeling of shock interaction with a truck. AIAA Aerospace Sciences Meeting, number AIAA-1996-0795, January 1996.
33. Baum JD, Luo H, Mestreau E, Löhner R, Pelessone D, Charman C.A coupled CFD/CSD methodology for modeling weapon detonation and fragmentation. AIAA Aerospace Sciences Meeting, number AIAA-1999-0794, January 1999.
34. Löhner R, Yang C, Oñate E. Simulation of flows with violent free surface motion and moving objects using unstructured grids. International Journal of Numerical Methods in Fluids 2007(53):1315-1338.
35. Camelli F, Löhner R. VLES study of flow and dispersion patterns in heterogeneous urban areas. AIAA Aerospace Sciences Meeting, number AIAA-2006-1419, January 2006.
36. Appanaboyina S, Mut F, Löhner R, Putman CM, Cebral JR. Computational fluid dynamics of stented intracranial aneurysms using adaptive embedded unstructured grids. International Journal for Numerical Methods in Fluids 2008; 5(57):475-493.
37. Mavriplis D. Three-dimensional unstructured multigrid for the Euler equations. AIAA CFD Conference, number AIAA-1991-1549-CP, June 1991.
38. Peraire J, Peiro J, Morgan K.A three-dimensional finite element multigrid solver for the Euler equations. AIAA Aerospace Sciences Meeting, number AIAA-1992-0449, January 1992.
39. Anderson WK, Gropp WD, Kaushik DK, Keyes DE, Smith BF. Achieving high sustained performance in an unstructured mesh CFD application. Supercomputing 1999, IEEE Computer Society, November 1999.
40. Aumann P, Barnewitz H, Becker K, Heinrich R, Roll B, Galle M, Kroll N, Gerhold T, Schwamborn D, Franke M.MEGAFLOW: parallel complete aircraft CFD. Parallel Computing 2001; 27:415-440.
41. Nakahashi K, Ito Y, Togashi F. Some challenges of realistic flow simulations by unstructured grid CFD. International Journal for Numerical Methods in Fluids 2008; 43:769-783.
42. Löhner R. Renumbering strategies for unstructured-grid solvers operating on shared-memory, cache-based parallel machines. Computer Methods in Applied Mechanics and Engineering 1998; 163:95-109.
43. Ramamurti R, Löhner R.A parallel implicit incompressible flow solver using unstructured meshes. Computers and Fluids 1996; 5:119-132.
44. Löhner R, Ramamurti R.A load balancing algorithm for unstructured grids. International Journal of Computational Fluid Dynamics 1995; 5:39-58.
45. Löhner R. Applied CFD Techniques. J. Wiley & Sons: Hoboken, 2008.
46. Löhner R, Morgan K, Peraire J, Vahdati M. Finite element flux-corrected transport (FEM-FCT) for the Euler and Navier-Stokes equations. International Journal for Numerical Methods in Fluids 1987; 7(10):1093-1109.
47. Luo H, Baum JD, Löhner R. Edge-based finite element scheme for the Euler equations. AIAA Journal 1994; 32(6):1183-1190.
48. Löhner R, Yang C, Cebral JR, Camelli F, Soto O, Waltz J. Improving the speed and accuracy of projection-type incompressible flow solvers. Computer Methods in Applied Mechanics and Engineering 2006; 23-24(195):3087-3109.