Using GPUs to improve multigrid solver performance on a cluster

International Journal of Computational Science and Engineering

Volumn 4, Issue 1, 2008, Pages 36-55

  Göddeke, Dominik ^a   Strzodka, Robert ^b   Mohd Yusof, Jamaludin ^c   McCormick, Patrick ^c   Wobker, Hilmar ^a   Becker, Christian ^a   Turek, Stefan ^a  

^a TU DORTMUND UNIVERSITY   (Germany)

^b MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^c LOS ALAMOS NATIONAL LABORATORY   (United States)

Author keywords

Domain decomposition; Finite element calculations; Floating point co processors; GPUs; Graphics processing units; Mixed precision; Multigrid solvers; Parallel scientific computing

Indexed keywords

APPLICATION PROGRAMS; COMPUTER GRAPHICS; DIGITAL ARITHMETIC; DOMAIN DECOMPOSITION METHODS; FINITE ELEMENT METHOD; GRAPHICS PROCESSING UNIT; ITERATIVE METHODS; PROGRAM PROCESSORS;

FLOATING-POINT CO-PROCESSORS; GPUS; MIXED PRECISION; MULTIGRID SOLVER; PARALLEL SCIENTIFIC COMPUTING;

COMPUTER HARDWARE;

EID: 49249120833     PISSN: 17427185     EISSN: 17427193     Source Type: Journal    
DOI: 10.1504/IJCSE.2008.021111     Document Type: Article

References (70)

1. AGEIA Technologies Inc. (2006) AGEIA PhysX Processor, http:// ageia.com/physx/index.html
2. AMD Inc. (2006) Torrenza Technology, http://enterprise.amd.com/ us-en/AMD-Business/Technology-Home/Torrenza.a%spx
3. Becker, C. (2007) Strategien und Methoden zur Ausnutzung der High-Performance-Computing-Ressourcen moderner Rechnerarchitekturen für Finite Element Simulationen und ihre Realisierung in FEAST (Finite Element Analysis & Solution Tools), PhD Thesis, Universität Dortmund, Fachbereich Mathematik.
4. Becker, C., Kilian, S. and Turek, S. (2007) FEAST - Finite Element Analysis and Solution Tools, http://www.feast.uni-dortmund.de
5. Bolz, J., Farmer, I., Grinspun, E. and Schröder, P. (2003) 'Sparse matrix solvers on the GPU: Conjugate gradients and multigrid', ACM Transactions on Graphics (TOG), Vol. 22, No. 3, pp.917-924.
6. Bondalapati, K. and Prasanna, V.K. (2002) 'Reconfigurable computing systems', Proceedings of the IEEE, Vol. 90, No. 7, pp.1201-1217.
7. Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M. and Hanrahan, P. (2004) 'Brook for GPUs: Stream computing on graphics hardware', ACM Transactions on Graphics (TOG), Vol. 23, No. 3, pp.777-786.
8. ClearSpeed Technology, Inc. (2006) ClearSpeed Advance Accelerator Boards, www.clearspeed.com/products/cs_advance/
9. Colella, P., Dunning, T.H., Gropp, W.D. and Keyes, D.E. (2003) A Science-Based Case for Large-Scale Simulation, Technical report, DOE Office of Science, http://www.pnl.gov/scales
10. Compton, K. and Hauck, S. (2002) 'Reconfigurable computing: A survey of systems and software', ACMComputing Surveys, Vol. 34, No. 2, pp.171-210.
11. Cray Inc. (2006) Cray XD1 Supercomputer, www.cray.com/products/xd1
12. Dally, W.J., Hanrahan, P., Erez, M., Knight, T.J., Labonté, F., Ahn, J.-H., Jayasena, N., Kapasi, U.J., Das, A., Gummaraju, J. and Buck, I. (2003) 'Merrimac: Supercomputing with streams', SC '03: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, p.35.
13. Davis, T.A. (2004) 'A column pre-ordering strategy for the unsymmetric-pattern multifrontal method', ACM Transactions on Mathematical Software, Vol. 30, No. 2, pp.165-195.
14. DeHon, A. (2002) 'Very large scale spatial computing', Lecture Notes in Computer Science, Proceedings of the Third International Conference on Unconventional Models of Computation, Vol. 2509, pp.27-36.
15. Demmel, J., Hida, Y., Kahan, W., Li, X.S., Mukherjee, S. and Riedy, E.J. (2006) 'Error bounds from extra-precise iterative refinement', ACM Transactions on Mathematical Software, Vol. 32, No. 2, pp.325-351.
16. Douglas, C.C., Hu, J., Karl, W., Kowarschik, M., Rüde, U. and Weiß, C. (2000a) 'Fixed and adaptive cache aware algorithms for multigrid methods', in Dick, E., Riemslagh, K. and Vierendeels, J. (Eds. : Multigrid Methods VI, Springer, Berlin, Vol. 14, pp.87-93.
17. Douglas, C.C., Hu, J., Kowarschik, M., Rüde, U. and Weiß, C. (2000b) 'Cache optimisation for structured and unstructured grid multigrid', Electronic Transactions on Numerical Analysis, Vol. 10, pp.21-40.
18. DRC Computer Corporation (2006) DRC Reconfigurable Processor Units http://www.drccomputer.com/drc/modules.html
19. Fan, Z., Qiu, F., Kaufman, A. and Yoakum-Stover, S. (2004) 'GPU cluster for high performance computing', SC '04: Proceedings of the 2004 ACM/ IEEE Conference on Supercomputing, p.47.
20. Fung, J. and Mann, S. (2004) 'Using multiple graphics cards as a general purpose parallel computer: Applications to computer vision', Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), Vol. 1, pp.805-808.
21. Garg, R.P. and Sharapov, I. (2001) Techniques for Optimising Applications: High Performance Computing, Sun Microsystems Inc.
22. Geddes, K.O. and Zheng, W.W. (2003) 'Exploiting fast hardware floating point in high precision computation', ISSAC '03: Proceedings of the 2003 International Symposium on Symbolic and Algebraic Computation, pp.111-118.
23. Göddeke, D. (2006) GPGPU Coding Tutorials, Technical Report, University of Dortmund, Institute of Applied Mathematics and Numerics, http://www.mathematik.uni-dortmund.de/goeddeke/gpgpu/
24. Göddeke, D., Strzodka, R. and Turek, S. (2007) 'Performance and accuracy of hardware-oriented native-, emulated- and mixed-precision solvers in FEM simulations', International Journal of Parallel, Emergent and Distributed Systems, Vol. 22, No. 4, pp.221-256.
25. Goodnight, N., Woolley, C., Lewin, G., Luebke, D. and Humphreys, G. (2003) 'A multigrid solver for boundary value problems using programmable graphics hardware', Graphics Hardware 2003, pp.102-111.
26. Govindaraju, N.K., Sud, A., Yoon, S.-E. and Manocha, D. (2003) 'Interactive visibility culling in complex environments using occlusion-switches', SI3D '03: Proceedings of the 2003 symposium on Interactive 3D Graphics, pp.103-112.
27. GPGPU (2007) General-Purpose Computation Using Graphics Hardware, http://www.gpgpu.org
28. Gropp, W.D., Kaushik, D.K., Keyes, D.E. and Smith, B.F. (2001) 'High performance parallel implicit CFD', Parallel Computing, Vol. 27, pp.337-362.
29. Guo, Z., Najjar, W., Vahid, F. and Vissers, K. (2004) 'A quantitative analysis of the speedup factors of FPGAs over processors', FPGA '04: Proceedings of the 2004 ACM/SIGDA 12th International Symposium on Field Programmable Gate Arrays, pp.162-170.
30. Harris, M. (2005) 'Mapping computational concepts to GPUs', in Pharr, M. (Ed.): GPUGems 2: Programming Techniques for High-Performance Graphics and General-Purpose Computation, Addison-Wesley, Boston, MA, chapter 31, pp.493-508.
31. Harrison, O. and Waldron, J. (2007) 'Optimising data movement rates for parallel processing applications on graphics processors', Proceedings of the 25th International Conference Parallel and Distributed Computing and Networks (PDCN 2007), pp.251-256.
32. Hartenstein, R. (2001) 'A decade of reconfigurable computing: A visionary retrospective', Design, Automation and Test in Europe 2001, Proceedings, pp.642-649.
33. Hartenstein, R. (2003) 'Data-stream-based computing: Models and architectural resources', International Conference on Microelectronics, Devices and Materials (MIDEM 2003), pp.228-235.
34. Hoßfeld, F. (2001) Perspektiven für Supercomputer-Architekturen Technical Report, FZ Jülich - Zentralinstitut für Angewandte Mathematik.
35. IEC (2000) Letter Symbols to be Used in Electrical Technology - Part 2: Telecommunications and Electronics, 2nd ed., IEC, Geneva.
36. Intel, Inc. (2006) Geneseo: PCI Express Technology Advancement, http://www.intel.com/technology/pciexpress/devnet/innovation.htm
37. Keyes, D.E. (2002) 'Terascale implicit methods for partial differential equations', Contemporary Mathematics, Vol. 306, pp.29-84.
38. Khailany, B., Dally, W.J., Rixner, S., Kapasi, U.J., Owens, J.D. and Towles, B. (2003) 'Exploring the VLSI scalability of stream processors', Proceedings of the Ninth Symposium on High Performance Computer Architecture, 153-164.
39. Kilian, S. (2001) ScaRC: Ein verallgemeinertes Gebietszerlegungs-/ Mehrgitterkonzept auf Parallelrechnern, PhD Thesis, Universität Dortmund, Fachbereich Mathematik.
40. Kornhuber, R., Periaux, J., Widlund, O.B., Hoppe, R., Pironneau, O. and Xu, J. (Eds.) (2005) Domain Decomposition Methods in Science and Engineering, Lecture Notes in Computational Science and Engineering, Vol. 40, Springer.
41. Kowarschik, M., Weiß, C., Karl, W. and Rüde, U. (2000) 'Cache-aware multigrid methods for solving poissons equation in two dimensions', Computing, Vol. 64, pp.381-399.
42. Krüger, J. and Westermann, R. (2003) 'Linear algebra operators for GPU implementation of numerical algorithms', ACM Transactions on Graphics (TOG), Vol. 22, No. 3, pp.908-916.
43. Langou, J., Langou, J., Luszczek, P., Kurzak, J., Buttari, A. and Dongarra, J. (2006) 'Tools and techniques for performance - exploiting the performance of 32 bit floating point arithmetic in obtaining 64 bit accuracy (revisiting iterative refinement for linear systems)', SC '06: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, p.113.
44. Li, X.S., Demmel, J.W., Bailey, D.H., Henry, G., Hida, Y., Iskandar, J., Kahan, W., Kang, S.Y., Kapur, A., Martin, M.C., Thompson, B.J., Tung, T. and Yoo, D.J. (2002) 'Design, implementation and testing of extended and mixed precision BLAS', ACM Transactions on Mathematical Software, Vol. 28, No. 2, pp.152-205.
45. Martin, R., Peters, G. and Wilkinson, J. (1966) 'Handbook series linear algebra: Iterative refinement of the solution of a positive definite system of equations', Numerische Mathematik, Vol. 8, pp.203-216.
46. Meuer, H., Strohmaier, E., Dongarra, J.J. and Simon, H.D. (2006) Top500 Supercomputer Sites, http://www.top500.org/
47. NVIDIA Corporation (2007) NVIDIA CUDA Compute Unified Device Architecture Programming Guide, http://developer.nvidia.com/cuda
48. Owens, J.D., Luebke, D., Govindaraju, N., Harris, M., Krüger, J., Lefohn, A.E. and Purcell, T.J. (2007) 'A survey of general-purpose computation on graphics hardware', Computer Graphics Forum, Vol. 26, No. 1, pp.80-113.
49. Peercy, M., Segal, M. and Gerstmann, D. (2006) 'A performance-oriented data parallel virtual machine for GPUs', ACM SIGGRAPH 2006 Conference Abstracts and Applications, p.184.
50. Pham, D., Asano, S., Bolliger, M., Day, M.N., Hofstee, H.P., Johns, C., Kahle, J., Kameyama, A., Keaty, J., Masubuchi, Y., Riley, M., Shippy, D., Stasiak, D., Suzuoki, M., Wang, M., Warnock, J., Weitzel, S., Wendel, D., Yamazaki, T. and Yazawa, K. (2005) 'The design and implementation of a first-generation CELL processor', Solid-State Circuits Conference 2005, Digest of Technical Papers, Vol. 1, pp.184-592
51. Rüde, U. (1999) 'Technological trends and their impact on the future of supercomputers', in Bungartz, H.-J., Durst, F. and Zenger, C. (Eds.): High Performance Scientific and Engineering Computing, Lecture notes in Computational Science and Engineering, Vol. 8, pp.459-471.
52. Rumpf, M. and Strzodka, R. (2005) 'Graphics processor units: New prospects for parallel computing', in Bruaset, A.M. and Tveito, A. (Eds.): Numerical Solution of Partial Differential Equations on Parallel Computers, Lecture Notes in Computational Science and Engineering, Springer, Vol. 51, pp.89-134.
53. Sankaralingam, K., Nagarajan, R., Liu, H., Kim, C., Huh, J., Burger, D., Keckler, S.W. and Moore, C.R. (2003) 'Exploiting ILP, TLP, and DLP with the polymorphous TRIPS architecture', ACM SIGARCH Computer Architecture News, Vol. 31, No. 2, pp.422-433.
54. SEMATECH (2006) International Technology Roadmap for Semiconductors (ITRS), http://www.sematech.org/corporate/annual
55. Sheaffer, J.W., Luebke, D.P. and Skadron, K. (2007) 'A hardware redundancy and recovery mechanism for reliable scientific computation on graphics processors', in Aila, T. and Segal, M. (Eds.): Graphics Hardware 2007, pp.55-64.
56. Smith, B.F., Bjorstad, P.E. and Gropp, W.D. (1996) Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations, Cambridge University Press, Cambridge, UK.
57. Stanford University Graphics Lab (2006) GPUbench - How Much Does Your GPUbench? http://graphics.stanford.edu/projects/gpubench/results
58. Strzodka, R. (2004) Hardware Efficient PDE Solvers in Quantised Image Processing, PhD Thesis, University of Duisburg-Essen.
59. Strzodka, R., Doggett, M. and Kolb, A. (2005) 'Scientific computation for simulations on programmable graphics hardware', Simulation Modelling Practice and Theory, Special Issue: Programmable Graphics Hardware, Vol. 13, No. 8, pp.667-680.
60. Strzodka, R., Droske, M. and Rumpf, M. (2003) 'Fast image registration in DX9 graphics hardware', Journal of Medical Informatics and Technologies, Vol. 6, pp.43-49.
61. Suh, J., Kim, E.-G., Crago, S.P., Srinivasan, L. and French, M.C. (2003) 'A performance analysis of PIM, stream processing, and tiled processing on memory-intensive signal processing kernels', in DeGroot, D. (Ed.): ISCA '03: Proceedings of the 30th Annual International Symposium on Computer Architecture, Computer Architecture News, pp.410-421.
62. Taylor, M.B., Kim, J.S., Miller, J., Wentzlaff, D., Ghodrat, F., Greenwald, B., Hoffmann, H., Johnson, P., Lee, J-W., Lee, W., Ma, A., Saraf, A., Seneski, M., Shnidman, N., Strumpen, V., Frank, M., Amarasinghe, S.P. and Agarwal, A. (2002) 'The raw microprocessor: A computational fabric for software circuits and general purpose programs', IEEE Micro, Vol. 22, No. 2, pp.25-35.
63. Toselli, A. and Widlund, O.B. (2004) Domain Decomposition Methods - Algorithms and Theory, Springer Series in Computational Mathematics, Vol. 34, Springer.
64. Turek, S. (1999) Efficient Solvers for Incompressible Flow Problems: An Algorithmic and Computational Approach, Springer, Berlin.
65. Turek, S., Becker, C. and Kilian, S. (2003) 'Hardware-oriented numerics and concepts for PDE software', Future Generation Computer Systems Vol. 22, Nos. 1-2, pp.217-238.
66. Whaley, R.C., Petitet, A. and Dongarra, J.J. (2001) 'Automated empirical optimisation of software and the ATLAS project', Parallel Computing Vol. 27, Nos. 1-2, pp.3-35.
67. Wilkes, M. (2000) 'The memory gap (keynote)', Solving the Memory Wall Problem Workshop, http://www.ece.neu.edu/conf/wall2k/wilkes1.pdf
68. Williams, S., Shalf, J., Oliker, L., Kamil, S., Husbands, P. and Yelick, K. (2006) 'The potential of the Cell processor for scientific computing', CF '06: Proceedings of the ACM International Conference on Computing Frontiers, pp.9-20.
69. XtremeData Inc. (2006) The XD1000 FPGA Coprocessor Module for Socket 940, http://www.xtremedatainc.com/Products.html
70. Zielke, G. and Drygalla, V. (2003) 'Genaue Lösung linearer Gleichungssysteme', GAMM-Mitteilungen, Vol. 2, No. 1, pp.7-107.