Towards dense linear algebra for hybrid GPU accelerated manycore systems

Parallel Computing

Volumn 36, Issue 5-6, 2010, Pages 232-240

  Tomov, Stanimire ^a   Dongarra, Jack ^a,b,c   Baboulin, Marc ^a,d  

^a University of Tennessee Knoxville   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

Dense linear algebra; Graphics processing units; Hybrid computing; Multicore processors; Parallel algorithms

Indexed keywords

DENSE LINEAR ALGEBRA; EFFICIENT ALGORITHM; GRAPHICS PROCESSING UNIT; GRAPHICS PROCESSOR; HIGH PERFORMANCE COMPUTING; HYBRID COMPONENT; HYBRID COMPUTING; LU FACTORIZATION; MANY-CORE; MULTI CORE; MULTI-CORE PROCESSOR;

ALGEBRA; LEARNING ALGORITHMS; NANOTECHNOLOGY; PARALLEL ALGORITHMS; PARALLEL ARCHITECTURES; PROGRAM PROCESSORS;

COMPUTER GRAPHICS EQUIPMENT;

EID: 77953130739     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2009.12.005     Document Type: Article

References (38)

1. E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, D. Sorensen, LAPACK user's guide, SIAM, third ed., 1999.
2. M. Baboulin, J. Dongarra, S. Tomov, Some issues in dense linear algebra for multicore and special purpose architectures, Technical Report UT-CS-08-615, University of Tennessee, 2008, LAPACK Working Note 200.
3. G. Ballard, J. Demmel, O. Holtz, O. Schwartz, Minimizing communication in linear algebra, Technical Report, LAPACK Working Note 218, May 2009.
4. S. Barrachina, M. Castillo, F. Igual, R. Mayo, E. Quintana-Ortí, Solving dense linear systems on graphics processors, Technical Report ICC 02-02-2008, Universidad Jaime I, February, 2008.
5. A. Buttari, J. Dongarra, J. Kurzak, J. Langou, P. Luszczek, S. Tomov, The impact of multicore on math software, PARA 2006, in: B. Kågström et al. (Ed.), Lecture Notes in Computer Science, vol. 4699, Springer, 2007, pp. 1-10.
6. Buttari A., Dongarra J., Kurzak J., Luszczek P., and Tomov S. Using mixed precision for sparse matrix computations to enhance the performance while achieving 64-bit accuracy. ACM Trans. Math. Software 34 4 (2008)
7. A. Buttari, J. Langou, J. Kurzak, J. Dongarra, A class of parallel tiled linear algebra algorithms for multicore architectures, Technical Report UT-CS-07-600, University of Tennessee, 2007, LAPACK Working Note 191.
8. J. Demmel, J. Dongarra, B. Parlett, W. Kahan, M. Gu, D. Bindel, Y. Hida, X. Li, O. Marques, E. Riedy, C. Vömel, J. Langou, P. Luszczek, J. Kurzak, A. Buttari, J. Langou, S. Tomov, Prospectus for the next LAPACK and ScaLAPACK libraries, in: PARA'06: State-of-the-Art in Scientific and Parallel Computing (Umeå, Sweden), High Performance Computing Center North (HPC2N) and the Department of Computing Science, Umeå University, Springer, June 2006.
9. J. Demmel, L. Grigori, M. Hoemmen, J. Langou, Communication-avoiding parallel and sequential QR factorizations, CoRR abs/0806.2159, 2008.
10. Dongarra J., Luszczek P., and Petitet A. The LINPACK benchmark: past, present, and future. Concurrency and Computation: Practice and Experience 15 (2003) 820
11. J. Dongarra, S. Moore, G. Peterson, S. Tomov, J. Allred, V. Natoli, D. Richie, Exploring new architectures in accelerating CFD for air force applications, in: Proceedings of HPCMP Users Group Conference 2008, July 14-17, 2008. .
12. K. Fatahalian, J. Sugerman, P. Hanrahan, Understanding the efficiency of GPU algorithms for matrix-matrix multiplication, in: HWWS '04: Proceedings of the ACM Siggraph/Eurographics Conference on Graphics Hardware (New York, NY, USA), ACM, 2004, pp. 133-137.
13. M. Fatica, Accelerating LINPACK with CUDA on heterogenous clusters, in: GPGPU-2: Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units (New York, NY, USA), ACM, 2009, pp. 46-51.
14. N. Galoppo, N. Govindaraju, M. Henson, D. Manocha, LU-GPU: efficient algorithms for solving dense linear systems on graphics hardware, in: SC '05: Proceedings of the 2005 ACM/IEEE Conference on Supercomputing (Washington, DC, USA), IEEE Computer Society, 2005, p. 3.
15. L. Grigori, J. Demmel, H. Xiang, Communication avoiding Gaussian elimination, Technical Report 6523, INRIA, 2008.
16. Wolfgang Gruener, Larrabee, CUDA and the quest for the free lunch, TGDaily. .
17. Higham N. Accuracy and Stability of Numerical Algorithms. second ed. (2002), SIAM
18. Hruska J. AMD fusion now pushed back to 2011. Art Technica (2008)
19. Kågström B., Ling P., and van Loan C. GEMM-based level 3 BLAS: high-performance model implementations and performance evaluation benchmark. ACM Trans. Math. Software 24 3 (1998) 268-302
20. Julie Langou, Julien Langou, P. Luszczek, J. Kurzak, A. Buttari, J. Dongarra, Exploiting the performance of 32 bit floating point arithmetic in obtaining 64 bit accuracy (revisiting iterative refinement for linear systems), in: SC '06: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing (New York, NY, USA), ACM, 2006, p. 113.
21. Y. Li, J. Dongarra, S. Tomov, A note on auto-tuning GEMM for GPUs, Technical Report, LAPACK Working Note 212, January 2009.
22. NVIDIA, Nvidia Tesla doubles the performance for CUDA developers, Computer Graphics World (06/30/2008).
23. NVIDIA, NVIDIA CUDA Programming Guide, 6/07/2008, Version 2.0.
24. Owens J., Houston M., Luebke D., Green S., Stone J., and Phillips J. GPU computing. Proceedings of the IEEE 96 5 (2008) 879-899
25. Owens J., Luebke D., Govindaraju N., Harris M., Krüger J., Lefohn A., and Purcell T. A survey of general-purpose computation on graphics hardware. Comput. Graphics Forum 26 1 (2007) 80-113
26. D. Parker, Random butterfly transformations with applications in computational linear algebra, Technical Report CSD-950023, Computer Science Department, UCLA, 1995.
27. D. Parker, B. Pierce, The randomizing FFT: an alternative to pivoting in Gaussian elimination, Technical Report CSD-950037, Computer Science Department, UCLA, 1995.
28. Pharr M., and Fernando R. GPU Gems 2: Programming Techniques for High-performance Graphics and General-purpose Computation (gpu gems) (2005), Addison-Wesley Professional
29. G. Quintana-Ortí, F.Igual, E.Quintana-Ortí, R. van de Geijn, Solving dense linear systems on platforms with multiple hardware accelerators, in: PPoPP '09: Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (New York, NY, USA), ACM, 2009, pp. 121-130.
30. G. Quintana-Orti, E. Quintana-Orti, E. Chan, F. van Zee, R. van de Geijn, Programming algorithms-by-blocks for matrix computations on multithreaded architectures, Technical Report TR-08-04, University of Texas at Austin, 2008, FLAME Working Note 29.
31. Seiler L., Carmean D., Sprangle E., Forsyth T., Abrash M., Dubey P., Junkins S., Lake A., Sugerman J., Cavin R., Espasa R., Grochowski E., Juan T., and Hanrahan P. Larrabee: a many-core × 86 architecture for visual computing. ACM Trans. Graph. 27 3 (2008) 1-15
32. S. Tomov, M. Baboulin, J. Dongarra, S. Moore, V. Natoli, G. Peterson, D. Richie, Special-purpose hardware and algorithms for accelerating dense linear algebra, in: Parallel Processing for Scientific Computing, Atlanta, March 12-14, 2008. .
33. S. Tomov, J. Dongarra, Accelerating the reduction to upper Hessenberg form through hybrid GPU-based computing, Technical Report 219, LAPACK Working Note, May 2009.
34. V. Volkov, J. Demmel, Benchmarking gpus to tune dense linear algebra, in: SC '08: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing (Piscataway, NJ, USA), IEEE Press, 2008, pp. 1-11.
35. LU, QR, Cholesky factorizations using vector capabilities of GPUs, Technical Report UCB/EECS-2008-49, EECS Department, University of California, Berkeley, May 2008.
36. Using GPUs to accelerate linear algebra routines, Poster at PAR lab winter retreat, January 9, 2008. .
37. General-purpose computation using graphics hardware, .
38. Nvidia cuda zone, NVIDIA. .