A scalable framework for heterogeneous GPU-based clusters

Annual ACM Symposium on Parallelism in Algorithms and Architectures

Volumn , Issue , 2012, Pages 91-100

  Song, Fengguang ^a   Dongarra, Jack ^a,b,c  

^a UNIVERSITY OF TENNESSEE   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

Distributed runtime; Heterogeneous clusters; Hybrid CPU GPU architectures; Linear algebra; Manycore scheduling

Indexed keywords

CHOLESKY FACTORIZATIONS; COMPUTATIONAL PERFORMANCE; CPU CORES; DATA DEPENDENCIES; DATAFLOW PROGRAMMING; DISTRIBUTED DYNAMICS; DISTRIBUTED MEMORY; DISTRIBUTED MEMORY CLUSTERS; DYNAMIC SCHEDULING; ENTIRE SYSTEM; FASTER RATES; GPU CLUSTERS; HETEROGENEOUS CLUSTERS; HETEROGENEOUS SYSTEMS; HIGH ENERGY EFFICIENCY; MANY-CORE; MULTI-LEVEL PARTITIONING; PARALLEL SOFTWARE; PCI EXPRESS; PROCESSING UNITS; RUNTIME SYSTEMS; RUNTIMES;

COMMUNICATION; COMPUTER PERIPHERAL EQUIPMENT; COMPUTER PROGRAMMING; DATA FLOW ANALYSIS; ENERGY EFFICIENCY; LINEAR ALGEBRA; MATLAB; PROGRAM PROCESSORS; SCHEDULING;

CLUSTERING ALGORITHMS;

EID: 84864149777     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2312005.2312025     Document Type: Conference Paper

References (25)

1. E. Agullo, C. Augonnet, J. Dongarra, M. Faverge, J. Langou, H. Ltaief, and S. Tomov. LU factorization for accelerator-based systems. ICL Technical Report ICL-UT-10-05, Innovative Computing Laboratory, University of Tennessee, 2010.
2. E. Agullo, C. Augonnet, J. Dongarra, M. Faverge, H. Ltaief, S. Thibault, and S. Tomov. QR factorization on a multicore node enhanced with multiple GPU accelerators. In IPDPS 2011, Alaska, USA, 2011.
3. E. Agullo, C. Augonnet, J. Dongarra, H. Ltaief, R. Namyst, J. Roman, S. Thibault, and S. Tomov. Dynamically scheduled Cholesky factorization on multicore architectures with GPU accelerators. In Symposium on Application Accelerators in High Performance Computing (SAAHPC), Knoxville, USA, 2010.
4. E. Agullo, J. Dongarra, B. Hadri, J. Kurzak, J. Langou, J. Langou, H. Ltaief, P. Luszczek, and A. YarKhan. PLASMA Users' Guide. Technical report, ICL, UTK, 2011.
5. E. Anderson, Z. Bai, C. Bischof, L. Blackford, J. Demmel, J. Dongarra, J. D. Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LAPACK Users' Guide. SIAM, 1992.
6. C. Augonnet, S. Thibault, R. Namyst, and P.-A. Wacrenier. StarPU: A unified platform for task scheduling on heterogeneous multicore architectures. Concurr. Comput. : Pract. Exper., Special Issue: Euro-Par 2009, 23:187-198, Feb. 2011.
7. E. Ayguadé, R. M. Badia, F. D. Igual, J. Labarta, R. Mayo, and E. S. Quintana-Ortí. An extension of the StarSs programming model for platforms with multiple GPUs. In Proceedings of the 15th International Euro-Par Conference on Parallel Processing, Euro-Par '09, pages 851-862. Springer-Verlag, 2009.
8. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Communication-optimal parallel and sequential Cholesky decomposition. In Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures, SPAA '09, pages 245-252. ACM, 2009.
9. O. Beaumont, V. Boudet, A. Petitet, F. Rastello, and Y. Robert. A proposal for a heterogeneous cluster ScaLAPACK (dense linear solvers). IEEE Transactions on Computers, 50:1052-1070, 2001. (Pubitemid 33048369)
10. L. S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. Whaley. ScaLAPACK Users' Guide. SIAM, 1997.
11. P. Boulet, J. Dongarra, Y. Robert, and F. Vivien. Static tiling for heterogeneous computing platforms. Parallel Computing, 25(5):547-568, 1999.
12. J. W. Demmel, L. Grigori, M. F. Hoemmen, and J. Langou. Communication-optimal parallel and sequential QR and LU factorizations. LAPACK Working Note 204, UTK, August 2008.
13. J. J. Dongarra, P. Luszczek, and A. Petitet. The LINPACK Benchmark: past, present, and future. Concurrency and Computation: Practice and Experience, 15:803-820, 2003.
14. M. Fatica. Accelerating Linpack with CUDA on heterogenous clusters. In Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units, GPGPU-2, pages 46-51. ACM, 2009.
15. M. Fogué, F. D. Igual, E. S. Quintana-ortŠ, and R. V. D. Geijn. Retargeting PLAPACK to clusters with hardware accelerators. FLAME Working Note 42, 2010.
16. J. R. Humphrey, D. K. Price, K. E. Spagnoli, A. L. Paolini, and E. J. Kelmelis. CULA: Hybrid GPU accelerated linear algebra routines. In SPIE Defense and Security Symposium (DSS), April 2010.
17. A. Lastovetsky and R. Reddy. Data distribution for dense factorization on computers with memory heterogeneity. Parallel Comput., 33:757-779, December 2007. (Pubitemid 350122765)
18. V. Marjanović, J. Labarta, E. Ayguadé, and M. Valero. Overlapping communication and computation by using a hybrid MPI/SMPSs approach. In Proceedings of the 24th ACM International Conference on Supercomputing, ICS '10, pages 5-16. ACM, 2010.
19. NVIDIA. CUDA Toolkit 4.0 CUBLAS Library, 2011.
20. G. Quintana-Ortí, F. D. Igual, E. S. Quintana-Ortí, and R. A. van de Geijn. Solving dense linear systems on platforms with multiple hardware accelerators. In Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming, PPoPP '09, pages 121-130. ACM, 2009.
21. C. J. Rossbach, J. Currey, M. Silberstein, B. Ray, and E. Witchel. PTask: Operating system abstractions to manage GPUs as compute devices. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles, SOSP '11, pages 233-248. ACM, 2011.
22. S. Sharma, C.-H. Hsu, and W. chun Feng. Making a case for a Green500 list. In IEEE International Parallel and Distributed Processing Symposium (IPDPS 2006)/ Workshop on High Performance - Power Aware Computing, 2006.
23. F. Song, S. Tomov, and J. Dongarra. Efficient support for matrix computations on heterogeneous multi-core and multi-GPU architectures. LAPACK Working Note 250, UTK, June 2011.
24. S. Tomov, R. Nath, P. Du, and J. Dongarra. MAGMA Users' Guide. Technical report, ICL, UTK, 2011.
25. J. Vetter, R. Glassbrook, J. Dongarra, K. Schwan, B. Loftis, S. McNally, J. Meredith, J. Rogers, P. Roth, K. Spafford, and S. Yalamanchili. Keeneland: Bringing heterogeneous GPU computing to the computational science community. Computing in Science Engineering, 13(5):90-95, sept.-oct. 2011.