Scheduling for numerical linear algebra library at scale

Advances in Parallel Computing

Volumn 18, Issue , 2009, Pages 3-26

  Kurzak, Jakub ^a   Ltaief, Hatem ^a   Dongarra, Jack J ^a,b,c   Badia, Rosa M ^d  

^a UNIVERSITY OF TENNESSEE   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d BARCELONA SUPERCOMPUTING CENTER   (Spain)

Author keywords

Cholesky; linear algebra; matrix factorization; multicore; QR; scheduling; task graph

Indexed keywords

EID: 84906552224     PISSN: 09275452     EISSN: None     Source Type: Book Series    
DOI: 10.3233/978-1-60750-073-5-3     Document Type: Conference Paper

References (42)

1. E. Anderson, Z. Bai, C. Bischof, L. S. Blackford, J. W. Demmel, J. J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen. LA-PACK Users' Guide. SIAM, Philadelphia, PA, 1992. http://www.netlib. org/lapack/lug/.
2. L. S. Blackford, J. Choi, A. Cleary, E. D'Azevedo, J. Demmel, I. Dhillon, J. J. Dongarra, S. Hammarling, G. Henry, A. Petitet, K. Stanley, D. Walker, and R. C. Whaley. ScaLAPACK Users' Guide. SIAM, Philadelphia, PA, 1997. http://www.netlib.org/scalapack/slug/.
3. Co-Array Fortran. http://www.co-array.org/
4. The Berkeley Unified Parallel C (UPC) project. http://upc.lbl.gov/
5. Titanium project home page. http://titanium.cs.berkeley.edu/.
6. Cray, Inc. Chapel Language Specification 0.775. http://chapel.cs. washington.edu/spec-0.775.pdf.
7. Sun Microsystems, Inc. The Fortress Language Specification, Version 1.0, 2008. http://research.sun.com/projects/plrg/Publications/fortress.1.0.pdf.
8. V. Saraswat and N. Nystrom. Report on the Experimental Language X10, Version 1.7, 2008. http://dist.codehaus.org/x10/documentation/languagespec/x10- 170.pdf.
9. R. D. Blumofe, C. F. Joerg, B. C. Kuszmaul, C. E. Leiserson, K. H. Randall, and Y. Zhou. Cilk: An efficient multithreaded runtime system. In Principles and Practice of Parallel Programming, Proceedings of the fifth ACM SIGPLAN symposium on Principles and Practice of Parallel Programming, PPOPP'95, pages 207-216, Santa Barbara, CA, July 19-21 1995. ACM. DOI: 10.1145/209936.209958
10. Intel Threading Building Blocks. http://www. threadingbuildingblocks.org/ .
11. J. Reinders. Intel Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism. O'Reilly Media, Inc., 2007. ISBN: 0596514808.
12. OpenMP Architecture Review Board. OpenMP Application Program Interface, Version 3.0, 2008. http://www.openmp.org/mp-documents/spec30. pdf
13. The community of OpenMP users, researchers, tool developers and providers. http://www.compunity.org/.
14. E. Ayguadé, N. Copty, A. Duran, J. Hoeflinger, Y. Lin, F. Massaioli, E. Su, P. Un-nikrishnan, and G. Zhang. A proposal for task parallelism in OpenMP. In A Practical Programming Model for the Multi-Core Era, 3rd International Workshop on OpenMP, IWOMP 2007, Beijing, China, June 3-7 2007. Lecture Notes in Computer Science 4935:1-12. DOI: 10.1007/978-3-540-69303-1-1.
15. A. Duran, J. M. Perez, R. M. Ayguadé, E. amd Badia, and J. Labarta. Extending the OpenMP tasking model to allow dependent tasks. In OpenMP in a New Era of Parallelism, 4th International Workshop, IWOMP 2008, West Lafayette, IN, May 12-14 2008. Lecture Notes in Computer Science 5004:111-122. DOI: 10.1007/978-3-540-79561-2-10.
16. Barcelona Supercomputing Center. SMP Superscalar (SMPSs) User's Manual, Version 2.0, 2008. http://www.bsc.es/media/1002.pdf.
17. Supercomputing Technologies Group, MIT Laboratory for Computer Science. Cilk 5.4.6 Reference Manual, 1998. http://supertech.csail.mit.edu/cilk/manual-5. 4.6.pdf.
18. P. Bellens, J. M. Perez, R. M. Badia, and J. Labarta. CellSs: A programming model for the Cell BE architecture. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing, Tampa, Florida, November 11-17 2006. ACM. DOI: 10.1145/1188455.1188546.
19. J. M. Perez, P. Bellens, R. M. Badia, and J. Labarta. CellSs: Making it easier to program the Cell Broadband Engine processor. IBM J. Res. & Dev., 51(5):593-604, 2007. DOI: 10.1147/rd.515.0593.
20. J. E. Smith and G. S. Sohi. The microarchitecture of superscalar processors. Proceedings of the IEEE, 83(12):1609-1624, 1995.
21. D. J. Kuck, R. H. Kuhn, D. A. Padua, B. Leasure, and M. Wolfe. Dependence graphs and compiler optimizations. In Proceedings of the 8th ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, pages 207-218, Williams-burg, VA, January 1981. ACM. DOI: 10.1145/209936.209958.
22. J. Kurzak, A. Buttari, and J. J. Dongarra. Solving systems of linear equation on the CELL processor using Cholesky factorization. Trans. Parallel Distrib. Syst., 19(9):1175-1186, 2008. DOI: TPDS.2007.70813.
23. J. Kurzak and J. J. Dongarra. QR factorization for the CELL processor. Scientific Programming. (accepted).
24. R. E. Lord, J. S. Kowalik, and S. P. Kumar. Solving linear algebraic equations on an MIMD computer. J. ACM, 30(1):103-117, 1983. DOI: 10.1145/322358.322366.
25. R. C. Agarwal and F. G. Gustavson. A parallel implementation of matrix multiplication and LU factorization on the IBM 3090. In Proceedings of the IFIP WG 2.5 Working Conference on Aspects of Computation on Asynchronous Parallel Processors, pages 217-221, Stanford, CA, August 22-25 1988. North-Holland Publishing Company. ISBN: 0444873104.
26. R. C. Agarwal and F. G. Gustavson. Vector and parallel algorithms for Cholesky factorization on IBM 3090. In Proceedings of the 1989 ACM/IEEE conference on Supercomputing, pages 225-233, Reno, NV, November 13-17 1989. ACM. DOI: 10.1145/76263.76287.
27. J. Kurzak and J. J. Dongarra. Implementing linear algebra routines on multi-core processors with pipelining and a look ahead. In Applied Parallel Computing, State of the Art in Scientific Computing, 8th International Workshop, PARA 2006, Umeå, Sweden, June 18-21 2006. Lecture Notes in Computer Science 4699:147-156. DOI: 10.1007/978-3-540-75755-9-18.
28. A. Buttari, J. J. Dongarra, P. Husbands, J. Kurzak, and K. Yelick. Multithreading for synchronization tolerance in matrix factorization. In Scientific Discovery through Advanced Computing, SciDAC 2007, Boston, MA, June 24-28 2007. Journal of Physics: Conference Series 78:012028, IOP Publishing. DOI: 10.1088/1742-6596/78/1/012028.
29. A. Buttari, J. Langou, J. Kurzak, and J. J. Dongarra. Parallel tiled QR factorization for multicore architectures. Concurrency Computat.: Pract. Exper., 20(13):1573-1590, 2008. DOI: 10.1002/cpe.1301.
30. A. Buttari, J. Langou, J. Kurzak, and J. J. Dongarra. A class of parallel tiled linear algebra algorithms for multicore architectures. Parellel Comput. Syst. Appl. (accepted).
31. D. Irony, G. Shklarski, and S. Toledo. Parallel and fully recursive multi-frontal sparse Cholesky. Future Gener. Comput. Syst., 20(3):425-440, 2004. DOI: 10.1016/j.future.2003.07.007.
32. E. Elmroth and F. G. Gustavson. New serial and parallel recursive QR factorization algorithms for SMP systems. In Applied Parallel Computing, Large Scale Scientific and Industrial Problems, 4th International Workshop, PARA'98, Umeå, Sweden, June 14-17 1998. Lecture Notes in Computer Science 1541:120-128. DOI: 10.1007/BFb0095328.
33. E. Elmroth and F. G. Gustavson. Applying recursion to serial and parallel QR factorization leads to better performance. IBM J. Res. & Dev., 44(4):605-624, 2000.
34. E. Elmroth and F. G. Gustavson. High-performance library software for QR factorization. In Applied Parallel Computing, New Paradigms for HPC in Industry and Academia, 5th International Workshop, PARA 2000, Bergen, Norway, June 18-20 2000. Lecture Notes in Computer Science 1947:53-63. DOI: 10.1007/3-540-70734-4- 9.
35. B. C. Gunter and R. A. van de Geijn. Parallel out-of-core computation and updating the QR factorization. ACM Transactions on Mathematical Software, 31(1):60-78, 2005. DOI: 10.1145/1055531.1055534.
36. F. G. Gustavson. New generalized matrix data structures lead to a variety of high-performance algorithms. In Proceedings of the IFIP WG 2.5 Working Conference on Software Architectures for Scientific Computing Applications, pages 211-234, Ottawa, Canada, October 2-4 2000. Kluwer Academic Publishers. ISBN: 0792373391.
37. F. G. Gustavson, J. A. Gunnels, and J. C. Sexton. Minimal data copy for dense linear algebra factorization. In Applied Parallel Computing, State of the Art in Scientific Computing, 8th International Workshop, PARA 2006, Umeå, Sweden, June 18-21 2006. Lecture Notes in Computer Science 4699:540-549. DOI: 10.1007/978-3-540-75755-9-66.
38. E. Elmroth, F. G. Gustavson, I. Jonsson, and B. Kågström. Recursive blocked algorithms and hybrid data structures for dense matrix library software. SIAM Review, 46(1):3-45, 2004. DOI: 10.1137/S0036144503428693.
39. N. Park, B. Hong, and V. K. Prasanna. Analysis of memory hierarchy performance of block data layout. In Proceedings of the 2002 International Conference on Parallel Processing, ICPP'02, pages 35-44, Vancouver, Canada, August 18-21 2002. IEEE Computer Society. DOI: 10.1109/ICPP.2002.1040857.
40. N. Park, B. Hong, and V. K. Prasanna. Tiling, block data layout, and memory hierarchy performance. IEEE Trans. Parallel Distrib. Syst., 14(7):640-654, 2003. DOI: 10.1109/TPDS.2003.1214317.
41. C. Bischof and C. van Loan. The WY representation for products of Householder matrices. J. Sci. Stat. Comput., 8:2-13, 1987.
42. R. Schreiber and C. van Loan. A storage-efficient WY representation for products of Householder transformations. J. Sci. Stat. Comput., 10:53-57, 1991.