Scheduling dense linear algebra operations on multicore processors

Concurrency and Computation: Practice and Experience

Volumn 22, Issue 1, 2010, Pages 15-44

  Kurzak, Jakub ^a   Ltaief, Hatem ^a   Dongarra, Jack ^a,b,c   Badia, Rosa M ^d  

^a University of Tennessee   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d BARCELONA SUPERCOMPUTING CENTER   (Spain)

Author keywords

Cholesky; Direct acyclic graph; Dynamic scheduling; Factorization; Linear algebra; LU; Matrix factorization; Multicore; QR; Scheduling; Task graph

Indexed keywords

DATA FLOW ANALYSIS; FACTORIZATION; GRAPH THEORY; LINEAR ALGEBRA; LUTETIUM; PARALLEL PROCESSING SYSTEMS; SCHEDULING; SOFTWARE ARCHITECTURE;

CHOLESKY; DIRECT ACYCLIC GRAPHS; DYNAMIC SCHEDULING; MATRIX FACTORIZATIONS; MULTI CORE; TASK GRAPH;

MULTICORE PROGRAMMING;

EID: 73149105729     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1467     Document Type: Article

References (46)

1. Anderson E, Bai Z, Bischof C, Blackford LS, Demmel JW, Dongarra JJ, Du Croz J, Greenbaum A, Hammarling S, McKenney A, Sorensen D. LAPACK Users' Guide. SIAM: Philadelphia, PA, 1992. Available at: http://www.netlib. org/lapack/lug/ [2 June 2009].
2. Blackford LS, Choi J, Cleary A, D'Azevedo E, Demmel J, Dhillon I, Dongarra JJ, Hammarling S, Henry G, Petitet A, Stanley K, Walker D, Whaley RC. ScaLAPACK Users' Guide. SIAM: Philadelphia, PA, 1997. Available at: http://www.netlib.org/scalapack/slug/ [2 June 2009].
3. Co-Array Fortran. Available at: http://www.co-array.org/ [2 June 2009].
4. The Berkeley Unified Parallel C (UPC) project. Available at: http://upc.lbl.gov/ [2 June 2009].
5. Titanium project home page. Available at: http://titanium.cs.berkeley. edu/ [2 June 2009].
6. Cray, Inc. Chapel Language Specification 0.775. Available at: http://chapel.cs.washington.edu/spec-0.775.pdf [2 June 2009].
7. Sun Microsystems Inc. The Fortress Language Specification, Version 1.0, 2008. Available at: http://research. sun.com/projects/plrg/ Publications/fortress.1.0.pdf [2 June 2009].
8. Saraswat V, Nystrom N. Report on the Experimental Language X10, Version 1.7, 2008. Available at: http://dist.codehaus.org/x10/documentation/ languagespec/x10-170.pdf [2 June 2009].
9. Blumofe RD, Joerg CF, Kuszmaul BC, Leiserson CE, Randall KH, Zhou Y. Cilk: An efficient multithreaded runtime system. Principles and Practice of Parallel Programming, Proceedings of the Fifth ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP'95. ACM: Santa Barbara, CA, 19-21 July 1995; 207-216. DOI: 10.1145/209936.209958.
10. Intel Threading Building Blocks. Available at: http://www. threadingbuildingblocks.org/ [2 June 2009].
11. Reinders J. Intel Threading Building Blocks: Outfitting C++ for Multi-core Processor Parallelism. O'Reilly Media, Inc., 2007. Available at: http://www.amazon.com/exec/obidos/ASIN/0596514808/ ISBN: 0596514808 [2 June 2009].
12. OpenMP Architecture Review Board. OpenMP Application Program Interface, Version 3.0, 2008. Available at: http://www.openmp.org/mp- documents/spec30.pdf [2 June 2009].
13. The community of OpenMP users, researchers, tool developers and providers. Available at: http://www.compunity.org/ [2 June 2009].
14. Ayguadé E, Copty N, Duran A, Hoeflinger J, Lin Y, Massaioli F, Su E, Unnikrishnan P, Zhang G. A proposal for task parallelism in OpenMP. A Practical Programming Model for the Multi-Core Era, 3rd International Workshop on OpenMP, IWOMP 2007 (Lecture Notes in Computer Science, vol. 4935), Beijing, China. Springer: Berlin, 3-7 June 2007; 1-12. DOI: 10.1007/978-3-540- 69303-1.1.
15. Duran A, Perez JM, Ayguadé RM, Badia Labarta J. Extending the OpenMP tasking model to allow dependent tasks. OpenMP in a New Era of Parallelism, 4th International Workshop, IWOMP 2008 (Lecture Notes in Computer Science, vol. 5004), West Lafayette, IN. Springer: Berlin, 12-14 May 2008; 111-122. DOI: 10.1007/978-3-540-79561-2.10.
16. Barcelona Supercomputing Center. SMP Superscalar (SMPSs) User's Manual, Version 2.0, 2008. Available at: http://www.bsc.es/media/1002.pdf [2 June 2009].
17. Supercomputing Technologies Group. Cilk 5.4.6 Reference Manual, MIT Laboratory for Computer Science, 1998. Available at: http://supertech.csail. mit.edu/cilk/manual-5.4.6.pdf [2 June 2009].
18. Bellens P, Perez JM, Badia RM, Labarta J. CellSs: A programming model for the Cell BE architecture. Proceedings of the 2006 ACM/IEEE Conference on Supercomputing. Tampa, FL. ACM: New York, 11-17 November 2006; DOI: 10.1145/1188455.1188546.
19. Perez JM, Bellens P, Badia RM, Labarta J. CellSs: Making it easier to program the Cell Broadband Engine processor. IBM Journal of Research and Development 2007; 51(5): 593-604. DOI: 10.1147/rd.515.0593.
20. Smith JE, Sohi GS. The microarchitecture of superscalar processors. Proceedings of the IEEE 1995; 83(12): 1609-1624.
21. Kuck DJ, Kuhn RH, Padua DA, Leasure B, Wolfe M. Dependence graphs and compiler optimizations. Proceedings of the 8th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM: Williamsburg, VA, January 1981; 207-218. DOI: 10.1145/209936.209958.
22. Kurzak J, Buttari A, Dongarra JJ. Solving systems of linear equation on the CELL processor using Cholesky factorization. IEEE Transactions on Parallel and Distributed Systems 2008; 19(19): 1175-1186. DOI: TPDS.2007.70813.
23. Kurzak J, Dongarra JJ. QR factorization for the CELL processor. Scientific Programming 2009; 17(1-2): 31-42. DOI: 10.3233/SPR-2009-0268.
24. Lord RE, Kowalik JS, Kumar SP. Solving linear algebraic equations on an MIMD computer. Journal of the ACM 1983; 30(1): 103-117. DOI: 10.1145/322358.322366.
25. Agarwal RC, Gustavson FG. A parallel implementation of matrix multiplication and LU factorization on the IBM 3090. Proceedings of the IFIP WG 2.5 Working Conference on Aspects of Computation on Asynchronous Parallel Processors. North-Holland Publishing Company: Stanford, CA, 22-25 August 1988; 217-221. ISBN: 0444873104.
26. Agarwal RC, Gustavson FG. Vector and parallel algorithms for Cholesky factorization on IBM 3090. Proceedings of the 1989 ACM/IEEE Conference on Supercomputing, Reno, NV. ACM: New York, 13-17 November 1989; 225-233. DOI: 10.1145/76263.76287.
27. Kurzak J, Dongarra JJ. Implementing linear algebra routines on multi-core processors with pipelining and a look ahead. Applied Parallel Computing, State of the Art in Scientific Computing, 8th International Workshop, PARA 2006 (Lecture Notes in Computer Science, vol. 4699), Umea, Sweden. Springer: Berlin, 18-21 June 2006; 147-156. DOI: 10.1007/978-3-540-75755-9-18.
28. Buttari A, Dongarra JJ, Husbands P, Kurzak J, Yelick K. Multithreading for synchronization tolerance in matrix factorization. Scientific Discovery Through Advanced Computing, SciDAC 2007 (Journal of Physics: Conference Series, vol. 78: 012028), Boston, MA. IOP Publishing: Bristol, U.K., 24-28 June 2007. DOI: 10.1088/1742-6596/78/1/012028.
29. Buttari A, Langou J, Kurzak J, Dongarra JJ. Parallel tiled QR factorization for multicore architectures. Concurrency and Computation: Practice and Experience 2008; 20(13): 1573-1590. DOI: 10.1002/cpe.1301.
30. Buttari A, Langou J, Kurzak J, Dongarra JJ. A class of parallel tiled linear algebra algorithms for multicore architectures. Parallel Computing: Systems and Applications 2009; 35: 38-53. DOI: 10.1016/j.parco.2008. 10.002
31. Dongarra JJ, Kaufman L, Hammarling S. Squeezing the most out of eigenvalue solvers on high-performance computers. Linear Algebra and its Applications 1986; 77: 113-136.
32. Bischof C, van Loan C. The WY representation for products of Householder matrices. Journal on Scientific and Statistical Computing 1987; 8: 2-13.
33. Schreiber R, van Loan C. A storage-efficient WY representation for products of Householder transformations. Journal on Scientific and Statistical Computing 1991; 10: 53-57.
34. Elmroth E, Gustavson FG. Applying recursion to serial and parallel QR factorization leads to better performance. IBM Journal of Research and Development 2000; 44(4): 605-624.
35. Elmroth E, Gustavson FG. High-performance library software for QR factorization. Applied Parallel Computing, New Paradigms for HPC in Industry and Academia, 5th International Workshop, PARA 2000 (Lecture Notes in Computer Science, vol. 1947), Bergen, Norway. Springer: Berlin, 18-20 2000; 53-63. DOI: 10.1007/3-540-70734-4.9.
36. Elmroth E, Gustavson FG. New serial and parallel recursive QR factorization algorithms for SMP systems. Applied Parallel Computing, Large Scale Scientific and Industrial Problems, 4th International Workshop, PARA'98 (Lecture Notes in Computer Science, vol. 1541), Umea, Sweden. Springer: Berlin, 14-17 June 1998; 120-128. DOI: 10.1007/BFb0095328. Available at: http://dx.doi.org/10.1007/BFb0095328 [2 June 2009].
37. Gill PE, Golub GH, Murray WA, Saunders MA. Methods for modifying matrix factorizations. Mathematics ofComputation 1974; 28(126): 505-535.
38. Berry MW, Dongarra JJ, Kim Y. LAPACK working note 68: A highly parallel algorithm for the reduction of a nonsymmetric matrix to block upper-Hessenberg form. Technical Report UT-CS-94-221, Computer Science Department, University of Tennessee, 1994. Available at: http://www.netlib.org/lapack/ lawnspdf/lawn68.pdf [2 June 2009].
39. Gustavson FG. Recursion leads to automatic variable blocking for dense linear-algebra algorithms. IBM Journal of Research and Development 1997; 41(6): 737-756. DOI: 10.1147/rd.416.0737.
40. Gustavson FG. New generalized matrix data structures lead to a variety of high-performance algorithms. Proceedings of the IFIP WG 2.5 Working Conference on Software Architectures for Scientific Computing Applications. Kluwer Academic Publishers: Ottawa, Canada, 2-4 October 2000; 211-234. ISBN: 0792373391.
41. Gustavson FG, Gunnels JA, Sexton JC. Minimal data copy for dense linear algebra factorization. Applied Parallel Computing, State of the Art in Scientific Computing, 8th International Workshop, PARA 2006 PARA 2006.(Lecture Notes in Computer Science, vol. 4699), Umeå, Sweden. Springer: Berlin, 18-21 June 2006; 540-549. DOI: 10.1007/978-3-540-75755-9.66.
42. Elmroth E, Gustavson FG, Jonsson I, Kågström B. Recursive blocked algorithms and hybrid data structures for dense matrix library software. SIAM Review 2004; 46(1): 3-45. DOI: 10.1137/S0036144503428693.
43. Gunter BC, van de Geijn RA. Parallel out-of-core computation and updating the QR factorization. ACM Transactions on Mathematical Software 2005; 31(1): 60-78. DOI: 10.1145/1055531.1055534.
44. Quintana-Ortí ES, van de Geijn RA. Updating an LU factorization with pivoting. ACM Trans. Math. Softw 2008; 35(2): 11. DOI: 10.1145/1377612.1377615.
45. Dongarra JJ, Duff IS, Sorensen DC, van der Vorst HA. Numerical Linear Algebra for High-Performance Computers. SIAM: Philadelphia, 1998. ISBN: 0898714281.
46. Demmel JW. Applied Numerical Linear Algebra. SIAM: Philadelphia, 1997. ISBN: 0898713897.