Programming matrix algorithms-by-blocks for thread-level parallelism

ACM Transactions on Mathematical Software

Volumn 36, Issue 3, 2009, Pages

  Quintana Ortí, Gregorio ^a   Quintana Ortí, Enrique S ^a   Geijn, Robert A Van De ^b   Zee, Field G Van ^b   Chan, Ernie ^b  

^a UNIVERSITAT JAUME I   (Spain)

^b University of Texas at Austin   (United States)

Author keywords

High performance; Libraries; Linear algebra; Multithreaded architectures

Indexed keywords

ANECDOTAL EVIDENCES; APRIORI; CLASSICAL ALGORITHMS; CORE FUNCTIONALITY; DATA DEPENDENCIES; DESIGN DECISIONS; HIGH PRODUCTIVITY; HIGH-PERFORMANCE; LU FACTORIZATION; MATRIX ALGORITHMS; MULTITHREADED ARCHITECTURES; OUT OF ORDER; OUT-OF-CORE COMPUTATION; PERFORMANCE IMPROVEMENTS; PROBLEM DOMAIN; PROGRAMMABILITY; QR FACTORIZATIONS; RUNTIME SYSTEMS; SEPARATION OF CONCERNS; THREAD LEVEL PARALLELISM; VIABLE SOLUTIONS;

ABSTRACTING; ALGEBRA; ALGORITHMS; FACTORIZATION; LIBRARIES; PHILOSOPHICAL ASPECTS;

COMPUTATION THEORY;

EID: 70349755577     PISSN: 00983500     EISSN: 15577295     Source Type: Journal    
DOI: 10.1145/1527286.1527288     Document Type: Article

References (46)

1. ADDISON, C., REN,Y., AND VAN WAVEREN, M. 2003. OpenMP issues arising in the development of parallel BLAS and LAPACK libraries. Sci. Program. 11, 2, 95-104.
2. AGARWAL,R.C. AND GUSTAVSON, F. G. 1989. Vector and parallel algorithms for Cholesky factorization on IBM 3090. In SC '89: Proceedings of the ACM/IEEE Conference on Supercomputing, New York. 225-233.
3. ANDERSON, E., BAI, Z., BISCHOF, C., BLACKFORD, L. S., DEMMEL, J., DONGARRA, J. J., CROZ,J.D., HAMMARLING, S., GREENBAUM, A., MCKENNEY, A., AND SORENSEN, D. 1999. LAPACK Users' Guide, 3rd Ed. Society for Industrial and Applied Mathematics, Philadelphia.
4. BALAY, S., BUSCHELMAN, K., EIJKHOUT, V., GROPP,W.D., KAUSHIK, D., KNEPLEY,M.G., MCINNES,L.C., SMITH,B.F., AND ZHANG, H. 2004. PETSc users manual. Tech. rep. ANL-95/11-Revision 2.1.5, Argonne National Laboratory, Argonne.
5. Bientinesi, P., Gunter,B., and van de Geijn, R. A. 2008. Families of algorithms related to the inversion of a symmetric positive definite matrix. ACM Trans. Math. Softw. 35,1, 1-22.
6. BIENTINESI, P., QUINTANA-ORTÍ,E.S., and VAN DE GEIJN, R. A. 2005. Representing linear algebra algorithms in code: The FLAME application programming interfaces. ACM Trans. Math. Softw. 31, 1, 27-59.
7. BUTTARI, A., LANGOU, J., KURZAK,J., AND DONGARRA, J. 2007. A class of parallel tiled linear algebra algorithms for multicore architectures. LAPACK Working Note 191 UT-CS-07-600. University of Knoxville.
8. BUTTARI, A., LANGOU, J., KURZAK,J., AND DONGARRA, J. 2008. Parallel tiled QR factorization for multicore architectures. Concurr. Computat. Pract. Experi. 20, 13, 1573-1590.
9. CHAN, E., QUINTANA-ORTÍ, E. S., QUINTANA-ORTÍ, G., AND VAN DE GEIJN, R. 2007a. SuperMatrix out- of-order scheduling of matrix operations for SMP and multi-core architectures. In Proceedings of the 19th ACM Symposium on Parallelism in Algorithms and Architectures, San Diego, 116-125.
10. CHAN, E., VAN ZEE, F. G., BIENTINESI, P., QUINTANA-ORTÍ, E. S., QUINTANA-ORTÍ, G., AND VAN DE GEIJN, R. 2008. SuperMatrix: A multithreaded runtime scheduling system for algorithms-by-blocks. In Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Salt Lake City, 123-132.
11. CHAN, E., VAN ZEE, F. G., QUINTANA-ORTÍ, E. S., QUINTANA-ORTÍ, G., AND VAN DE GEIJN, R. 2007b. Satisfying your dependencies with SuperMatrix. In Proceedings of the 2007 IEEE International Conference on Cluster Computing Austin, 91-99.
12. CHOI, J., DONGARRA, J. J., POZO, R., AND WALKER, D. W. 1992. ScaLAPACK: A scalable linear algebra library for distributed memory concurrent computers. In Proceedings of the 4th Symposium on the Frontiers of Massively Parallel Computation, McLean, 120-127.
13. COLLINS,T. AND BROWNE, J. C. 1995. Matrix++: An object-oriented environment for parallel high- perfomance matrix computations. In Proceedings of the 28th Annual Hawaii International Conference on System Sciences, Maui, 202-211.
14. DONGARRA, J. J., BUNCH, J. R., MOLER,C.B., and STEWART, G. W. 1979. LINPACK Users' Guide. SIAM, Philadelphia.
15. DONGARRA, J. J., DU CROZ, J., HAMMARLING,S., AND DUFF, I. 1990. A set of level 3 basic linear algebra sub-programs. ACM Trans. Math. Softw. 16, 1, 1-17.
16. DONGARRA,J.J., DU CROZ, J., HAMMARLING,S., AND HANSON, R. J. 1988. An extended set of Fortran basic linear algebra subprograms. ACM Trans. Math. Softw. 14, 1, 1-17.
17. Edwards,H.C. and van de Geijn, R. A. 2006. On application interfaces to parallel dense matrix libraries: Just let me solve my problem! FLAME Working Note #18 TR-2006-2015 Department of Computer Sciences, University of Texas at Austin.
18. ELMROTH, E.,GUSTAVSON, F., JONSSON, I., AND KAGSTROM, B. 2004. Recursive blocked algorithms and hybrid data structures for dense matrix library software. SIAM Rev. 46, 1, 3-45.
19. GOLUB,G.H. AND VAN LOAN, C. F. 1996. Matrix Computations, 3rd Ed. Johns Hopkins University Press, Baltimore.
20. GOTO,K. AND VAN DE GEIJN, R. A. 2008. Anatomy of a high-performance matrix multiplication. ACM Trans. Math. Softw. 34, 3, 1-25.
21. Gropp, W., Lusk, E., and Skjellum, A. 1994. Using MPI. MIT Press, Cambridge.
22. GUNNELS, J. A., GUSTAVSON,F.G., HENRY,G.M., AND VAN DE GEIJN, R. A. 2001. FLAME: Formal linear algebra methods environment. ACM Trans. Math. Softw. 27, 4, 422-455.
23. GUNTER,B.C. AND VAN DE GEIJN, R. A. 2005. Parallel out-of-core computation and updating the QR factorization. ACM Trans. Math. Softw. 31, 1, 60-78.
24. Guo, J., Bikshandi, G., Fraguela, B., Garzaran, M., and Padua, D. 2008. Programming with tiles. In Proceedings of the 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Salt Lake City, 111-122.
25. GUSTAVSON,F.G., KARLSSON, L., AND KAGSTROM, B. 2007. Three algorithms for Cholesky factorization on distributed memory using packed storage. in Proceedings of the Workshop on State- of-the-Art in Scientific Computing. Lecture Notes in Computer Science, vol.4699. Springer, Berlin/Heidelberg, Germany, 550-559.
26. HERRERO, J. R. 2006. A framework for efficient execution of matrix computations. Ph.D. dissertation. Polytechnic University of Catalonia, Barcelona, Spain.
27. JOFFRAIN, T., QULNTANA-ORTÍ, E. S., AND VAN DE GEIJN, R. A. 2004. Rapid development of high- performance out-of-core solvers. In Proceedings of the Workshop on State-of-the-Art in Scientific Computing. Lecture Notes in Computer Science, vol.3732. Springer, Berlin/Heidelberg, Germany, 413-422.
28. KURZAK,J. AND DONGARRA, J. 2006. Implementing linear algebra routines on multi-core processors with pipelining and a look ahead. LAPACK Working Note 178 UT-CS-06-581. University of Tennessee, Knoxville.
29. LAWSON, C. L., HANSON, R. J., KINCAID, D. R., AND KROGH, F. T. 1979. Basic linear algebra subprograms for Fortran usage. ACM Trans. Math. Softw. 5, 3, 308-323.
30. LOW, T. M. AND VAN DE GEIJN, R. 2004. An API for manipulating matrices stored by blocks. FLAME Working Note #12 TR-2004-2015 Department of Computer Sciences, University of Texas at Austin, Austin.
31. LU, H., COX, A. L., DWARKADAS, S., RAJAMONY, R., AND ZWAENEPOEL, W. 1997. Compiler and software distributed shared memory support for irregular applications. In Proceedings of the 6th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Las Vegas, 48-56. (Pubitemid 127452540)
32. MARKER, B. A., VAN ZEE, F. G., GOTO, K., QUINTANA-ORTÍ, G., and VAN DE GEIJN, R. A. 2007. Toward scalable matrix multiply on multithreaded architectures. In Proceedings of the 13th International European Conference on Parallel and Distributed Computing (Rennes, France). 748-757.
33. NIEPLOCHA, J., HARRISON, R., AND LITTLEFIELD, R. 1996. Global arrays: A nonuniform memory access programming model for high-performance computers. J. Supercomput. 10, 2 (June), 197-220.
34. QULNTANA-ORTÍ, E. S. AND VAN DE GEIJN, R. 2009. Updating an LU factorization with pivoting. ACM Trans. Math. Softw.
35. QULNTANA-ORTÍ, G., QULNTANA-ORTÍ, E. S., CHAN, E., VAN DE GEIJN, R., AND VAN ZEE, F. G. 2008a. Design of scalable dense linear algebra libraries for multithreaded architectures: The Lu factorization. In Proceedings of the Workshop on Multithreaded Architectures and Applications, Miami, 1-8.
36. QULNTANA-ORTÍ, G., QULNTANA-ORTÍ, E. S., CHAN, E., VAN ZEE,F.G., AND VAN DE GEIJN, R. A. 2008b. Scheduling of QR factorization algorithms on SMP and multi-core architectures. In Proceedings of the 16th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (Toulouse, France). 301-307.
37. QULNTANA-ORTÍ, G., QULNTANA-ORTÍ, E. S., REMÓn, A., AND VAN DE GEIJN, R. 2008c. An algorithm- by-blocks for SuperMatrix band Cholesky factorization. In Proceedings of the 8th International Meeting on High-Performance Computing for Computational Science (Toulouse, France). 1-13.
38. SKAGESTELN, G. 1972. Rekursiv unterteilte matrizen sowie methoden zur erstellung von rechnerprogrammen fur ihre verarbeitung. Ph.D. dissertation. Universiẗt Stuttgart, Stuttgart, Germany
39. SORENSEN, D. C. 1985. Analysis of pairwise pivoting in Gaussian elimination. IEEE Trans. Comput. 34, 3, 274-278.
40. STRAZDINS, P. 2001. A comparison of lookahead and algorithmic blocking techniques for parallel matrix factorization. Int. J. Parall. Distrib. Syst. Netw. 4, 1, 26-35.
41. TOLEDO, S. 1999. A survey of out-of-core algorithms in numerical linear algebra. In External Memory Algorithms, J. Abello and J. S. Vitter, Eds. American Mathematical Society, Boston, 161-179.
42. VALSALAM,V. AND SKJELLUM, A. 2002. A framework for high-performance matrix multiplication based on hierarchical abstractions, algorithms and optimized low-level kernels. Concurr. Com- putat. Pract. Exper. 14, 10, 805-840.
43. VAN DE GEIJN, R. A. 1997. Using PLAPACK: Parallel Linear Algebra Package. The MIT Press.
44. VAN DE GEIJN,R.A. AND QUINTANA-ORTÍ, E. S. 2008. The Science of Programming Matrix Computations. www.lulu.com.
45. VON HANXLEDEN, R., KENNEDY, K., KOELBEL, C. H., DAS, R., AND SALTZ, J. H. 1992. Compiler analysis for irregular problems in Fortran D. In Proceedings of the 5th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol.757. Springer, Berlin/Heidelberg, Germany, 97-111.
46. WISE,D.S., FRENS,J.D., GU,Y., AND ALEXANDER, G. A. 2001. Language support for Morton-order matrices. in Proceedings of the 8th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Snowbird, 24-33.