Parallel implementation of BLAS: General techniques for level 3 BLAS

Concurrency Practice and Experience

Volumn 9, Issue 9, 1997, Pages 837-857

  Chtchelkanova, Almadena ^a   Gunnels, John ^a   Morrow, Greg ^a   Overfelt, James ^a   Van De Geijn, Robert A ^a  

^a University of Texas at Austin   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SOFTWARE; CONCURRENCY CONTROL; INTERFACES (COMPUTER); MATRIX ALGEBRA; PARALLEL ALGORITHMS; PERFORMANCE; SUBROUTINES; TECHNOLOGY;

BASIC LINEAR ALGEBRA SUBPROGRAM; INTEL PARAGON SYSTEM; MATRIX MATRIX OPERATION; PARALLEL MATRIX MATRIX MULTIPLICATION;

PARALLEL PROCESSING SYSTEMS;

EID: 0031221523     PISSN: 10403108     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1096-9128(199709)9:9<837::AID-CPE267>3.0.CO;2-2     Document Type: Article

References (46)

1. E. Anderson, A. Benzoni, J. Dongarra, S. Moulton, S. Ostrouchov, B. Tourancheau and R. van de Geijn, 'LAPACK for distributed memory architectures: progress report', in Proceedings of the Fifth SIAM Conference on Parallel Processing for Scientific Computing, SIAM, Philadelphia, 1992, pp. 625-630.
2. J. Choi, J. J. Dongarra, R. Pozo and D. W. Walker, 'Scalapack: A scalable linear algebra library for distributed memory concurrent computers', Proceedings of the Fourth Symposium on the Frontiers of Massively Parallel Computation, IEEE Comput. Soc. Press, 1992, pp. 120-127.
3. J. Demmel, J. Dongarra, R. van de Geijn and D. Walker, 'LAPACK for distributed memory architectures: The next generation', in Proceedings of the Sixth SIAM Conference on Parallel Processing for Scientific Computing, Norfolk, March 1993.
4. J. Dongarra and R. van de Geijn, 'A parallel dense linear solve library routine', in Proceedings of the 1992 Intel Supercomputer Users' Group Meeting, Dallas, Oct. 1992.
5. J. Dongarra, R. van de Geijn and D.Walker, 'Scalability issues affecting the design of a dense linear algebra library', Special Issue on Scalability of Parallel Algorithms, J. Parallel Distrib. Comput., 22, (3), (1994).
6. G. C. Fox, M. A. Johnson, G. A. Lyzenga, S. W. Otto, J. K. Salmon and D. W. Walker, Solving Problems on Concurrent Processors, Vol. 1, Prentice Hall, Englewood Cliffs, N.J., 1988.
7. W. Lichtenstein and S. L. Johnsson, 'Block-cyclic dense linear algebra', Harvard University, Center for Research in Computing Technology, TR-04-92, Jan. 1992.
8. R. van de Geijn, 'Massively parallel LINPACK benchmark on the Intel Touchstone DELTA and iPSC/860 systems: Preliminary report', TR-91-28, Department of Computer Sciences, University of Texas, Aug. 1991.
9. E. Anderson, Z. Bai, C. Bischof, J. Demmel, J. Dongarra, J. DuCroz, A. Greenbaum, S. Hammarling, A. McKenney and D. Sorensen, 'Lapack: A portable linear algebra library for high performance computers', Proceedings of Supercomputing '90, IEEE Press, 1990, pp. 1-10.
10. E. Anderson, Z. Bai, J. Demmel, J. Dongarra, J. DuCroz, A. Greenbaum, S. Hammarling, A. McKenney, S. Ostrouchov and D. Sorensen, LAPACK Users' Guide, SIAM, Philadelphia, 1992.
11. C. L. Lawson, R. J. Hanson, D. R. Kincaid and F. T. Krogh, 'Basic linear algebra subprograms for Fortran usage', TOMS, 5, (3), 308-323 (1979).
12. J. J. Dongarra, J. Du Croz, S. Hammarling and R. J. Hanson, 'An extended set of FORTRAN basic linear algebra subprograms', TOMS, 14, (1), 1-17 (1988).
13. J. J. Dongarra, J. Du Croz, S. Hammarling and I. Duff, 'A set of Level 3 basic linear algebra subprograms', TOMS, 16, (1), 1-16 (1990).
14. J. Choi, J. Dongarra, S. Ostrouchov, A. Petitet, D. Walker, and R. C. Whaley 'A proposal for a set of parallel basic linear algebra subprograms', LAPACK Working Note 100, University of Tennessee, CS-95-292, May 1995.
15. R. van de Geijn and J. Watts, 'SUMMA: Scalable universal matrix multiplication algorithm', TR-95-13, Department of Computer Sciences, University of Texas, April 1995. Also: LAPACK Working Note 96, May 1, Concurrency: Pract. Exp., 9, (4), 255-274 (1997).
16. R. van de Geijn and J. Watts, 'SUMMA: Scalable universal matrix multiplication algorithm', TR-95-13, Department of Computer Sciences, University of Texas, April 1995. Also: LAPACK Working Note 96, May 1, Concurrency: Pract. Exp., 9, (4), 255-274 (1997).
17. J. Choi, J. J. Dongarra and D. W. Walker, 'PUMMA: Parallel universal matrix multiplication algorithms on distributed memory concurrent computers', Concurrency: Pract. Exp., 6, (7), 543-570 (1994).
18. S. Huss-Lederman, E. Jacobson and A. Tsao, 'Comparison of scalable parallel matrix multiplication libraries', in Proceedings of the Scalable Parallel Libraries Conference, Starksville, MS, Oct. 1993.
19. S. Huss-Lederman, E. Jacobson, A. Tsao and G. Zhang, 'Matrix multiplication on the Intel Touchstone DELTA', Concurrency: Pract. Exp., 6, (7), 571-594 (1994).
20. B. Kågström and C. F. Van Loan, Gemm-based level-3 bias, 1991. Theory Center Technical Report, January 1991.
21. P. Mitra, D. Payne, L. Shuler, R. van de Geijn and J. Watts, 'Fast collective communication libraries, please', in the Proceedings of the Intel Supercomputing Users' Group Meeting 1995.
22. R. C. Agarwal, F. G. Gustavson, S. M. Balle, M. Joshi and P. Palkar, 'A high performance matrix multiplication algorithm for MPPs', IBM T.J. Watson Research Center, 1995.
23. A. Chtchelkanova, C. Edwards, J. Gunnels, G. Morrow, J. Overfelt and R A. van de Geijn, 'Towards usable and lean parallel linear algebra libraries', TR-96-09, Department of Computer Sciences, University of Texas, May 1996.
24. C.-T. Ho and S. L. Johnsson, 'Distributed routing algorithms for broadcasting and personalized communication in hypercubes', in Proceedings of the 1986 International Conference on Parallel Processing, IEEE, 1986, pp. 640-648.
25. R. van de Geijn, On global combine operations', J Parallel Distrib. Comput., 22, 324-328 (1994).
26. J. Watts and R. van de Geijn, 'A pipelined broadcast for multidimensional meshes', Parallel Process. Lett., 5, (2), 281-292 (1995).
27. A. Chtchelkanova, J. Gunnels, G. Morrow, J. Overfelt and R. A. van de Geijn, 'Parallel implementation of BLAS: General techniques for Level 3 BLAS', TR-95-40, Department of Computer Sciences, University of Texas, Oct. 1995.
28. C. Edwards, P. Geng, A. Patra, and R. van de Geijn, 'Parallel matrix distributions: have we been doing it all wrong?', Department of Computer Sciences, UT-Austin, Report TR95-39, Oct. 1995.
29. L. Auslander and A.Tsao, On parallelizable eigensolvers', Adv. Appl. Math., 13, 253-261, (1992).
30. Z. Bai and J. Demmel, 'Design of a parallel nonsymmetric eigenroutine toolbox, Part I', Parallel Processing for Scientific Computing, R. Sincovec, D. Keyes, M. Leuze, L. Petzold and D. Reed (Eds.), SIAM Publications, Philadelphia, PA, 1993, pp. 391-398.
31. Z. Bai, J. Demmel, J. Dongarra, A. Petitet, H. Robinson and K. Stanley, The Spectral Decomposition of Nonsymmetric Matrices on Distributed Memory Parallel Computers, LAPACK working note 91, University of Tennessee, Jan. 1995.
32. S. Lederman, A. Tsao and T. Turnbull, 'A parallelizable eigensolver for real diagonalizable matrices with real eigenvalues', Technical Report TR-91-042, Supercomputing Research Center, 1991.
33. K. Klimkowski and R. van de Geijn, 'Anatomy of an out-of-core dense linear solver', Vol III, Algorithms and Applications, Proceedings of the 1995 International Conference on Parallel Processing, pp. 29-33.
34. B. Grayson and R. van de Geijn, 'A high performance parallel strassen implementation'. Parallel Process. Lett., to be published.
35. R. C. Agarwal, F. G. Gustavson and M. Zubair, 'A high performance matrix multiplication algorithm on a distributed-memory parallel computer, using overlapped communication', IBM J. Res. Dev., 673-681 (1994).
36. L. E. Cannon, A Cellular Computer to Implement the Kalman Filter Algorithm, Ph.D. thesis, 1969, Montana State University.
37. J. Choi, J. J. Dongarra and D. W. Walker, 'Level 3 BLAS for distributed memory concurrent computers', CNRS-NSF Workshop on Environments and Tools for Parallel Scientific Computing, Saint Hilaire du Touvet, France, 7-8 Sept. 1992, Elsevier Science Publishers, 1992.
38. J. J. Dongarra, I. S. Duff, D. C. Sorensen and H. A. van der Vorst, Solving Linear Systems on Vector and Shared Memory Computers, SIAM, Philadelphia, 1991.
39. J. J. Dongarra, R. A. van de Geijn and R. Clint Whaley, 'Two dimensional basic linear algebra communication subprograms', in Proceedings of the Sixth SIAM Conference on Parallel Processing for Scientific Computing, Norfolk, March 1993.
40. G. Fox, S. Otto and A. Hey, 'Matrix algorithms on a hypercube I: Matrix multiplication', Parallel Comput., 3, 17-31 (1987).
41. G. H. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins University Press, 2nd edn., 1989.
42. W. Gropp, E. Lusk and A. Skjellum, Using MPI: Portable Programming with the Message-Passing Interface, The MIT Press, 1994.
43. B. Kågström and P. Ling, 'Level 2 and 3 BLAS routines for the IBM 3090 VF/400: Implementation and experiences', Technical Report UMINF-154.88, Information Processing, University of Umeå, S-901 87 Umeå, Sweden, 1988.
44. J. G. Lewis and R. A. van de Geijn, 'Implementing matrix-vector multiplication and conjugate gradient algorithms on distributed memory multicomputers', Supercomputing '93.
45. J. G. Lewis, D. G. Payne and R. A. van de Geijn, 'Matrix-vector multiplication and conjugate gradient algorithms on distributed memory computers', Scalable High Performance Computing Conference, 1994.
46. C. Lin and L. Snyder, 'A matrix product algorithm and its comparative performance on hypercubes', in Proceedings of Scalable High Performance Computing Conference, Q. Stout and M. Wolfe (eds.), IEEE Press, Los Alamitos, CA, 1992, pp. 190-193.