Self-adapting software for numerical linear algebra and LAPACK for clusters

Parallel Computing

Volumn 29, Issue 11-12 SPEC.ISS., 2003, Pages 1723-1743

  Chen, Zizhong ^a   Dongarra, Jack ^a   Luszczek, Piotr ^a   Roche, Kenneth ^a  

^a University of Tennessee   (United States)

Author keywords

High performance computing; LAPACK; Linear algebra; Numerical parallel libraries; PBLAS; ScaLAPACK; Self adapting software

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; COMPUTER SOFTWARE; LINEAR ALGEBRA; MATHEMATICAL MODELS;

NUMERICAL PARALLEL LIBRARIES;

PARALLEL PROCESSING SYSTEMS;

EID: 0242658775     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2003.05.014     Document Type: Article

References (69)

1. Moore G.E. Cramming more components onto integrated circuits. Electronics. 38(8):1965.
2. H.W. Meuer, E. Strohmaier, J.J. Dongarra, H.D. Simon, Top 500 Supercomputer Sites, 20th ed., The report can be downloaded from: 〈http://www.netlib.org/benchmark/top500.html〉, November 2002.
3. Brooks F.P. Jr. The Mythical Man-Month: Essays on Software Engineering. second ed. 1995;Addison Wesley Professional.
4. F.P. Brooks Jr., No silver bullet: essence and accidents of software engineering, Information Processing.
5. J. Dongarra, V. Eijkhout, Self-adapting numerical software for next generation applications, Technical Report, Innovative Computing Laboratory University of Tennessee, Available from: 〈http://icl.cs.utk.edu/iclprojects/pages/sans.html〉, August 2002.
6. Roche K.J., Dongarra J.J. Deploying parallel numerical library routines to cluster computing in a self adapting fashion. Joubert, Murli, Peters, Vanneschi Parallel Computing: Advances and Current Issues. 2002;Imperial College Press, London, England.
7. Garey M., Johnson D. Computers and Intractability: A Guide to the Theory of NP-Completeness. 1979;W.H. Freeman and Company, New York.
8. A compendium of NP optimization problems, Available from: 〈http://www.nada.kth.se/theory/problemlist.html〉.
9. Hochbaum D., Shmoys D. A polynomial approximation scheme for machine scheduling on uniform processors: using the dual approach. SIAM Journal of Computing. 17:1988;539-551.
10. Lenstra J., Shmoys D., Tardos E. Approximation algorithms for scheduling unrelated parallel machines. Mathematical Programming. 46:1990;259-271.
11. Gergov J. Approximation algorithms for dynamic storage allocation. Proceedings of the 4th Annual European Symposium on Algorithms. Lecture Notes in Computer Science. 1136:1996;52-56 Springer-Verlag.
12. Amaldi E., Kann V. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science. 209:1998;237-260.
13. Kann V. Strong lower bounds on the approximability of some NPO PB - complete maximization problems. Proceedings of the 20th International Symposium on Mathematical Foundations of Computer Science. Lecture Notes in Computer Science. 969:1995;227-236 Springer-Verlag.
14. Strassen V. Gaussian elimination is not optimal. Numerical Mathematics. 13:1969;354-356.
15. Coppersmith D., Winograd S. Matrix multiplication via arithmetic progressions. Journal of Symbolic Computation. 9:1990;251-280.
16. Demmel J.W. Applied Numerical Linear Algebra. 1997;Society for Industrial and Applied Mathematics, Philadelphia.
17. Dongarra J.J., Duff I.S., Sorensen D.C., van der Vorst H.A. Numerical Linear Algebra for High-Performance Computers. 1998;Society for Industrial and Applied Mathematics, Philadelphia.
18. Gustavson F.G. Recursion leads to automatic variable blocking for dense linear-algebra algorithms. IBM Journal of Research and Development. 41(6):1997;737-755.
19. Toledo S. Locality of reference in LU decomposition with partial pivoting. SIAM Journal on Matrix Analysis and Applications. 18(4):1997;1065-1081.
20. G. Ammons, T. Ball, J.R. Larus, Exploiting hardware performance counters with flow and context sensitive profiling, in: Proceedings of ACM SIGPLAN'97 Conference on Programming Language Design and Implementation, Las Vegas, Nevada, USA, 1997.
21. T. Ball, J.R. Larus, Efficient path profiling, in: Proceedings of MICRO '96, Paris, France, 1996.
22. Chang P.P., Mahlke S.A., Hwu W.W. Using profile information to assist classic code optimization. Software Practice and Experience. 21(12):1991;1301-1321.
23. Rice J.R. On the construction of poly-algorithms for automatic numerical analysis. Klerer M., Reinfelds J. Interactive Systems for Experimental Applied Mathematics. 1968;31-313 Academic Press.
24. Whaley R.C., Petitet A., Dongarra J.J. Automated empirical optimizations of software and the ATLAS project. Parallel Computing. 27(1-2):2001;3-35.
25. Dongarra J.J., Whaley C.R. Automatically tuned linear algebra software (ATLAS). Proceedings of SC'98 Conference. 1998;IEEE.
26. Dongarra J.J., Croz J.D., Duff I.S., Hammarling S. Algorithm 679: a set of level 3 basic linear algebra subprograms. ACM Transactions on Mathematical Software. 16:1990;1-17.
27. Dongarra J.J., Croz J.D., Duff I.S., Hammarling S. A set of level 3 basic linear algebra subprograms. ACM Transactions on Mathematical Software. 16:1990;18-28.
28. Dongarra J.J., Croz J.D., Hammarling S., Hanson R. An extended set of FORTRAN basic linear algebra subprograms. ACM Transactions on Mathematical Software. 14:1988;1-17.
29. Dongarra J.J., Croz J.D., Hammarling S., Hanson R. Algorithm 656: an extended set of FORTRAN basic linear algebra subprograms. ACM Transactions on Mathematical Software. 14:1988;18-32.
30. Anderson E., Bai Z., Bischof C., Blackford S.L., Demmel J.W., Dongarra J.J., Croz J.D., Greenbaum A., Hammarling S., McKenney A., Sorensen D.C. LAPACK User's Guide. third ed. 1999;Society for Industrial and Applied Mathematics, Philadelphia.
31. Bilmes J., Asanovic K., Chin C., Demmel J. Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology. Proceedings of International Conference on Supercomputing. 1997;ACM SIGARC, Vienna, Austria.
32. R. Weiss, H. Haefner, W. Schoenauer, LINSOL (LINear SOLver) - Description and User's Guide for the Parallelized Version, University of Karlsruhe Computing Center, 1995.
33. Barrett R., Berry M., Dongarra J., Eijkhout V., Romine C. Algorithmic bombardment for the iterative solution of linear systems: a poly-iterative approach. Journal of Computational and Applied Mathematics. 74(1-2):1996;91-109.
34. R. Agarwal, F. Gustavson, M. Zubair, A high-performance algorithm using preprocessing for the sparse matrix-vector multiplication, in: Proceedings of International Conference on Supercomputing, 1992.
35. E.-J. Im, K. Yelick, Optimizing sparse matrix-vector multiplication on SMPs, in: Ninth SIAM Conference on Parallel Processing for Scientific Computing, San Antonio, Texas, 1999.
36. E.-J. Im, Automatic optimization of sparse matrix-vector multiplication, Ph.D. Thesis, University of California, Berkeley, California, 2000.
37. Toledo S. Improving the memory-system performance of sparse matrix-vector multiplication. IBM Journal of Research and Development. 41(6):1997.
38. A. Pinar, M.T. Heath, Improving performance of sparse matrix-vector multiplication, in: Proceedings of SC'99, 1999.
39. Bik A., Wijshoff H. Advanced compiler optimizations for sparse computations. Journal of Parallel and Distributing Computing. 31:1995;14-24.
40. J. Ostergaard, OptimQR - a software package to create near-optimal solvers for sparse systems of linear equations, Available from: 〈http://ostenfeld.dk/jakob/OptimQR/〉.
41. S. Vadhiyar, G. Fagg, J.J. Dongarra, Performance modeling for self adapting collective communications for MPI, In: Los Alamos Computer Science Institute Symposium (LACSI 2001), Sante Fe, New Mexico, 2001.
42. Dongarra J.J., Luszczek P., Petitet A. The LINPACK benchmark: past, present, and future. Concurrency and Computation: Practice and Experience. 15:2003;1-18.
43. M. Frigo, S.G. Johnson, FFTW: an adaptive software architecture for the FFT, in: Proceedings International Conference on Acoustics, Speech, and Signal Processing, Seattle, Washington, USA, 1998.
44. M. Frigo, A fast Fourier transform compiler, in: Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation, Atlanta, Georgia, USA, 1999.
45. D. Mirkovic, S.L. Johnsson, Automatic performance tuning in the UHFFT library, in: 2001 International Conference on Computational Science, San Francisco, California, USA, 2001.
46. Foster I., Kesselman C. The Grid: Blueprint for a New Computing Infrastructure. 1999;Morgan Kaufmann, San Francisco.
47. Berman F. The GrADS project: software support for high level grid application development. International Journal of High Performance Computing Applications. 15:2001;327-344.
48. Petitet A., Blackford S., Dongarra J., Ellis B., Fagg G., Roche K., Vadhiyar S. Numerical libraries and the grid. International Journal of High Performance Computing Applications. 15:2001;359-374.
49. Manteuffel T.A. Adaptive procedure for estimating parameters for the nonsymmetric Tchebyshev iteration. Numerische Mathematik. 31:1978;183-208.
50. Manteuffel T.A. The Tchebyshev iteration for nonsymmetric linear systems. Numerische Mathematik. 28:1977;307-327.
51. Kaasschieter E.F. A practical termination criterion for the Conjugate Gradient method. BIT. 28:1988;308-322.
52. V. Eijkhout, Computational variants of the CGS and BiCGSTAB methods, Technical Report CS-94-241, Computer Science Department, The University of Tennessee Knoxville, August 1994 (Also LAPACK Working Note No. 78).
53. V. Eijkhout, Computational variants of the CGS and BiCGSTAB methods, Technical Report CS-94-241, Computer Science Department, The University of Tennessee Knoxville, August 1994 (Also LAPACK Working Note No. 78).
54. J. Choi, J. Dongarra, S. Ostrouchov, A. Petitet, D. Walker, R.C. Whaley, A proposal for a set of parallel basic linear algebra subprograms, Technical Report CS-95-292, University of Tennessee Knoxville, LAPACK Working Note 100, May 1995.
55. E. Elmroth, F.G. Gustavson, New serial and parallel recursive QR factorization algorithms for SMP systems, in: Proceedings of PARA 1998, 1998.
56. D. Irony, S. Toledo, Communication-efficient parallel dense LU using a 3-dimensional approach, in: Proceedings of the 10th SIAM Conference on Parallel Processing for Scientific Computing, Norfolk, Virginia, USA, 2001.
57. Forum M.P.I. MPI: A Message-Passing Interface Standard. The International Journal of Supercomputer Applications and High Performance Computing. 8:1994.
58. M.P.I. Forum, MPI: A Message-Passing Interface Standard (version 1.1), 1995, Available from: 〈http://www.mpi-forum.org/〉.
59. M.P.I. Forum, MPI-2: Extensions to the Message-Passing Interface, 18 July 1997, Available from: 〈 http://www.mpi-forum.org/docs/mpi-20.ps〉.
60. MPICH, Available from: 〈 http://www.mcs.anl.gov/mpi/mpich/〉.
61. LAM/MPI parallel computing, Available from: 〈 http://www.mpi.nd.edu/lam/〉.
62. Wolski R., Spring N., Hayes H. The network weather service: a distributed resource performance forecasting service for metacomputing. Future Generation Computing Systems. 14:1998.
63. Snavely A.et al. A framework for performance modeling and prediction. Proceedings of Supercomputing 2002. 2002;IEEE.
64. H.P. Forum, High performance fortran language specification, Technical Report CRPC-TR92225, Center for Research on Parallel Computing, Rice University, Houston, TX, May 1993.
65. A. Petitet, Algorithmic redistribution methods for block cyclic decompositions, Ph.D. Thesis, University of Tennessee, Knoxville, Tennessee, 1996.
66. Beaumont O., Legrand A., Rastello F., Robert Y. Dense linear algebra kernels on heterogeneous platforms: redistribution issues. Parallel Computing. 28(2):2002;155-185.
67. Lichtenstein W., Johnson S. Block-cyclic dense linear algebra. SIAM Journal on Scientific and Statistical Computing. 14:1993;1259-1288.
68. Blackford L.S., Choi J., Cleary A., D'Azevedo E.F., Demmel J.W., Dhillon I.S., Dongarra J.J., Hammarling S., Henry G., Petitet A., Stanley K., Walker D.W., Whaley R.C. ScaLAPACK Users' Guide. 1997;Society for Industrial and Applied Mathematics, Philadelphia.
69. Choi J., Dongarra J.J., Ostrouchov S., Petitet A., Walker D.W., Whaley R.C. The design and implementation of the ScaLAPACK LU, QR, and Cholesky factorization routines. Scientific Programming. 5:1996;173-184.