Using mixed precision for sparse matrix computations to enhance the performance while achieving 64-bit accuracy

ACM Transactions on Mathematical Software

Volumn 34, Issue 4, 2008, Pages

  Buttari, Alfredo ^a   Dongarra, Jack ^b   Kurzak, Jakub ^b   Luszczek, Piotr ^c   Tomov, Stanimir ^b  

^a UNIVERSITÉ DE LYON   (France)

^b UNIVERSITY OF TENNESSEE   (United States)

^c University of Tennessee Knoxville ^*

Author keywords

Floating point; Iterative refinement; Linear systems; Precision

Indexed keywords

FIELD PROGRAMMABLE GATE ARRAY (FPGA); FLOATING POINT ARITHMETICS; GRAPHICAL PROCESSING UNITS (GPU); ITERATIVE TECHNIQUES; KRYLOV SUBSPACE METHODS; SPARSE LINEAR ALGEBRA; SPARSE MATRIX COMPUTATIONS;

ALGEBRA; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); LEARNING ALGORITHMS; LINEAR ALGEBRA; MATRIX ALGEBRA;

ITERATIVE METHODS;

EID: 35548992612     PISSN: 00983500     EISSN: 15577295     Source Type: Journal    
DOI: 10.1145/1377596.1377597     Document Type: Article

References (51)

1. AMESTOY, P. R., DUFF, I. S., AND L'EXCELLENT, J.-Y. 2000. Multifrontal parallel distributed symmetric and unsymmetric solvers. Comput. Meth. Appl. Mech. Eng. 184, 501-520.
2. AMESTOY, P. R., DUFF, I. S., L'EXCELLENT, J.-Y., AND KOSTER, J. 2001. A fully asynchronous multifrontal solver using distributed dynamic scheduling. SIAM J. Matrix Anal. Appl. 23, 15-41.
3. AMESTOY, P. R., GUERMOUCHE, A., L'EXCELLENT, J.-Y., AND PRALET, S. 2006. Hybrid scheduling for the parallel solution of linear systems. Paral. Comput. 32, 136-156.
4. ANDERSON, E., BAI, Z., BISCHOF, C., BLACKFORD, S. L., DEMMEL, J. W., DONGARRA, J. J., CROZ, J. D., GREENBAUM, A., HAMMARLING, S., MCKENNEY, A., AND SORENSEN, D. C. 1999. LAPACK User's Guide 3rd Ed. Society for Industrial and Applied Mathematics, Philadelphia, PA.
5. ASHCRAFT, C., GRIMES, R., LEWIS, J., PEYTON, B. W., AND SIMON, H. 1987. Progress in sparse matrix methods in large sparse linear systems on vector supercomputers. Intern. J. of Super-comput. Appl. 1, 10-30.
6. AXELSSON, O. AND VASSILEVSKI, P. S. 1991. A black box generalized conjugate gradient solver with inner iterations and variable-step preconditioning. SIAM J. Matrix Anal. Appl. 12, 4, 625-644.
7. BALAY, S., BUSCHELMAN, K., GROPP, W. D., KAUSHIK, D., KNEPLEY, M. G., MCINNES, L. C., SMITH, B. F., AND ZHANG, H. 2001. PETSc Web page. http://www.mcs.anl.gov/petsc.
8. BARRETT, R., BERRY, M., CHAN, T. F., DEMMEL, J., DONATO, J. M., DONGARRA, J., EIJKHOUT, V., POZO, R., ROMINE, C., AND DER VORST, H. V. 1994. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. Society for Industrial and Applied Mathematics, Philadelphia, PA. http://www.netlib.org/templates/Templates.html.
9. BJÖRCK, A. 1990. Iterative refinement and reliable computing. In Reliable Numerical Computation, M. G. Cox and S. Hammarling, Eds. Oxford University Press, Oxford, UK, 249-266.
10. BUTTARI, A., DONGARRA, J., KURZAK, J., LUSZCZEK, P., AND TOMOV, S. 2006. Computations to enhance the performance while achieving the 64-bit accuracy. Tech. rep. UT-CS-06-584, University of Tennessee Knoxville. LAPACK Working Note 180.
11. DAVIS, T. A. 1999. A combined unifrontal/multifrontal method for unsymmetric sparse matrices. ACM Trans. Math. Softw. 25, 1-19.
12. DAVIS, T. A. 2004. A column pre-ordering strategy for the unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30, 196-199.
13. DEMMEL, J., DONGARRA, J., EIJKHOUT, V., FUENTES, E., PETITET, A., VUDUC, R., WHALEY, R. C., AND YELICK, K. 2005. Self-adapting linear algebra algorithms and software. Proc. IEEE 93, 2. http://www.spiral.net/ieee- special-issue/overview.html.
14. DEMMEL, J. W. 1997. Applied Numerical Linear Algebra. Society for Industrial and Applied Mathematics, Philadelphia, PA.
15. DEMMEL, J. W., EISENSTAT, S. C., GILBERT, J. R., Li, X. S., AND LIU, J. W. H. 1999a. A supernodal approach to sparse partial pivoting. SIAM J. Matrix Anal. Appl. 20, 3, 720-755.
16. DEMMEL, J. W., GILBERT, J. R., AND LI, X. S. 1999b. A asynchronous parallel supernodal algorithm for sparse gaussian elimination. SIAM J. Matrix Anal. Appl. 20, 3, 915-952.
17. ACM Transactions on Mathematical Software, Vol. 34, No. 4, Article 17, Pub. date: July 2008.
18. DONGARRA, J. J. AND EIJKHOUT, V. 2002. Self-adapting numerical software for next generation applications. Tech. rep. ICL-UT-02-07, Innovative Computing Lab, University of Tennessee, La-pack Working Note 157. http://icl.cs.utk.edu/iclprojects/pages/sans.html.
19. DUFF, I. S. AND REID, J. K. 1983. The multifrontal solution of indefinite sparse symmetric linear equations. ACM Trans. Math. Softw. 9, 3, 302-325.
20. DUFF, T A. D. I. S. 1997. An unsymmetric-pattern multifrontal method for sparse LU factorization. SIAMJ. Matrix Anal. Appl. 18, 140-158.
21. EMBREE, M. 2003. The tortoise and the hare restart gmres. SIAM Rev. 45, 259-266.
22. FORSYTHE, G. E. AND MOLER, C. B. 1967. Computer Solution of Linear Algebraic Systems. Prentice-Hall, Englewood Cliffs, NJ.
23. GODDEKE, D., STRZODKA, R., AND TUREK, S. 2005. Accelerating double precision FEM simulations with GPUs. In Simulationstechnique 18th Symposium in Erlangen. F. Hülsemann, M. Kowarschik, and U. Rude, Eds. Vol. Frontiers in Simulation. SCS Publishing House e.V., 139-144.
24. GOLUB, G. H. AND LOAN, C. F. V. 1989. Matrix Computations 2nd Ed. Johns Hopkins University Press, Baltimore, MD.
25. GOLUB, G. H. AND Ye, Q. 2000. Inexact preconditioned conjugate gradient method with innerouter iteration. SIAMJ. Scie. Comput. 21, 4, 1305-1320.
26. GROPP, W. D., KAUSHIK, D. K., KEYES, D. E., AND SMITH, B. F. 2000. Latency, bandwidth, and concurrent issue limitations in high-performance CFD. Tech. rep. ANL/MCS-P850-1000, Argonne National Laboratory.
27. GROPP, W. D., KAUSHIK, D. K., KEYES, D. E., AND SMITH, B. F. 2001. High-performance parallel implicit CFD. Parall. Comput. 27, 4, 337-362.
28. GURTIN, M. E. 1981. An Introduction to Continuum Mechanics. Academic Press, New York, NY.
29. HACKBUSCH, W. 1985. Multigrid Methods and Applications. Springer Series in Computational Mathematics, Vol. 4, Springer-Verlag, Berlin, Germany.
30. HIGHAM, N. J. 2002. Accuracy and Stability of Numerical Algorithms 2nd Ed. Society for Industrial and Applied Mathematics, Philadelphia, PA.
31. LANGOU, J., LANGOU, J., LUSZCZEK, P., KURZAK, J., BUTTARI, A., AND DONGARRA, J. 2006. Exploiting the performance of 32-bit floating point arithmetic in obtaining 64 bit accuracy. In Proceedings of the ACM / IEEE Conference on High Performance Networking and Computing (SC'06). Tampa, FL. http://icl.cs.utk.edu/iter-ref.
32. LI, X. S. 1996. SuperLU software, Ph.D. thesis, Computer Science Department, University of California at Berkeley. http://www.nersc.gov/xiaoye/ SuperLU/.
33. LI, X. S. AND DEMMEL, J. W. 2003. SuperLU-DIST: A scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM Trans. Math. Softw. 29, 110-140.
34. MOLER, C. B. 1967. Iterative refinement in floating point. J. ACM 14, 2, 316-321.
35. NOTAY, Y. 2000. Flexible conjugate gradients. SIAMJ. Scie. Comput. 22, 1444-1460.
36. QUARTERONI, A. AND VALLI, A. 1999. Domain Decomposition Methods for Partial Differential Equations. Oxford University Press, Cambridge, UK.
37. SAAD, Y. 1991. A flexible inner-outer preconditioned GMRES algorithm. Tech. rep. 91-279, Department of Computer Science and Egineering, University of Minnesota, Minneapolis, MN
38. SAAD, Y. 2003. Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, Philadelphia, PA.
39. Saad, Y. AND Schultz, M. H. 1986. GMRES: A generalized minimal residual method for solving nonsymmetric linear systems. SIAMJ. Sci. Statist. Comput., 856-869.
40. Saad, Y. AND WAU, K. 1996. DQGMRES: a direct quasi-minimal residual algorithm based on incomplete orthogonalization. Num. Linear Algeb. Appl. 3, 4, 329-343.
41. SIMONCINI, V. AND SZYLD, D. 2002a. Theory of inexact Krylov subspace methods and applications to scientific computing. Tech. rep. 02-4-12, Department of Mathematics, Temple University.
42. SIMONCINI, V. AND SZYLD, D. B. 2002b. Flexible inner-outer Krylov subspace methods. SIAMJ. Numer. Anal. 40, 6, 2219-2239.
43. SIMONCINI, V. AND SZYLD, D. B. 2005. The effect of non-optimal bases on the convergence of Krylov subspace methods. Numer. Math. 100, 4, 711-733.
44. STEWART, G. W. 2001. Matrix algorithms. Society for Industrial and Applied Mathematics, Philadelphia, PA.
45. STRZODKA, R. AND GÖDDEKE, D. 2006a. Mixed precision methods for convergent iterative schemes. EDGE'06, 23.-24. Chapel Hill, NC.
46. STRZODKA, R. AND GÖDDEKE, D. 2006b. Pipelined mixed precision algorithms on FPGAs for fast and accurate PDE solvers from low precision components. In IEEE Proceedings on Field - Programmable Custom Computing Machines (FCCM'06). IEEE Computer Society Press. To appear.
47. TURNER, K. AND WALKER, H. F. 1992. Efficient high accuracy solutions with gmres(m). SIAM J. Sci. Stat. Comput. 13, 3, 815-825.
48. VAN DEN ESHOF, J., SLEIJPEN, G. L. G., AND VAN GIJZEN, M. B. 2003. Relaxation strategies for nested Krylov methods. Technical report TR/PA/03/27, CERFACS, Toulouse, France.
49. VAN DER VORST, H. A. AND VUIK, C. 1994. GMRESR: a family of nested GMRES methods. Num. Linear Algeb. Appl. 1, 4, 369-386.
50. VUIK, C. 1995. New insights in gmres-like methods with variable preconditioners. J. Comput. Appl. Math. 61, 2, 189-204.
51. WILKINSON, J. H. 1965. The Algebraic Eigenvalue Problem. Oxford University Press, Oxford, UK.