The impact of high-performance computing in the solution of linear systems: Trends and problems

Journal of Computational and Applied Mathematics

Volumn 123, Issue 1-2, 2000, Pages 515-530

  Duff, Iain S ^a  

^a RUTHERFORD APPLETON LABORATORY   (United Kingdom)

Author keywords

65F05; 65F50; Direct methods; Matrix factorization; Multifrontal methods; Parallelism; Sparse matrices

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; COMPUTER SOFTWARE; DIGITAL ARITHMETIC; ITERATIVE METHODS; LEAST SQUARES APPROXIMATIONS; MATRIX ALGEBRA; PARALLEL PROCESSING SYSTEMS;

DIRECT METHOD; MATRIX FACTORIZATION; MULTIFRONTAL METHODS; SPARSE MATRICES;

LINEAR EQUATIONS;

EID: 0034319806     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0377-0427(00)00401-5     Document Type: Article

References (89)

1. M. Adlers, Computing sparse orthogonal factors in MATLAB, Technical Report LiTH-MAT-R-1998-19, Linköping University, 1999.
2. Alaghband G. Parallel sparse matrix solution and performance. Parallel Comput. 21(9):1995;1407-1430.
3. Alvarado F.L., Pothen A., Schreiber R. Highly parallel sparse triangular solution. George A., Gilbert J.R., Liu J.W.H. Graph Theory and Sparse Matrix Computation. 1993;Springer, Berlin.
4. Alvarado F.L., Schreiber R. Optimal parallel solution of sparse triangular systems. SIAM J. Sci. Comput. 14:1993;446-460.
5. -1 methods IEEE Trans. Power Systems. 3:1990;452-459.
6. P.R. Amestoy, Factorization of large sparse matrices based on a multifrontal approach in a multiprocessor environment, INPT Ph.D. Thesis TH / PA / 91 / 2, CERFACS, Toulouse, France, 1991.
7. Amestoy P.R., Davis T.A., Duff I.S. An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17(4):1996;886-905.
8. Amestoy P.R., Duff I.S. Vectorization of a multiprocessor multifrontal code. Internat. J. SuperComput. Appl. 3:1989;41-59.
9. P.R. Amestoy, I.S. Duff, J.-Y. L'Excellent, Multifrontal parallel distributed symmetric and unsymmetric solvers, Comput. Methods Appl. Mech. Eng. 184 (2000) 501-520.
10. Amestoy P.R., Duff I.S., L'Excellent J.-Y. Multifrontal solvers within the PARASOL environment. Kagstrom B., Dongarra J., Elmroth E., Wasniewski J. Applied Parallel Computing, PARA'98. Lecture Notes in Computer Science. Vol. 1541:1998;7-11 Springer, Berlin.
11. P.R. Amestoy, I.S. Duff, J.-Y. L'Excellent, J. Koster, A fully asynchronous multifrontal solver using distributed dynamic scheduling, Technical Report RAL-TR-1999-059, Rutherford Appleton Laboratory, 1999.
12. Amestoy P.R., Duff I.S., Puglisi C. Multifrontal QR factorization in a multiprocessor environment. Numer. Linear Algebra Appl. 3(4):1996;275-300.
13. B.S. Andersen, F. Gustavson, A. Karaivanov, J. Wasniewski, P.Y. Yalamov, LAWRA - Linear Algebra With Recursive Algorithms, Technical Report UNIC-99-01, UNI-C, Lyngby, Denmark, 1999.
14. Anderson E., Bai Z., Bischof C., Demmel J., Dongarra J., Du Croz J., Greenbaum A., Hammarling S., McKenney A., Ostrouchov S., Sorensen D. LAPACK Users' Guide. 2nd Edition:1995;SIAM Press, Philadelphia, PA.
15. Anderson E.C., Saad Y. Solving sparse triangular systems on parallel computers. Internat. J. High Speed Comput. 1:1989;73-95.
16. C. Ashcraft, R. Grimes, SPOOLES: An object-oriented sparse matrix library. Proceedings of the Ninth SIAM Conference on Parallel Processing, 1999. See http: // www.netlib.org / linalg / spooles .
17. Ashcraft C., Liu J.W.H. Robust ordering of sparse matrices using multisection. SIAM J. Matrix Anal. Appl. 19:1998;816-832.
18. A. Berger, J. Mulvey, E. Rothberg, R. Vanderbei, Solving multistage stochastic programs using tree dissection, Technical Report SOR-97-07, Programs in Statistics and Operations Research, Princeton University, Princeton, NJ, 1995.
19. Blackford L.S., Choi J., Cleary A., D'Azevedo E., Demmel J., Dhillon I., Dongarra J., Hammarling S., Henry G., Petitet A., Stanley K., Walker D., Whaley R.C. ScaLAPACK Users' Guide. 1997;SIAM Press, Philadelphia, PA.
20. J. Borchardt, F. Grund, D. Horn, Parallel numerical methods for large systems of differential-algebraic equations in industrial applications, Technical Report 382, Weierstraß-Institut für Angewandte Analysis und Stochastik, Berlin, 1997.
21. Bunch J.R., Kaufman L., Parlett B.N. Decomposition of a symmetric matrix. Numer. Math. 27:1976;95-110.
22. D.A. Calahan, Parallel solution of sparse simultaneous linear equations, Proceedings 11th Annual Allerton Conference on Circuits and System Theory, University of Illinois, 1973, pp. 729-735.
23. L.M. Carvalho, Preconditioned Schur complement methods in distributed memory environments, INPT Ph.D. Thesis TH / PA / 97 / 41, CERFACS, Toulouse, France, 1997.
24. Davis T.A., Duff I.S. An unsymmetric-pattern multifrontal method for sparse LU factorization. SIAM J. Matrix Anal. Appl. 18(1):1997;140-158.
25. Davis T.A., Yew P.C. A nondeterministic parallel algorithm for general unsymmetric sparse LU factorization. SIAM J. Matrix Anal. Appl. 11:1990;383-402.
26. J.W. Demmel, J.R. Gilbert, X.S. Li, SuperLU users' guide, Technical Report, Computer Science Division, U.C. Berkeley, Berkeley, California, February 1995. (available from netlib).
27. Demmel J.W., Eisenstat S.C., Gilbert J.R., Li X.S., Liu W.H. A supernodal approach to sparse partial pivoting. SIAM J. Matrix Anal. Appl. 20:1999;720-755.
28. F. Dobrian, G. Kumfert, A. Pothen, Object-oriented design for sparse direct solvers, Technical Report 99-2, Institute for Computer Applications in Science and Engineering (ICASE), MS 132C, NASA Langley Research Center, Hampton, VA 23681-0001, USA, 1999.
29. J.J. Dongarra, Performance of various computers using standard linear algebra software, Technical Report CS-89-85, University of Tennessee, Knoxville, Tennessee, 1999. Updated version at Web address http: // www.netlib.org / benchmark / performance.ps .
30. Dongarra J.J., Bunch J.R., Moler C.B., Stewart G.W. LINPACK User's Guide. 1979;SIAM Press, Philadelphia, PA.
31. Dongarra J.J., Du Croz J.J., Hammarling S., Hanson R.J. An extented set of Fortran Basic Linear Algebra Subprograms. ACM Trans. Math. Software. 14:1988;1-17.
32. J.J. Dongarra, V. Eijkhout, Numerical linear algebra algorithms and software, this issue, J. Comput. Appl. Math. 123 (2000) 489-514.
33. Dongarra J.J., Du Croz J., Duff I.S., Hammarling S. A set of Level 3 basic linear algebra subprograms. ACM Trans. Math. Software. 16:1990;1-17.
34. Dongarra J.J., Duff I.S., Sorensen D.C., van der Vorst H.A. Numerical Linear Algebra for High-Performance Computers. 1998;SIAM Press, Philadelphia, PA.
35. Duff I.S. Full matrix techniques in sparse Gaussian elimination. Watson G.A. Numerical Analysis Proceedings, Dundee 1981. Lecture Notes in Mathematics. Vol. 912:1981;71-84 Springer, Berlin.
36. Duff I.S. Design features of a frontal code for solving sparse unsymmetric linear systems out-of-core. SIAM J. Sci. Statist. Comput. 5:1984;270-280.
37. Duff I.S. Parallel implementation of multifrontal schemes. Parallel Comput. 3:1986;193-204.
38. Duff I.S. Multiprocessing a sparse matrix code on the Alliant FX. / 8 J. Comput. Appl. Math. 27:1989;229-239.
39. Duff I.S., Erisman A.M., Reid J.K. Direct Methods for Sparse Matrices. 1986;Oxford University Press, Oxford, England.
40. I.S. Duff, R.G. Grimes, J.G. Lewis, The Rutherford-Boeing sparse matrix collection, Technical Report RAL-TR-97-031, Rutherford Appleton Laboratory, 1997. Also Technical Report ISSTECH-97-017 from Boeing Information & Support Services, Seattle and Report TR / PA / 97 / 36 from CERFACS, Toulouse.
41. Duff I.S., Koster J. The design and use of algorithms for permuting large entries to the diagonal of sparse matrices. SIAM J. Matrix Anal. Appl. 20(4):1999;889-901.
42. I.S. Duff, J. Koster, On algorithms for permuting large entries to the diagonal of a sparse matrix, Technical Report RAL-TR-1999-030, Rutherford Appleton Laboratory, 1999. Also appeared as Report TR / PA / 99 / 13, CERFACS, Toulouse, France.
43. Duff I.S., Reid J.K. The multifrontal solution of indefinite sparse symmetric linear systems. ACM Trans. Math. Software. 9:1983;302-325.
44. I.S. Duff, H.A. van der Vorst, Developments and trends in the parallel solution of linear systems, Parallel Comput. 25 (1999) 1931-1970.
45. Dumitrescu B., Doreille M., Roch J.-L., Trystram D. Two-dimensional block partitionings for the parallel sparse Cholesky factorization. Numer. Algorithms. 16(1):1997;17-38.
46. C. Fu, X. Jiao, T. Yang, Efficient sparse LU factorization with partial pivoting on distributed memory architectures, Technical report, University of California at Santa Barbara, 1997; IEEE Trans. Parallel Distrib. Systems, submitted for publication.
47. Gallivan K.A., Marsolf B.A., Wijshoff H.A.G. Solving large nonsymmetric sparse linear systems using MCSPARSE. Parallel Comput. 22:1996;1291-1333.
48. George A. Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10:1973;345-363.
49. George A., Liu J.W.H. The design of a user interface for a sparse matrix package. ACM Trans. Math. Software. 5(2):1979;139-162.
50. George A., Liu J.W.H. Computer Solution of Large Sparse Positive Definite Systems. 1981;Prentice-Hall, Englewood Cliffs, NJ.
51. A. George, J.W.H. Liu, E.G. Ng, User's guide for SPARSPAK: Waterloo sparse linear equations package, Technical Report CS-78-30 (Revised), University of Waterloo, Canada, 1980.
52. George A., Poole J.W., Voigt R. Incomplete nested dissection for solving. n × n grid problems SIAM J. Numer. Anal. 15:1978;663-673.
53. Geschiere J.P., Wijshoff H.A.G. Exploiting large grain parallelism in a sparse direct linear system solver. Parallel Comput. 21(8):1995;1339-1364.
54. Gilbert J.R., W.H. Liu J. Elimination structures for unsymmetric sparse LU factors. SIAM J. Matrix Anal. Appl. 14:1993;334-354.
55. A. Gupta, M. Joshi, V. Kumar, WSSMP: Watson Symmetric Sparse Matrix Package. Users Manual: Version 2.0 β , Technical Report RC 20923 (92669), IBM T. J. Watson Research Centre, P.O. Box 218, Yorktown Heights, NY 10598, July 1997.
56. A. Gupta, G. Karypis, V. Kumar, Highly scalable parallel algorithms for sparse matrix factorization, Technical Report TR-94-63, Department of Computer Science, University of Minnesota, 1994.
57. Gupta A., Karypis G., Kumar V. Highly scalable parallel algorithms for sparse matrix factorization. IEEE Trans. Parallel Distrib. Systems. 8:1997;502-520.
58. Gustavson F.G. Recursion leads to automatic variable blocking for dense linear algebra algorithms. IBM J. Res. Develop. 41:1997;737-755.
59. Gustavson F., Henriksson A., Jonsson I., Kågström B., Ling P. Recursive blocked data formats and BLASs for dense linear algebra algorithms. Kågstrom B., Dongarra J., Elmroth E., Wasniewski J. Applied Parallel Computing, Fourth International Workshop, PARA'98. 1998;195-206 Springer, Berlin.
60. Gustavson F.G., Liniger W.M., Willoughby R.A. Symbolic generation of an optimal Crout algorithm for sparse systems of linear equations. J. ACM. 17(1):1970;87-109.
61. M.T. Heath, P. Raghavan, Performance of a fully parallel sparse solver, Proceedings of SHPCC '94, Scalable High-Performance Computing Conference, May 23-25, 1994, Knoxville, Tennessee, Los Alamitos, California, IEEE Computer Society Press, Silver Spring, MD, 1994, pp. 334-341.
62. B. Hendrickson, R. Leland, The CHACO User's Guide, Version 2.0, Technical Report SAND94-2692, Sandia National Laboratories, Albuquerque, October 1994.
63. Hendrickson B., Rothberg E. Improving the runtime and quality of nested dissection ordering. SIAM J. Sci. Comput. 20:1998;468-489.
64. HSL, Harwell Subroutine Library, A Catalogue of Subroutines (Release 12). AEA Technology, Harwell Laboratory, Oxfordshire, England, 1996. For information concerning HSL contact: Dr Nick Brealey, HSL Manager, AEA Technology Products & Systems, Culham Science Centre, Oxon OX14 3ED, England (tel: +44-1235-463404, fax: +44-1235-463480, email: hsl@aeat.co.uk).
65. G. Karypis, V. Kumar, Parallel multilevel graph partitioning, Technical Report TR-95-036, Department of Computer Science, University of Minnesota, May 1995.
66. G. Karypis, V. Kumar, METIS: Unstructured graph partitioning and sparse matrix ordering system, Technical Report TR 97-061, Department of Computer Science, University of Minnesota, Minnesota, 1997.
67. J. Koster, On the parallel solution and the reordering of unsymmetric sparse linear systems. INPT Ph.D. Thesis TH / PA / 97 / 51, CERFACS, Toulouse, France, 1997.
68. Lawson C.L., Hanson R.J., Kincaid D.R., Krogh F.T. Basic linear algebra subprograms for Fortran usage. ACM Trans. Math. Software. 5:1979;308-323.
69. X.S. Li, J.W. Demmel, Making sparse Gaussian elimination scalable by static pivoting. Proceedings of Supercomputing, Orlando, Florida, November 1998.
70. Liu J.W.H. On the storage requirement in the out-of-core multifrontal method for sparse factorization. ACM Trans. Math. Software. 12:1987;249-264.
71. Liu J.W.H. Reordering sparse matrices for parallel elimination. Parallel Comput. 11:1989;73-91.
72. Markowitz H.M. The elimination form of the inverse and its application to linear programming. Management Sci. 3:1957;255-269.
73. Matstoms P. Parallel sparse QR factorization on shared memory architectures. Parallel Comput. 21:1995;473-486.
74. Ng E.G., Peyton B.W. Block sparse Cholesky algorithms on advanced uniprocessor computers. SIAM J. Sci. Comput. 14:1993;1034-1056.
75. B.W. Peyton, A. Pothen, X. Yuan, Partitioning a chordal graph into transitive subgraphs for parallel sparse triangular solution, Technical Report ORNL / TM-12270, Engineering Physics and Mathematics Division, Oak Ridge National Laboratory, Tennessee, December 1992.
76. Pothen A., Simon H.D., Liou K.P. Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl. 11(3):1990;430-452.
77. P. Raghavan, Efficient parallel sparse triangular solution using selective inversion, Technical Report CS-95-314, Department of Computer Science, University of Tennessee, Knoxville, Tennessee, 1995.
78. Raghavan P. Parallel ordering using edge contraction. Parallel Comput. 23(8):1997;1045-1067.
79. E. Rothberg, Exploring the tradeoff between imbalance and separator size in nested dissection ordering, Technical Report Unnumbered, Silicon Graphics Inc, 1996.
80. Rothberg E. Performance of panel and block approaches to sparse Cholesky factorization on the iPSC. / 860 and Paragon multicomputers SIAM J. Sci. Comput. 17(3):1996;699-713.
81. E. Rothberg, A. Gupta. An evaluation of left-looking, right-looking and multifrontal approaches to sparse Cholesky factorization on hierarchical-memory machines, Technical Report STAN-CS-91-1377, Department of Computer Science, Stanford University, 1991.
82. E. Rothberg, R. Schreiber, Improved load distribution in parallel sparse Cholesky factorization, Technical Report 94-13, Research Institute for Advanced Computer Science, 1994.
83. Y. Saad, H.A. van der Vorst, Iterative solution of linear systems in the 20th century, this issue, J. Comput. Appl. Math. 123 (2000) 1-33.
84. Tinney W.F., Walker J.W. Direct solutions of sparse network equations by optimally ordered triangular factorization. Proc. IEEE. 55:1967;1801-1809.
85. van der Stappen A.F., Bisseling R.H., van de Vorst J.G.G. Parallel sparse LIU decomposition on a mesh network of transputers. SIAM J. Matrix Anal. Appl. 14:1993;853-879.
86. R.C. Whaley, LAPACK Working Note 73: Basic Linear Algebra Communication Subprograms: analysis and implementation across multiple parallel architectures, Technical Report CS-94-234, Computer Science Department, University of Tennessee, Knoxville, Tennessee, May 1994.
87. Wilkinson J.H., Reinsch C. Handbook for Automatic Computation, Volume II Linear Algebra. 1971;Springer, Berlin.
88. Z. Zlatev, J. Waśniewski, P.C. Hansen, Tz. Ostromsky, PARASPAR: a package for the solution of large linear algebraic equations on parallel computers with shared memory, Technical Report 95-10, Technical University of Denmark, Lyngby, 1995.
89. Z. Zlatev, J. Waśniewski, K. Schaumburg. Introduction to PARASPAR, Solution of large and sparse systems of linear algebraic equations, specialised for parallel computers with shared memory, Technical Report 93-02, Technical University of Denmark, Lyngby, 1993.