Highly scalable parallel algorithms for sparse matrix factorization

IEEE Transactions on Parallel and Distributed Systems

Volumn 8, Issue 5, 1997, Pages 502-520

  Gupta, Anshul ^b   Karypis, George ^c   Kumar, Vipin ^a,c  

^a IEEE   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c University of Minnesota   (United States)

Author keywords

Cholesky factorization; High performance computing; Parallel processing; Parallel scientific computing; Scalability analysis; Sparse linear systems; Sparse matrices

Indexed keywords

COMPUTER ARCHITECTURE; LEAST SQUARES APPROXIMATIONS; LINEAR PROGRAMMING; MATRIX ALGEBRA; PARALLEL PROCESSING SYSTEMS; PROBLEM SOLVING; SUPERCOMPUTERS;

CHOLESKY FACTORIZATION ALGORITHMS;

PARALLEL ALGORITHMS;

EID: 0031140712     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/71.598277     Document Type: Article

References (62)

1. C. Ashcraft. "The Domain/Segment Partition for the Factorization of Sparse Symmetric Positive Definite Matrices," Technical Report ECA-TR-148, Boeing Computer Services, Seattle, Wash., 1990.
2. C. Ashcraft. "The Fan-Both Family of Column-Based Distributed Cholesky Factorization Algorithms," Graph Theory and Sparse Matrix Computations, A. George, J.R. Gilbert, and J.W.-H. Liu, eds. New York: Springer-Verlag, 1993.
3. C. Ashcraft, S.C. Eisenstat, J.W.-H. Liu, and A.H. Sherman, "A Comparison of Three Column Based Distributed Sparse Factorization Schemes," Technical Report YALEU/DCS/RR-810, Yale Univ., New Haven, Conn., 1990. Also in Proc. Fifth SIAM Conf. Parallel Processing for Scientific Computing, 1991.
4. J.M. Conroy, "Parallel Nested Dissection," Parallel Computing, vol. 16, pp 139-156, 1990.
5. J.M. Conroy, S.G. Kratzer, and R.F. Lucas, "Multifrontal Sparse Solvers in Message Passing and Data Parallel Environments - A Comparitive Study," Proc. PARCO, 1993.
6. I.S. Duff, R.G. Grimes, and J.G. Lewis, "Users' Guide for the Harwell-Boeing Sparse Matrix Collection" (release I), Technical Report TR/PA/92/86, Research and Technology Division, Boeing Computer Services, Seattle, Wash., 1992.
7. I.S. Duff and J.K. Reid, "The Multifrontal Solution of Indefinite Sparse Symmetric Linear Equations," ACM Trans. Math. Software, vol. 9, pp. 302-325, 1983.
8. K.A. Gallivan, R.J. Plemmons, and A.H. Sameh, "Parallel Algorithms for Dense Linear Algebra Computations," SIAM Rev., vol. 32, no. 1, pp. 54-135, Mar. 1990. Also appears in K.A. Gallivan et al. Parallel Algorithms for Matrix Computations. Philadelphia: SIAM, 1990.
9. G.A. Geist and E.G.-Y. Ng, "Task Scheduling for Parallel Sparse Cholesky Factorization," Int'l J. Parallel Programming, vol. 18, no. 4, pp. 291-314, 1989.
10. G.A. Geist and C.H. Romine, "LU Factorization Algorithms on Distributed-Memory Multiprocessor Architectures," SIAM J. Scientific and Statistical Computing, vol. 9, no. 4, pp. 639-649, 1988. Also available as Technical Report ORNL/TM-10383, Oak Ridge National Laboratory, Oak Ridge, Tenn., 1987.
11. A. George, "Nested Dissection of a Regular Finite-Element Mesh," SIAM J. Numerical Analysis, vol. 10, pp. 345-363, 1973.
12. A. George, M.T. Heath, J.W.-H. Liu, and E.G.-Y. Ng, "Sparse Cholesky Factorization on a Local Memory Multiprocessor," SIAM J. Scientific and Statistical Computing, vol. 9, pp. 327-340, 1988.
13. A. George and J.W.-H. Liu, Computer Solution of Large Sparse Positive Definite Systems. Englewood Cliffs, N.J.: Prentice Hall, 1981.
14. A. George, J.W.-H. Liu, and E.G.-Y. Ng, "Communication Reduction in Parallel Sparse Cholesky Factorization on a Hypercube," Hypercube Multiprocessors 1987, M.T. Heath, ed., pp. 576-586. Philadelphia: SIAM, 1987.
15. A. George, J.W.-H. Liu, and E.G.-Y. Ng, "Communication Results for Parallel Sparse Cholesky Factorization on a Hypercube," Parallel Computing, vol. 10, no. 3, pp. 287-298, May 1989.
16. J.R. Gilbert and R. Schreiber, "Highly Parallel Sparse Cholesky Factorization," SIAM J. Scientific and Statistical Computing, vol. 13, pp. 1,151-1,172, 1992.
17. A. Grama, A. Gupta, and V. Kumar, "Isoefficiency: Measuring the Scalability of Parallel Algorithms and Architectures," IEEE Parallel and Distributed Technology, vol. 1, no. 3, pp. 12-21, Aug. 1993. Also available as Technical Report TR 93-24, Dept. of Computer Science, Univ. of Minnesota, Minneapolis, Minn.
18. A. Gupta and E. Rothberg, "An Efficient Block-Oriented Approach to Parallel Sparse Cholesky Factorization," Proc. Supercomputing '93, 1993.
19. A. Gupta, "Analysis and Design of Scalable Parallel Algorithms for Scientific Computing," PhD thesis, Univ. of Minnesota, Minneapolis, Minn., 1995.
20. A. Gupta, "Fast and Effective Algorithms for Graph Partitioning and Sparse Matrix Reordering," Technical Report RC 20496 (90799), IBM T.J. Watson Research Center, Yorktown Heights, N.Y., 10 July 1996.
21. A. Gupta, "Graph Partitioning Based Sparse Matrix Ordering Algorithms for Interior-Point Methods," Technical Report RC 20467 (90480), IBM T.J. Watson Research Center, Yorktown Heights, N.Y., 21 May 1996. Also available at ftp://ftp.cs.umn.edu/ users/kumar/anshul/IP-ordering.ps.Z.
22. A. Gupta, "WGPP: Watson Graph Partitioning (and Sparse Matrix Ordering) Package: Users Manual," Technical Report RC 20453 (90427), IBM T. J. Watson Research Center, Yorktown Heights, N.Y., 6 May 1996. Available on the WWW at the IBM Research CyberJournal site at http://watson.ibm.com:8080.
23. A. Gupta and V. Kumar, "Performance Properties of Large Scale Parallel Systems," J. Parallel and Distributed Computing, vol. 19, pp. 234-244, 1993. Also available as Technical Report TR 92-32, Dept. of Computer Science, Univ. of Minnesota, Minneapolis, Minn.
24. A. Gupta and V. Kumar, "A Scalable Parallel Algorithm for Sparse Matrix Factorization," Technical Report 94-19, Dept. of Computer Science, Univ. of Minnesota, Minneapolis, Minn., 1994. A short version appears in Proc. Supercomputing '94. Technical report available in users/kumar at anonymous FTP site ftp.cs.umn.edu.
25. A. Gupta and V. Kumar, "Parallel Algorithms for Forward and Back Substitution in Direct Solution of Sparse Linear Systems," Proc. Supercomputing '95, Dec. 1995.
26. M.T. Heath, E.G.-Y. Ng, and B.W. Peyton, "Parallel Algorithms for Sparse Linear Systems," SIAM Rev., vol. 33, pp. 420-460, 1991. Also appears in K. A. Gallivan et al., Parallel Algorithms for Matrix Computations. Philadelphia: SIAM, 1990.
27. M.T. Heath and P. Raghavan, "Distributed Solution of Sparse Linear Systems," Technical Report 93-1793, Dept. of Computer Science, Univ. of Illinois, Urbana, Ill., 1993.
28. L. Hulbert and E. Zmijewski, "Limiting Communication in Parallel Sparse Cholesky Factorization," SIAM J. Scientific and Statistical Computing, vol. 12, no. 5, pp. 1,184-1,197, Sept. 1991.
29. G. Karypis, A. Gupta, and V. Kumar, "Parallel Formulation of Interior Point Algorithms," Technical Report 94-20, Dept. of Computer Science, Univ. of Minnesota, Minneapolis, Minn., Apr. 1994. A short version appears in Proc. Supercomputing '94.
30. G. Karypis and V. Kumar, "Analysis of Multilevel Graph Partitioning," Technical Report TR 95-037, Dept. of Computer Science, Univ. of Minnesota, 1995.
31. G. Karypis and V. Kumar, "A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs," Technical Report TR 95-035, Dept. of Computer Science, Univ. of Minnesota, 1995.
32. G. Karypis and V. Kumar, "Parallel Multilevel Graph Partitioning," Technical Report TR 95-036, Dept. of Computer Science, Univ. of Minnesota, 1995.
33. S.G. Kratzer and A.J. Cleary, "Sparse Matrix Factorization on simd Parallel Computers," Graph Theory and Sparse Matrix Computations, A. George, J.R. Gilbert, and J.W.-H. Liu, eds. New York: Springer-Verlag, 1993.
34. V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing: Design and Analysis of Algorithms. Redwood City, Calif.: Benjamin/Cummings, 1994.
35. V. Kumar, A. Grama, A. Gupta, and G. Karypis. Solutions Manual for Introduction to Parallel Computing. Redwood City, Calif.: Benjamin/Cummings, 1994.
36. V. Kumar and A. Gupta, "Analyzing Scalability of Parallel Algorithms and Architectures," J. Parallel and Distributed Computing, vol. 22, no. 3, pp. 379-391, 1994. Also available as Technical Report TR 91-18, Dept. of Computer Science Department, Univ. of Minnesota, Minneapolis, Minn.
37. R.J. Lipton, D.J. Rose, and R.E. Tarjan, "Generalized Nested Dissection," SIAM J. Numerical Analysis, vol. 16, pp. 346-358, 1979.
38. R.J. Lipton and R.E. Tarjan, "A Separator Theorem for Planar Graphs," SIAM J. Applied Math., vol. 36, pp. 177-189, 1979.
39. J.W.-H. Liu, "The Multifrontal Method for Sparse Matrix Solution: Theory and Practice," Technical Report CS-90-04, York Univ., Ontario, Canada, 1990. Also appears in SIAM Rev., vol. 34, pp. 82-109, 1992.
40. R.F. Lucas, "Solving Planar Systems of Equations on Distributed-Memory Multiprocessors," PhD thesis, Dept. of Electrical Eng., Stanford Univ., Palo Alto, Calif., 1987.
41. R.F. Lucas, T. Blank, and J.J. Tiemann, "A Parallel Solution Method for Large Sparse Systems of Equations," IEEE Trans. Computer-Aided Design, vol. 6, no. 6, pp. 981-991, Nov. 1987.
42. F. Manne, "Load Balancing in Parallel Sparse Matrix Computations," PhD thesis, Univ. of Bergen, Norway, 1993.
43. M. Mu and J.R. Rice, "A Grid-Based Subtree-Subcube Assignment Strategy for Solving Partial Differential Equations on Hypercubes," SIAM J. Scientific and Statistical Computing, vol. 13, no. 3, pp. 826-839, May 1992.
44. V.K. Naik and M. Patrick, "Data Traffic Reduction Schemes Cholesky Factorization on Aynchronous Multiprocessor Systems," Proc. Supercomputing '89, 1989. Also available as Technical Report RC 14500, IBM T.J. Watson Research Center, Yorktown Heights, N.Y.
45. S.F. Nugent, "The iPSC/2 Direct-Connect Communications Technology," Proc. Third Conf. Hypercubes, Concurrent Computers, and Applications, pp. 51-60, 1988.
46. D.P. O'Leary and G.W. Stewart, "Assignment and Scheduling in Parallel Matrix Factorization," Linear Algebra and its Applications, vol. 77, pp. 275-299, 1986.
47. V. Pan and J.H. Reif, "Efficient Parallel Solution of Linear Systems," Proc. 17th Ann. ACM Symp. Theory of Computing, pp. 143-152, 1985.
48. C.H. Papadimitriou and K. Steiglitz, Combinatorial Optimization, Algorithms and Complexity. Prentice Hall, 1982.
49. A. Pothen, H.D. Simon, and K.-P. Liou, "Partioning Sparce Matrices with Eigenvectors of Graphs," SIAM J. Math. Analysis and Applications, vol. 11, no. 3, pp. 430-452, 1990.
50. A. Pothen, H.D. Simon, and L. Wang, "Spectral Nested Dissection," Technical Report 92-01, Computer Science Dept., Pennsylvania State Univ., University Park, Penn., 1992.
51. A. Pothen, H.D. Simon, L. Wang, and S.T. Bernard, "Towards a Fast Implementation of Spectral Nested Dissection," Proc. Supercomputing '92, pp. 42-51, 1992.
52. A. Pothen and C. Sun, "Distributed Multifrontal Factorization Using Clique Trees," Proc. Fifth SIAM Conf. Parallel Processing for Scientific Computing, pp. 34-40, 1991.
53. R. Pozo and S.L. Smith, "Performance Evaluation of the Parallel Multifrontal Method in a Distributed-Memory Environment," Proc. Sixth SIAM Conf. Parallel Processing for Scientific Computing, pp. 453-456, 1993.
54. P. Raghavan, "Distributed Sparse Matrix Factorization: QR and Cholesky Factorizations," PhD thesis, Pennsylvania State Univ., University Park, Penn., 1991.
55. E. Rothberg, "Performance of Panel and Block Approaches to Sparse Cholesky Factorization on the iPSC/860 and Paragon Systems," Proc. 1994 Scalable High Performance Computing Conf., May 1994.
56. E. Rothberg and A. Gupta, "An Efficient Block-Oriented Approach to Parallel Sparse Cholesky Factorization," Proc. Supercomputing '92, 1992.
57. E. Rothberg and R. Schreiber, "Improved Load Distribution in Parallel Sparse Cholesky Factorization," Proc. Supercomputing '94, 1994.
58. R. Schreiber, "Scalability of Sparse Direct Solvers," Technical Report RIACS TR 92.13, NASA Ames Research Center, Moffet Field, Calif., May 1992. Also appears in Sparse Matrix Computations: Graph Theory Issues and Algorithms (An IMA Workshop Volume), A. George, J.R. Gilbert, and J.W.-H. Liu, eds. New York: Springer-Verlag, 1993.
59. B. Speelpening, "The Generalized Element Method," Technical Report UIUCDCS-R-78-946, Dept. of Computer Science, Univ. of Illinois, Urbana, Ill., Nov. 1978.
60. C. Sun, "Efficient Parallel Solutions of Large Sparse SPD Systems on Distributed-Memory Multiprocessors," Technical Report CTC92TR102, Advanced Computing Research Inst., Center for Theory and Simulation in Science and Eng., Cornell Univ., Ithaca, N.Y., Aug. 1992.
61. S. Venugopal and V.K. Naik, "Effects of Partitioning and Scheduling Sparse Matrix Factorization on Communication and Load Balance," Proc. Supercomputing '91, pp. 866-875, 1991.
62. P.H. Worley and R. Schreiber, "Nested Dissection on a Mesh Connected Processor Array," New Computing Environments: Parallel, Vector, and Systolic, A. Wouk, ed., pp. 8-38. Philadelphia: SIAM, 1986.