Automatic Data Layout for Distributed-Memory Machines

ACM Transactions on Programming Languages and Systems

Volumn 20, Issue 4, 1998, Pages 869-916

  Kennedy, Ken ^a   Kremer, Ulrich ^b  

^a Rice University   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTER SIMULATION; DATA STRUCTURES; DISTRIBUTED COMPUTER SYSTEMS; FORTRAN (PROGRAMMING LANGUAGE); INTEGER PROGRAMMING; PROGRAM COMPILERS; RESPONSE TIME (COMPUTER SYSTEMS); ALGORITHMS; COMPUTER ARCHITECTURE; COMPUTER SOFTWARE PORTABILITY; DATA REDUCTION; EMBEDDED SYSTEMS; GRAPH THEORY; MATHEMATICAL MODELS;

DATA LAYOUT; DISTRIBUTED MEMORY MACHINE; PRODUCTION COMPILERS;

COMPUTER AIDED SOFTWARE ENGINEERING; PARALLEL PROCESSING SYSTEMS;

CODE GENERATION; MATHEMATICAL MAPPINGS;

EID: 0032108102     PISSN: 01640925     EISSN: None     Source Type: Journal    
DOI: 10.1145/291891.291901     Document Type: Article

References (66)

1. ADVE, V., CARLE, A., GRANSTON, E., HIRANANDANI, S., KENNEDY, K., KOELBEL, C., KREMER, U., MELLOR-CRUMMEY, J., TSENG, C.-W., AND WARREN, S. 1994. Requirements for data-parallel programming environments. IEEE Parallel Distrib. Tech. 2, 3, 48-58.
2. ADVE, V. AND MELLOR-CRUMMEY, J. 1998. Advanced code generation for High Performance Fortran. In Compilation Techniques and Run Time Systems for Scalable Parallel Systems. Lecture Notes in Computer Science, vol. 1366. Springer-Verlag, Berlin. To appear.
3. AHO, A. V., SETHI, R., AND ULLMAN, J. 1986. Compilers: Principles, Techniques, and Tools, 2nd ed. Addison-Wesley, Reading, Mass.
4. ALTMAN, E. R., GOVINDARAJAN, R., AND GAO, G. R. 1995. Scheduling and mapping: Software pipelining in the presence of structural hazards. In Proceedings of the SIGPLAN '95 Conference on Programming Language Design and Implementation. ACM, New York, 139-150.
5. ALTMAN, R. G. E. R. AND GAO, G. R. 1994. Minimizing register requirements under resource-constrained rate-optimal software pipelining. In Proceedings of the 27th Annual International Symposium on Microarchitecture.
6. ANDERSON, J. 1997. Automatic computation and data decomposition for multiprocessors. Ph.D. thesis, Stanford University, Palo Alto, Cal.
7. ANDERSON, J. AND LAM, M. 1993. Global optimizations for parallelism and locality on scalable parallel machines. In Proceedings of the SIGPLAN '93 Conference on Programming Language Design and Implementation. ACM, New York, 112-125.
8. AYGUADÉ, E., GARCIA, J., GIRONES, M., LABARTA, J., TORRES, J., AND VALERO, M. 1994. Detecting and using affinity in an automatic data distribution tool. In Proceedings of the 7th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 892. Springer-Verlag, Berlin, 61-75.
9. BALASUNDARAM, V., FOX, G., KENNEDY, K., AND KREMER, U. 1991. A static performance estimator to guide data partitioning decisions. In Proceedings of the 3rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, New York, 213-223.
10. BAU, D., KODUKULA, I., KOTLYAR, V., PINGALI, K., AND STODGHILL, P. 1994. Solving alignment using elementary linear algebra. In Proceedings of the 7th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 892. Springer-Verlag, Berlin, 46-60.
11. BIXBY, R. 1992. Implementing the Simplex method: The initial basis. ORSA J. Comput. 4, 3, 267-284.
12. BIXBY, R., KENNEDY, K., AND KREMER, U. 1994. Automatic data layout using 0-1 integer programming. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT94). 111-122.
13. CALLAHAN, D. AND KENNEDY, K. 1988. Compiling programs for distributed-memory multiprocessors. J. Supercomput. 2, 151-169.
14. CHAPMAN, B., MEHROTRA, P., AND ZIMA, H. 1992. Vienna Fortran - A Fortran language extension for distributed memory multiprocessors. In Languages, Compilers, and Run-Time Environments for Distributed Memory Machines, J. Saltz and P. Mehrotra, Eds. North-Holland, Amsterdam, The Netherlands, 39-62.
15. CHATTERJEE, S., GILBERT, J., SCHREIBER, R., AND TENG, S.-H. 1993. Automatic array alignment in data-parallel programs. In Proceedings of the 20th Annual ACM Symposium on the Principles of Programming Languages. ACM, New York, 16-28.
16. CHATTERJEE, S., GILBERT, J., SCHREIBER, R., AND TENG, S.-H. 1995. Optimal evaluation of array expressions on massively parallel machines. ACM Trans. Program. Lang. Syst. 17, 1 (Jan.), 123-156.
17. CHATTERJEE, S., GILBERT, J. R., SCHREIBER, R., AND SHEFFLER, T. 1994. Array distribution in data-parallel programs. Lecture Notes in Computer Science, vol. 892. Springer-Verlag, Berlin, 76-91.
18. CORMEN, T. H., LEISERSON, C. E., AND RIVEST, R. L. 1990. Introduction to Algorithms. The MIT Press, Cambridge, Mass.
19. DAMIAN-IORDACHE, M. AND PEMMARAJU, S. 1998. Automatic data decompositions for message-passing machines. In Proceedings of the 10th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 1366. Springer-Verlag, Berlin, 64-78.
20. D'HOLLANDER, E. 1989. Partitioning and labeling of index sets in do loops with constant dependence. In Proceedings of the 1989 International Conference on Parallel Processing. II139-II144.
21. DYBVIG, R. K. 1987. The Scheme Programming Language. Prentice-Hall, Inc., Englewood Cliffs, N.J.
22. FEAUTRIER, P. 1994. Fine-grain scheduling under resource constraints. In Proceedings of the 7th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 892. Springer-Verlag, Berlin, 1-15.
23. FOX, G., HIRANANDANI, S., KENNEDY, K., KOELBEL, C., KREMER, U., TSENG, C., AND WU, M. 1990. Fortran D language specification. Tech. Rep. TR90-141, Dept. of Computer Science, Rice University, Houston, Tex.
24. FOX, G., JOHNSON, M., LYZENGA, G., OTTO, S., SALMON, J., AND WALKER, D. 1988. Solving Problems on Concurrent Processors. Vol. 1. Prentice-Hall, Englewood Cliffs, N.J.
25. GARCIA, J. 1997. Automatic data distribution for massively parallel processors. Ph.D. thesis, Universitat Politècnica de Catalunya, Barcelona, Spain.
26. GARCIA, J., AYGUADE, E., AND LABARTA, J. 1995. A novel approach towards automatic data distribution. In Proceedings of the Workshop on Automatic Data Layout and Performance Prediction (AP'95). Available as Rice University Tech. Rep. CRPC-TR95-548.
27. GARCIA, J., AYGUADÉ, E., AND LABARTA, J. 1996. Dynamic data distribution with control flow analysis. In Proceedings of Supercomputing '96. Available at URL http://www.supercomp.org/sc96/proceedings.
28. GUPTA, M. 1992. Automatic data partitioning on distributed memory multicomputers. Ph.D. thesis, University of Illinois at Urbana-Champaign, Ill.
29. GUPTA, M. AND BANERJEE, P. 1992. Demonstration of automatic data partitioning techniques for parallelizing compilers on multicomputers. IEEE Trans. Parallel Distrib. Syst. 3, 2 (Apr.), 179-193.
30. HECHT, M. S. 1977. Flow Analysis of Computer Programs. North Holland.
31. HIRANANDANI, S., KENNEDY, K., KOELBEL, C., KREMER, U., AND TSENG, C. 1991. An overview of the Fortran D programming system. In Proceedings of the 4th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 589. Springer-Verlag, Berlin.
32. HUANG, C.-H. AND SADAYAPPAN, P. 1991. Communication-free hyperplane partitioning of nested loops. In Proceedings of the 4th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 589. Springer-Verlag, Berlin.
33. HUDAK, D. AND ABRAHAM, S. 1990. Compiler techniques for data partitioning of sequentially iterated parallel loops. In Proceedings of the 1990 ACM International Conference on Super-computing. ACM, New York, 187-200.
34. KELLY, W. AND PUGH, W. 1996. Minimizing communication while preserving parallelism. In Proceedings of the 1996 ACM International Conference on Supercomputing. ACM, New York, 52-60.
35. KENNEDY, K. AND KREMER, U. 1995. Automatic data layout for High Performance Fortran. In Proceedings of Supercomputing '95, Available at URL http://www.supercomp.org/sc95/proceedings.
36. KNOBE, K., LUKAS, J., AND STEELE, G. 1990. Data optimization: Allocation of arrays to reduce communication on SIMD machines. J. Parallel Distrib. Comput. 8, 2 (Feb.), 102-118.
37. KNOBE, K. AND NATARAJAN, V. 1993. Data optimization: Minimizing residual interprocessor data motion on SIMD machines. J. Supercomput. 7, 387-415.
38. KOELBEL, C., LOVEMAN, D., SCHREIBER, R., STEELE, JR., G., AND ZOSEL, M. 1994. The High Performance Fortran Handbook. The MIT Press, Cambridge, Mass.
39. KOELBEL, C. AND MEHROTRA, P. 1991. Compiling global name-space parallel loops for distributed execution. IEEE Trans. Parallel Distrib. Syst. 2, 4 (Oct.), 440-451.
40. KREMER, U. 1995. Automatic data layout for distributed memory machines. Ph.D. thesis, Rice University, Houston, Tex. Available as CRPC-TR95-559-S.
41. KREMER, U. 1997. Optimal and near-optimal solutions for hard compilation problems. Parallel Process. Lett. 7, 2, 371-378.
42. KREMER, U. 1998. Automatic data layout with read-only replication and memory constraints. In Proceedings of the 10th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 1366. Springer-Verlag, Berlin, 419-422.
43. KREMER, U. AND RAMÉ, M. 1994. Compositional oil reservoir simulation in Fortran D: A feasibility study on Intel iPSC/860. Int. J. Supercomput. Appl. 8, 2 (Summer), 119-128.
44. LEE, P. AND TSAI, T.-B. 1993. Compiling efficient programs for tightly-coupled distributed memory computers. In Proceedings of the 1993 International Conference on Parallel Processing. 11161-11165.
45. LI, J. 1991. Compiling Crystal for distributed memory machines. Ph.D. thesis, Dept. of Computer Science, Yale University, New Haven, Conn.
46. LI, J. AND CHEN, M. 1991a. Compiling communication-efficient programs for massively parallel machines. IEEE Trans. Parallel Distrib. Syst. 2, 3 (July), 361-376.
47. LI, J. AND CHEN, M. 1991b. The data alignment phase in compiling programs for distributed-memory machines. J. Parallel Distrib. Comput. 13, 2 (Oct.), 213-221.
48. NEMHAUSER, G. L. AND WOLSEY, L. A. 1988. Integer and Combinatorial Optimization. John Wiley & Sons, New York.
49. NING, Q., DONGEN, V. V., AND GAO, G. R. 1995. Automatic data and computation decomposition for distributed memory machines. In Proceedings of the 28th Annual Hawaii International Conference on System Sciences. IEEE, New York.
50. NING, Q. AND GAO, G. R. 1993. A novel framework of register allocation for software pipelining. In Proceedings of the 20th Annual ACM Symposium on the Principles of Programming Languages. ACM, New York, 29-42.
51. PALERMO, D. 1996. Compiler techniques for optimizing communication and data distribution for distributed-memory multicomputers. Ph.D. thesis, University of Illinois at Urbana-Champaign, Ill. Available as CRHC-96-09.
52. PALERMO, D. AND BANERJEE, P. 1995. Automatic selection of dynamic data partitioning schemes for distributed-memory multicomputers. Technical Report CRHC-95-09, Center for Reliable and High-Performance Computing, Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Ill.
53. PALERMO, D. AND BANERJEE, P. 1996. Interprocedural array redistribution with data-flow analysis. In Proceedings of the 9th Workshop on Languages and Compilers for Parallel Computing. Lecture Notes in Computer Science, vol. 1239. Springer-Verlag, Berlin, 435-449.
54. PHILIPPSEN, M. 1995. Automatic alignment of array data and processes to reduce communication time on DMPPs. In Proceedings of the 5th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM, New York, 156-165.
55. PRESBERG, D. L. 1996. Comparison of 3 HPF compilers for the IBM SP. NHSE Rev. 1996, 2. Available at http://www.crpc.rice.edu/NHSEreview/96-2.html.
56. PUGH, W. 1991. The Omega test: A fast and practical integer programming algorithm for dependence analysis. In Proceedings of Supercomputing '91. 4-13.
57. RAMANUJAM, J. AND SADAYAPPAN, P. 1989. A methodology for parallelizing programs for multicomputers and complex memory multiprocessors. In Proceedings of Supercomputing '89. 637-646.
58. ROGERS, A. AND PINGALI, K. 1989. Process decomposition through locality of reference. In Proceedings of the SIGPLAN '89 Conference on Programming Language Design and Imple-mentation. ACM, New York, 69-80.
59. ROSE, J. AND STEELE, G. 1987. C*: An extended C language for data parallel programming. Technical Report PL87-5, Thinking Machines, Inc., Cambridge, Mass.
60. SHEFFLER, T. J., SCHREIBER, R., PUGH, W., GILBERT, J. R., AND CHATTERJEE, S. 1996. Efficient distribution analysis via graph contraction. Int. J. of Parallel Prog. 24, 6 (Dec.), 599-620.
61. TANDRI, S. AND ABDELRAHMAN, T. 1997. Automatic partitioning of data and computation on scalable shared memory multiprocessors. In Proceedings of the 1997 International Conference on Parallel Processing. 64-73.
62. Thinking Machines Corp. 1989. CM Fortran Reference Manual, Version 5.2-0.6 ed. Thinking Machines Corp., Cambridge, Mass.
63. TSENG, C. 1993. An optimizing Fortran D compiler for MIMD distributed-memory machines. Ph.D. thesis, Rice University, Houston, Tex. Available as Rice COMP TR93-199.
64. WHOLEY, S. 1991. Automatic data mapping for distributed-memory parallel computers. Ph.D. thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pa.
65. WHOLEY, S. 1992. Automatic data mapping for distributed-memory parallel computers. In Proceedings of the 1992 ACM International Conference on Supercomputing. ACM, New York, 25-34.
66. ZIMA, H., BAST, H.-J., AND GERNDT, M. 1988. SUPERB: A tool for semi-automatic MIMD/SIMD parallelization. Parallel Comput. 6, 1-18.