Minimizing data and synchronization costs in one-way communication

IEEE Transactions on Parallel and Distributed Systems

Volumn 11, Issue 12, 2000, Pages 1232-1251

  Kandemir, Mahmut ^a,d   Choudhary, Alok ^b,d   Banerjee, Prithviraj ^b,d   Ramanujam, J ^c,d   Shenoy, Nagaraj ^b,d  

^a The Pennsylvania State University   (United States)

^b Northwestern University   (United States)

^c LOUISIANA STATE UNIVERSITY   (United States)

^d IEEE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPILER OPTIMIZATIONS; GLOBAL DATA FLOW FRAMEWORK; HIGH PERFORMANCE FORTRAN; MESSAGE PASSING; ONE WAY COMMUNICATION; POLYHEDRAL ALGEBRA; REDUNDANT SYNCHRONIZATION;

DATA COMMUNICATION SYSTEMS; DATA FLOW ANALYSIS; FORTRAN (PROGRAMMING LANGUAGE); LINEAR ALGEBRA; PROGRAM COMPILERS; RESPONSE TIME (COMPUTER SYSTEMS); SYNCHRONIZATION; THEOREM PROVING;

PARALLEL PROCESSING SYSTEMS;

EID: 0034482961     PISSN: 10459219     EISSN: None     Source Type: Journal    
DOI: 10.1109/71.895791     Document Type: Article

References (57)

1. V. Adve, J. Mellor-Crummey, and A. Sethi, "An Integer Set Framework for HPF Analysis and Code Generation," Technical Report TR97-275, Computer Science Dept, Rice Univ., 1997.
2. V. Adve and J. Mellor-Crummey, "Advanced Code Generation for High Performance Fortran," Languages, Compilation Techniques, and Run-Time Systems for Scalable Parallel Systems, chapter 18, Springer-Verlag (to appear), 2001.
3. G. Agrawal and J. Saltz, "Interprocedural Data Flow Based Optimizations for Distributed Memory Compilation," Software Practice and Experience, vol. 27, no. 5, pp. 519-545, 1997.
4. A.V. Aho, R. Sethi, and J. Ullman, Compilers: Principles, Techniques, and Tools, second ed. Reading, Mass.: Addison-Wesley, 1986.
5. S. Amarasinghe and M. Lam, "Communication Optimization and Code Generation for Distributed Memory Machines," Proc. SIGPLAN '93 Conf. Programming Language Design and Implementation, pp. 126-138, June 1993.
6. A. Ancourt, F. Coelho, F. Irigoin, and R. Keryell, "A Linear Algebra Framework for Static HPF Code Distribution," Scientific Programming, vol. 6, no. 1, pp. 3-28, Spring 1997.
7. P. Banerjee, J. Chandy, M. Gupta, E. Hodges, J. Holm, A. Lain, D. Palermo, S. Ramaswamy, and E. Su, "The PARADIGM Compiler for Distributed-Memory Multicomputers," Computer, vol. 28, no. 10, pp. 37-47, Oct. 1995.
8. E. Barton, J. Cownie, and M. McLaren, "Message Passing on the Meiko CS-2," Parallel Computing, vol. 20, no. 4, pp. 497-507, Apr. 1994.
9. D. Callahan and K. Kennedy, "Analysis of Interprocedural Side Effects in a Parallel Programming Environment," J. Parallel and Distributed Computing, vol. 5, no. 5, pp. 517-550, Oct. 1988.
10. S. Chakrabarti, M. Gupta, and J.-D. Choi, "Global Communication Analysis and Optimization," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 68-78, May 1996.
11. B. Chapman, P. Mehrotra, and H. Zima, "Programming in Vienna Fortran," Scientific Programming, vol. 1, no. 1, pp. 31-50, Fall 1992.
12. Cray Research Inc., Cray T3D System Architecture Overview, 1993.
13. D. Culler, A. Dusseau, S. Goldstein, A. Krishnamurthy, S. Lumetta, T. von Eicken, and K. Yelick, "Parallel Programming in Split-C," Proc. Supercomputing '93, Nov. 1993.
14. E. Duesterwald, R. Gupta, and M.L. Soffa, "A Practical Data-Flow Framework for Array Reference Analysis and Its Application in Optimizations," Proc. ACM SIGPLAN '93 Conf. Programming Language Design and Implementation, pp. 68-77, June 1993.
15. A. Geist, PVM: Parallel Virtual Machine: A Users' Guide and Tutorial for Networked Parallel Computing. Scientific and Eng. Computation Series, 1994.
16. C. Gong, R. Gupta, and R. Melhem, "Compilation Techniques for Optimizing Communication on Distributed-Memory Systems," Proc. Int'l Conf. Parallel Processing, vol. II, p. 39-46, Aug. 1993.
17. E. Granston and A. Veidenbaum, "Detecting Redundant Accesses to Array Data," Proc. Supercomputing '91, pp. 854-865, Nov. 1991.
18. M. Gupta and P. Banerjee, "Demonstration of Automatic Data Partitioning Techniques for Parallelizing Compilers on Multicomputers," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 179-193, Mar. 1992.
19. M. Gupta and E. Schonberg, "Static Analysis to Reduce Synchronization Costs in Data-Parallel Programs," Proc. ACM Conf. Principles of Programming Languages, pp. 322-332, 1996.
20. M. Gupta, E. Schonberg, and H. Srinivasan, "A Unified Data-Flow Framework for Optimizing Communication," Languages and Compilers for Parallel Computing, K. Pingali et al., eds., Lecture Notes in Computer Science, vol. 892, pp. 266-282, 1995.
21. M. Gupta, S. Midkiff, E. Schonberg, V. Seshadri, D. Shields, K. Wang, W. Ching, and T. Ngo, "An HPF Compiler for the IBM SP2," Proc. Supercomputing 95, Dec. 1995.
22. M.W. Hall, S. Hiranandani, K. Kennedy, and C.-W. Tseng, "Inter-Procedural Compilation of Fortran D for MIMD Distributed-Memory Machines," Proc. Supercomputing '92, Nov. 1992.
23. M.W. Hall, B. Murphy, S. Amarasinghe, S. Liao, and M. Lam, "Inter-Procedural analysis for Parallelization," Proc. Eighth Int'l Workshop Languages and Compilers for Parallel Computers, pp. 61-80, Aug. 1995.
24. R. v. Hanxleden and K. Kennedy, "A Code Placement Framework and Its Application to Communication Generation," Technical Report CRPC-TR93337-S, Center for Research on Parallel Computation, Rice Univ., Oct. 1993.
25. R. v. Hanxleden and K. Kennedy, "Give-N-Take - A Balanced Code Placement Framework," Proc. ACM SIGPLAN '94 Conf. Programming Language Design and Implementation, June 1994.
26. K. Hayashi, T. Doi, T. Horie, Y. Koyanagi, O. Shiraki, N. Imamura, T. Shimizu, H. Ishihata, and T. Shindo, "AP1000+: Architectural Support of Put/Get Interface for Parallelizing Compiler," Proc. Sixth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 196-207, Oct. 1994.
27. J. Heinlein, K. Gharachorloo, S. Dresser, and A. Gupta, "Integrating Message Passing in the Stanford FLASH Multiprocessor," Proc. Sixth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, Oct. 1994.
28. High Performance Fortran Forum, "High Performance Fortran Language Specification," Scientific Programming, vol. 2, nos. 1-2, pp. 1-170, 1993.
29. S. Hinrichs, "Compiler-Directed Architecture-Dependent Communication Optimizations," PhD thesis, School of Computer Science, Carnegie Mellon Univ., Pittsburgh, Pa., June 1995.
30. S. Hinrichs, "Synchronization Elimination in the Deposit Model," Proc. 1996 Int'l Conf. Parallel Processing, pp. 87-94, 1996.
31. S. Hiranandani, K. Kennedy, and C. Tseng, "Compiling Fortran D for MIMD Distributed-Memory Machines," Comm. ACM, vol. 35, no. 8, pp. 66-80, Aug. 1992.
32. M. Kandemir, P. Banerjee, A. Choudhary, J. Ramanujam, and N. Shenoy, "A Generalized Framework for Global Communication Optimization," Proc. Int'l Parallel Processing Symp. (IPPS '98), pp. 69-73, Mar. 1998.
33. M. Kandemir, P. Banerjee, A. Choudhary, J. Ramanujam, and N. Shenoy, "A Global Communication Optimization Technique Based on Data Flow Analysis and Linear Algebra," ACM Trans. Programming Languages and Systems (TOPLAS), vol. 21, no. 6, pp. 1,251-1,297, Nov. 1999.
34. W. Kelly, V. Maslov, W. Pugh, E. Rosser, T. Shpeisman, and D. Wonnacott, "The Omega Library Interface Guide," Technical Report CS-TR-3445, Dept. of Computer Science, Univ. of Maryland, College Park, Mar. 1995.
35. K. Kennedy and N. Nedeljkovic, "Combining Dependence and Data-Flow Analyses to Optimize Communication," Proc. Ninth Int'l Parallel Processing Symp., pp. 340-346, Apr. 1995.
36. K. Kennedy and A. Sethi, "Resource-Based Communication Placement Analysis," Languages and Compilers for Parallel Computing, D. Sehr et al., eds., Lecture Notes in Computer Science, vol. 1,239, pp. 369-388, Springer-Verlag, 1997.
37. J. Knoop, O. Ruthing, and B. Steffen, "Optimal Code Motion: Theory and Practice," ACM Trans. Programming Languages and Systems, vol. 16, no. 4, pp. 1,117-1,155, July 1994.
38. C. Koelbel, D. Lovemen, R. Schreiber, G. Steele, and M. Zosel, High Performance Fortran Handbook. The MIT Press, 1994.
39. D. Kranz, K. Johnson, A. Agarwal, J. Kubiatowicz, and B.H. Lim, "Integrating Message Passing and Shared Memory: Early Experience," Proc. Fourth ACM Symp. Principles and Practice of Parallel Programming (PPOPP '93), May 1993.
40. U. Kremer, "Automatic Data Layout for Distributed Memory Machines," PhD thesis, Technical Report CRPC-TR95559-S, Center for Research on Parallel Computation, Rice Univ., Houston, Tex., Oct. 1995.
41. Message Passing Interface Forum, MPI-2: Extensions to the Message-Passing Interface, July 1997.
42. M. O'Boyle and F. Bodin, "Compiler Reduction of Synchronization in Shared Virtual Memory Systems," Proc. Int'l Conf. Supercomputing, pp. 318-327, July 1995.
43. C. Polychronopoulos, M.B. Girkar, M.R. Haghighat, C.L. Lee, B.P. Leung, and D.A. Schouten, "Parafrase-2: An Environment for Parallelizing, Partitioning, Synchronizing, and Scheduling Programs on Multiprocessors," Proc. Int'l Conf. Parallel Processing, pp. II 39-48, Aug. 1989.
44. W. Pugh, "A Practical Algorithm for Exact Array Dependence Analysis," Comm. ACM, vol. 35, no. 8, pp. 102-114, Aug. 1992.
45. S. Reinhardt, J. Larus, and D. Wood, "Tempest and Typhoon: User-Level Shared Memory," Proc. 21st Int'l Symp. Computer Architecture, pp. 325-336, Apr. 1994.
46. A. Rogers and K. Pingali, "Process Decomposition through Locality of Reference," Proc. SIGPLAN '89 Conf. Programming Language Design and Implementation, pp. 69-80, June 1989.
47. S.L. Scott, "Synchronization and Communication in the T3E Multiprocessor," Proc. Seventh Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS '96), pp. 26-36, Oct. 1996.
48. T. Stricker, J. Stichnoth, D. O'Hallaron, S. Hinrichs, and T. Gross, Decoupling Synchronization and Data Transfer in Message Passing Systems of Parallel Computers," Proc. Ninth ACM Int'l Conf. Supercomputing, pp. 1-10, July 1995.
49. E. Su, "Compiler Framework for Distributed Memory Multicomputers," PhD thesis, Univ. of Illinois at Urbana-Champaign, Mar. 1997.
50. J. Subhlok, "Analysis of Synchronization in a Parallel Programming Environment," PhD thesis, Rice Univ., 1990.
51. C. Thekkath, H. Levy, E. Lazowska, "Separating Data and Control Transfer in Distributed Operating Systems," Proc. Sixth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 1-12, Oct. 1994.
52. C.-W. Tseng, "Compiler Optimizations for Eliminating Barrier Synchronization," Proc. Fifth ACM Symp. Principles and Practice of Parallel Programming (PPOPP '95), July 1995.
53. T. von Eicken, D. Culler, S. Goldstein, and K. Schauser, "Active Messages: A Mechanism for Integrated Communication and Computation," Proc. 19th Int'l Symp. Computer Architecture, pp. 256-266, May 1992.
54. M. Wolfe, High Performance Compilers for Parallel Computing. Addison-Wesley, 1996.
55. X. Yuan, R. Gupta, and R. Melhem, "An Array Data Flow Analysis Based Communication Optimizer," Proc. 10th Ann. Workshop Languages and Compilers for Parallel Computing, Aug. 1997.
56. X. Yuan, R. Gupta, and R. Melhem, "Demand-Driven Data Flow Analysis for Communication Optimization," Parallel Processing Letters, vol. 7, no. 4, pp. 359-370, Dec. 1997.
57. H. Zima, H. Bast, and M. Gerndt, "SUPERB: A Tool for Semi-Automatic MIMD/SIMD Parallelization," Parallel Computing, vol. 6, pp. 1-18, 1988.