Compiler Algorithms for Optimizing Locality and Parallelism on Shared and Distributed-Memory Machines

Journal of Parallel and Distributed Computing

Volumn 60, Issue 8, 2000, Pages 924-965

  Kandemir, M ^a   Ramanujam, J ^b   Choudhary, A ^c  

^a The Pennsylvania State University   (United States)

^b LOUISIANA STATE UNIVERSITY   (United States)

^c Northwestern University   (United States)

Author keywords

Array restructuring; Data reuse; Locality optimizations; Loop transformations; Memory performance; Parallelism; Spatial locality; Temporal locality

Indexed keywords

EID: 0000563616     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1006/jpdc.2000.1639     Document Type: Article

References (51)

1. Agarwal A., Bianchini R., Chaiken D., Johnson K. L., Kranz D., Kubiatowicz J., Lim B., Mackenzie K., Yeung D. The MIT Alewife machine: architecture and performance. Proc. 22nd International Symposium on Computer Architecture. 1995.
2. Anderson J. M., Amarasinghe S. P., Lam M. S. Data and computation transformations for multiprocessors. Proc. 5th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. July 1995.
3. Anderson J. M., Lam M. S. Global optimizations for parallelism and locality on scalable parallel machines. Proc. SIGPLAN Conference on Programming Language Design and Implementation. June 1993;. p. 112-125.
4. V. Balasundaram, G. Fox, K. Kennedy, and, U. Kremer, An interactive environment for data partitioning and distribution, in, 5th Distributed Memory Computing Conference, Charleston, SC, April 1990.
5. Banerjee P., Chandy J. A., Gupta M., Hodges E. W. IV, Holm J. G., Lain A., Palermo D. J., Ramaswamy S., Su E. The PARADIGM compiler for distributed-memory multicomputers. IEEE Comput. 28:October 1995;37-47.
6. Bik A. J. C., Wijshoff H. A. G. Technical Report. 1994.
7. Blume W., Doallo R., Eigenmann R., Grout J., Hoeflinger J., Lawrence T., Lee J., Padua D., Paek Y., Pottenger B., Rauchwerger L., Tu P. Parallel programming with Polaris. IEEE Comput. 29:December 1996;78-82.
8. Z. Bozkus, L. Meadows, D. Miles, S. Nakamoto, V. Schuster, and, M. Young, Techniques for compiling and executing HPF programs on shared-memory and distributed-memory parallel systems, in, Proc. 1st International Workshop on Parallel Processing, Bangalore, India, December 1994.
9. Brewer T., Astfalf G. The evolution of the HP/Convex Exemplar. Proc. COMPCON Spring'97: 42nd IEEE Computer Society International Conference. February 1997;. p. 81-86.
10. S. Carr, and, K. Kennedy, Blocking linear algebra codes for memory hierarchies, in, Proc. 4th SIAM Conference on Parallel Processing for Scientific Computing, Chicago, IL, December 1989.
11. R. Chandra, D. Chen, R. Cox, D. Maydan, N. Nedeljkovic, and, J. M. Anderson, Data-distribution support on distributed-shared memory multiprocessors, in, Proc. Programming Language Design and Implementation (PLDI), Las Vegas, NV, 1997.
12. M. Cierniak, and, W. Li, Unifying data and control transformations for distributed shared memory machines, in, Proc. SIGPLAN'95 Conference on Programming Language Design and Implementation, La Jolla, CA, June 1995.
13. S. Coleman, and, K. McKinley, Tile size selection using cache organization and data layout, in, Proc. SIGPLAN'95 Conference on Programming Language Design and Implementation, La Jolla, CA, June 1995.
14. Cormen T., Leiserson C., Rivest R. Introduction to Algorithms. 1990;The MIT Press, Cambridge.
15. K. M. Dixit, New CPU benchmark suites from SPEC, in, Proc. COMPCON'92 - 37th IEEE Computer Society International Conference, San Francisco, CA, February 1992.
16. Dongarra J. J., Croz J. D., Hammarling S., Duff I. A set of level 3 basic linear algebra subprograms. ACM Trans. Math. Software. 16:March 1990;1-17.
17. Gannon D., Jalby W., Gallivan K. Strategies for cache and local memory management by global program transformations. J. Parallel Distrib. Comput. 1988;587-616.
18. J. Garcia, E. Ayguade, and, J. Labarta, A novel approach towards automatic data distribution, in, Proc. Supercomputing'95, San Diego, December 1995.
19. Garey M. R., Johnson D. S. Computers and Intractability: A Guide to the Theory of NP-Completeness. 1979;Freeman, New York.
20. Gerndt M. Updating distributed variables in local computations. Concurrency Practice Experience. 2:September 1990;171-193.
21. Gupta M., Banerjee P. Demonstration of automatic data partitioning techniques for parallelizing compilers on multicomputers. IEEE Trans. Parallel Distrib. Systems. 3:March 1992;179-193.
22. Hennessy J. L., Patterson D. A. Computer Architecture: A Quantitative Approach. 1995;Morgan Kaufmann, San Mateo.
23. NWChem: A Computation Chemistry Package for Parallel Computers. 1995;Pacific Northwest Laboratory, Richland.
24. Hiranandani S., Kennedy K., Tseng C.-W. Compiling Fortran D for MIMD distributed-memory machines. Commun. Assoc. Comput. Mach. 35:August 1992;66-88.
25. Y.-J. Ju, and, H. Dietz, Reduction of cache coherence overhead by compiler data layout and loop transformations, in, Proc. 4th Workshop on Languages and Compilers for Parallel Computing, Santa Clara, CA, August 1991.
26. K. Kennedy, and, U. Kremer, Automatic data layout for High Performance Fortran, in, Proceedings of Supercomputing'95, San Diego, CA, December 1995.
27. K. Kennedy, and, K. S. McKinley, Optimizing for parallelism and data locality, in, Proc. 1992 ACM International Conference on Supercomputing (ICS'92), Washington, D.C, July 1992.
28. Kremer U. Automatic Data Layout for Distributed Memory Machines. 1995;Rice University, Houston.
29. M. S. Lam, E. Rothberg, and, M. E. Wolf, The cache performance and optimizations of blocked algorithms, in, Proc. 4th International Conference on Architectural Support for Programming Languages and Operating Systems, April, 1991.
30. J. Laudon, and, D. Lenoski, The SGI Origin: A CC-NUMA highly scalable server, in, Proc. 24th Annual International Symposium on Computer Architecture, May 1997.
31. D. Lenoski, J. Laudon, T. Joe, D. Nakahira, L. Stevens, A. Gupta, and J. Hennessy, The DASH prototype: implementation and performance, in Proc. 19th International Symposium on Computer Architecture, Gold Coast, Australia, May 1992, pp. 92-103.
32. Li W. Compiling for NUMA Parallel Machines. 1993;Cornell University, Ithaca.
33. McMahon F. Technical Report. 1986.
34. McKinley K., Carr S., Tseng C. W. Improving data locality with loop transformations. ACM Trans. Progr. Languages Systems. 18:July 1996;424-453.
35. M. O'Boyle and P. Knijnenburg, Non-singular data transformations: definition, validity, applications, in Proc. 6th Workshop on Compilers for Parallel Computers, Aachen, Germany, 1996, pp. 287-297.
36. D. Palermo and P. Banerjee, Automatic selection of dynamic data partitioning schemes for distributed-memory multicomputers, in Proc. 8th Workshop on Languages and Compilers for Parallel Computing, Columbus, OH, 1995, pp. 392-406.
37. C. Polychronopoulos, M. B. Girkar, M. R. Haghighat, C. L. Lee, B. P. Leung, and D. A. Schouten, Parafrase-2: an environment for parallelizing, partitioning, synchronizing, and scheduling programs on multiprocessors, in Proc. the International Conference on Parallel Processing, St. Charles, IL, August 1989, pp. 39-48.
38. J. Ramanujam, Non-unimodular transformations of nested loops, in Proc. Supercomputing 92, Minneapolis, MN, November 1992, pp. 214-223.
39. J. Ramanujam and A. Narayan, Integrating data distribution and loop transformations for distributed memory machines, in Proc. 7th SIAM Conference on Parallel Processing for Scientific Computing, February 1995 (D. Baileyet al., Eds.), pp. 668-673.
40. J. Ramanujam, and, A. Narayan, Automatic data mapping and program transformations, in, Proc. Workshop on Automatic Data Layout and Performance Prediction, Houston, TX, April 1995.
41. V. Sarkar, G. R. Gao, and, S. Han, Locality analysis for distributed shared-memory multiprocessors, in, Proc. the Ninth International Workshop on Languages and Compilers for Parallel Computing, Santa Clara, CA, August 1996.
42. Schrijver A. Theory of Linear and Integer Programming. Wiley-Interscience Series in Discrete Mathematics and Optimization. 1986;Wiley, New York.
43. T. J. Sheffler, R. Schreiber, J. R. Gilbert, and S. Chatterjee, Aligning parallel arrays to reduce communication, in Frontiers '95: The 5th Symposium on the Frontiers of Massively Parallel Computing, McLean, VA, February 1995, pp. 324-331.
44. C.-W. Tseng, J. Anderson, S. Amarasinghe, and, M. Lam, Unified compilation techniques for shared and distributed address space machines, in, Proc. 1995 International Conference on Supercomputing (ICS'95), Barcelona, Spain, July 1995.
45. R. Thekkath, A. P. Singh, J. P. Singh, S. John, and, J. Hennessey, An evaluation of a commercial CC-NUMA architecture: the CONVEX Exemplar SPP-1200, in, Proc. 11th International Parallel Processing Symposium, Geneva, Switzerland, April 1997.
46. E. Torrie, C.-W. Tseng, M. Martonosi, and, M. W. Hall, Evaluating the impact of advanced memory systems on compiler-parallelized codes, in, Proc. International Conference on Parallel Architectures and Compilations Techniques (PACT), Limassol, Cyprus, June 1995.
47. Wilson R. P., French R. S., Wilson C. S., Amarasinghe S. P., Anderson J. M., Tjiang S. W. K., Liao S.-W., Tseng C.-W., Hall M. W., Lam M. S., Hennessy J. L. SUIF: An infrastructure for research on parallelizing and optimizing compilers. ACM SIGPLAN Notices. 29:December 1994;31-37.
48. Wolf M., Lam M. A loop transformation theory and an algorithm to maximize parallelism. IEEE Trans. Parallel Distrib. Systems. 2:October 1991;452-471.
49. M. Wolf and M. Lam, A data locality optimizing algorithm, in Proc. ACM SIGPLAN 91 Conf. Programming Language Design and Implementation, June 1991, pp. 30-44.
50. Wolfe M. High Performance Compilers for Parallel Computing. 1996;Addison-Wesley, Reading.
51. Zima H., Chapman B. Compiling for distributed-memory systems. Proc. IEEE. 81:1993;264-287.