Can a shared-memory model serve as a bridging model for parallel computation?

Theory of Computing Systems

Volumn 32, Issue 3, 1999, Pages 327-359

  Gibbons, P B ^a   Matias, Y ^a,c   Ramachandran, V ^b  

^a LUCENT TECHNOLOGIES   (United States)

^b University of Texas at Austin   (United States)

^c TEL AVIV UNIVERSITY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ABSTRACTING; BANDWIDTH; CONFORMAL MAPPING; MESSAGE PASSING;

BANDWIDTH LIMITATION; BRIDGING MODELS; DATA PARALLELISM; DISTRIBUTED MEMORY; LIMITED COMMUNICATION; MAPPING ALGORITHMS; PARALLEL COMPUTATION; SHARED MEMORY MODEL;

MEMORY ARCHITECTURE;

EID: 0041353739     PISSN: 14324350     EISSN: 14330490     Source Type: Journal    
DOI: 10.1007/s002240000121     Document Type: Article

References (77)

1. M. Adler, P. B. Gibbons, Y. Matias, and V. Ramachandran. Modeling parallel bandwidth: local vs. global restrictions. In Proc. 9th ACM Symp. on Parallel Algorithms and Architectures, pages 94-105, June 1997.
2. A. Aggarwal, A. K. Chandra, and M. Snir. On communication latency in PRAM computations. In Proc. 1st ACM Symp. on Parallel Algorithms and Architectures, pages 11-21, June 1989.
3. A. Aggarwal, A. K. Chandra, and M. Snir. Communication complexity of PRAMs. Theoretical Computer Science, 71(1):3-28, 1990.
4. M. Ajtai, J. Komlos, and E. Szemeredi. Sorting in c lg n parallel steps. Combinatorica, 3(1):1-19, 1983.
5. A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Sheiman. LogGP: incorporating long messages into the LogP model - one step closer towards a realistic model for parallel computation. In Proc. 7th ACM Symp. on Parallel Algorithms and Architectures, pages 95-105, July 1995.
6. B. Alpern, L. Carter, and E. Feig. Uniform memory hierarchies. In Proc. 31st IEEE Symp. on Foundations of Computer Science, pages 600-608, October 1990.
7. R. J. Anderson and G. L. Miller. Optical Communication for Pointer Based Algorithms. Technical Report CRI 88-14, Computer Science Department, University of Southern California, Los Angeles, CA, 1988.
8. Y. Aumann and M. O. Rabin. Clock construction in fully asynchronous parallel systems and PRAM simulation. In Proc. 33rd IEEE Symp. on Foundations of Computer Science, pages 147-156, October 1992.
9. A. Bar-Noy, J. Bruck, C. T. Ho, S. Kipnis, and B. Schieber. Computing global combine operations in the multi-port postal model. In Proc. 5th IEEE Symp. on Parallel and Distributed Processing, pages 336-343, December 1993.
10. A. Bar-Noy and S. Kipnis. Designing broadcasting algorithms in the postal model for message-passing systems. In Proc. 4th ACM Symp. on Parallel Algorithms and Architectures, pages 13-22, June-July 1992.
11. A. Baumker and W. Dittrich. Fully dynamic search trees for an extension of the BSP model. In Proc. 8th ACM Symp. on Parallel Algorithms and Architectures, pages 233-242, June 1996.
12. A. Baumker, W. Dittrich, and F. Meyer auf der Heide. Truly Efficient Parallel Algorithms: 1-Optimal Multisearch for an Extension of the BSP Model. Technical Report, University of Paderborn, 1996.
13. P. Beame and J. Håstad. Optimal bounds for decision problems on the CRCW PRAM. Journal of the ACM, 36(3):643-670, July 1989.
14. G. E. Blelloch. Vector Models for Data-Parallel Computing. The MIT Press, Cambridge, MA, 1990.
15. G. E. Blelloch. Programming parallel algorithms. Communications of the ACM, 39(3):85-97, 1996.
16. G. E. Blelloch, P. B. Gibbons, Y. Matias, and M. Zagha. Accounting for memory bank contention and delay in high-bandwidth multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 8(9):943-958, 1997. Preliminary version appears in Proc. 7th ACM Symp. on Parallel Algorithms and Architectures, pages 84-94, July 1995.
17. G. E. Blelloch, P. B. Gibbons, Y. Matias, and M. Zagha. Accounting for memory bank contention and delay in high-bandwidth multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 8(9):943-958, 1997. Preliminary version appears in Proc. 7th ACM Symp. on Parallel Algorithms and Architectures, pages 84-94, July 1995.
18. G. E. Blelloch, P. B. Gibbons, Y. Matias, and M. Zagha. Accounting for memory bank contention and delay in high-bandwidth multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 8(9):943-958, 1997.
19. G. E. Blelloch, C. E. Leiserson, B. M. Maggs, C. G. Plaxton, S. J. Smith, and M. Zagha. A comparison of sorting algorithms for the Connection Machine CM-2. In Proc. 3rd ACM Symp. on Parallel Algorithms and Architectures, pages 3-16, July 1991.
20. R. Cole. Parallel merge sort. SIAM Journal on Computing, 17(4):770-785, 1988.
21. R. Cole and O. Zajicek. The APRAM: incorporating asynchrony into the PRAM model. In Proc. 1st ACM Symp. on Parallel Algorithms and Architectures, pages 169-178, June 1989.
22. R. Cole and O. Zajicek. The expected advantage of asynchrony. In Proc. 2nd ACM Symp. on Parallel Algorithms and Architectures, pages 85-94, July 1990.
23. D. Culler, R. Karp, D. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. LogP: towards a realistic model of parallel computation. In Proc. 4th ACM SIGPLAN Symp. on Principles and Practices of Parallel Programming, pages 1-12, May 1993.
24. R. Cypher and S. Konstantinidou. Bounds on the efficiency of message-passing protocols for parallel computers. In Proc. 5th ACM Symp. on Parallel Algorithms and Architectures, pages 173-181, June-July 1993.
25. A. Czumaj, Z. Galil, L. Ga̧sieniec, K. Park, and W. Plandowski. Work-time-optimal parallel algorithms for string problems. In Proc. 27th ACM Symp. on Theory of Computing, pages 713-722, May-June 1995.
26. P. de la Torre and C. P. Kruskal. Towards a single model of efficient computation in real parallel machines. Future Generation Computer Systems, 8:395-408, 1992.
27. P. de la Torre and C. P. Kruskal. Submachine locality in the bulk synchronous setting. In Proc. Euro-Par '96, pages 352-358, August 1996.
28. M. Dietzfelbinger and F. Meyer auf der Heide. Simple, efficient shared memory simulations. In Proc. 5th ACM Symp. on Parallel Algorithms and Architectures, pages 110-119, June-July 1993.
29. C. Dwork, M. Herlihy, and O. Waarts. Contention in shared memory algorithms. In Proc. 25th ACM Symp. on Theory of Computing, pages 174-183, May 1993.
30. S. Fortune and J. Wyllie. Parallelism in random access machines. In Proc. 10th ACM Symp. on Theory of Computing, pages 114-118, May 1978.
31. A. V. Gerbessiotis and L. Valiant. Direct bulk-synchronous parallel algorithms. Journal of Parallel and Distributed Computing, 22:251-267, 1994.
32. K. Gharachorloo, D. Lenoski, J. Laudon, P. Gibbons, A. Gupta, and J. Hennessy. Memory consistency and event ordering in scalable shared-memory multiprocessors. In Proc. 17th International Symp. on Computer Architecture, pages 15-26, May 1990.
33. P. B. Gibbons. A more practical PRAM model. In Proc. 1st ACM Symp. on Parallel Algorithms and Architectures, pages 158-168, June 1989. Full version in The Asynchronous PRAM: a semi-synchronous model for shared memory MIMD machines, Ph.D. thesis, University of California, Berkeley 1989.
34. P. B. Gibbons. A more practical PRAM model. In Proc. 1st ACM Symp. on Parallel Algorithms and Architectures, pages 158-168, June 1989. Full version in The Asynchronous PRAM: a semi-synchronous model for shared memory MIMD machines, Ph.D. thesis, University of California, Berkeley 1989.
35. P. B. Gibbons. What good are shared-memory models? In Proc. 1996 ICPP Workshop on Challenges for Parallel Processing, pages 103-114, August 1996. Invited position paper.
36. P. B. Gibbons, Y. Matias, and V. Ramachandran. Efficient low-contention parallel algorithms. Journal of Computer and System Sciences, 53(3):417-442, 1996. Special issue devoted to selected papers from the 1994 ACM Symp. on Parallel Algorithms and Architectures.
37. P. B. Gibbons, Y. Matias, and V. Ramachandran. Efficient low-contention parallel algorithms. Journal of Computer and System Sciences, 53(3):417-442, 1996. Special issue devoted to selected papers from the 1994 ACM Symp. on Parallel Algorithms and Architectures.
38. P. B. Gibbons, Y. Matias, and V. Ramachandran. The Queue-Read Queue-Write Asynchronous PRAM model. Theoretical Computer Science, 196:3-29, 1998.
39. P. B. Gibbons, Y. Matias, and V. Ramachandran. The Queue-Read Queue-Write PRAM model: accounting for contention in parallel algorithms. SIAM Journal on Computing, 1999. To appear. Preliminary version appears in Proc. 5th ACM-SIAM Symp. on Discrete Algorithms, pages 638-648, January 1994.
40. P. B. Gibbons, Y. Matias, and V. Ramachandran. The Queue-Read Queue-Write PRAM model: accounting for contention in parallel algorithms. SIAM Journal on Computing, 1999. To appear. Preliminary version appears in Proc. 5th ACM-SIAM Symp. on Discrete Algorithms, pages 638-648, January 1994.
41. M. Goodrich. Communication-efficient parallel sorting. In Proc. 28th ACM Symp. on Theory of Computing, pages 247-256, May 1996.
42. J. Greiner. A comparison of data-parallel algorithms for connected components. In Proc. 6th ACM Symp. on Parallel Algorithms and Architectures, pages 16-25, June 1994.
43. 3: an architecture-independent model for coarse-grained parallel machines. In Proc. 6th IEEE Symp. on Parallel and Distributed Processing, pages 544-551, 1994.
44. J. Hennessy and D. Patterson. Computer Architecture: A Quantitative Approach, second edition. Morgan Kaufmann, San Francisco, CA, 1996.
45. T. Heywood and S. Ranka. A practical hierarchical model of parallel computation: I. The model. Journal of Parallel and Distributed Computing, 16:212-232, 1992.
46. T-S. Hsu, V. Ramachandran, and N. Dean. Parallel implementation of algorithms for finding connected components in graphs. In Proc. AMS/DIMACS Parallel Implementation Challenge Workshop III, pages 395-416. DIMACS Series. American Mathematical Society, Providence, RI, 1997.
47. J. JáJá. An Introduction to Parallel Algorithms. Addison-Wesley, Reading, MA, 1992.
48. J. JáJá and K. W. Ryu. The Block Distributed Memory Model. Technical Report UMIACS-TR-94-5, Institute for Advanced Computer Studies, University of Maryland, College Park, MD, January 1994.
49. B. H. H. Juurlink. Ph.D. Thesis, Leiden University, 1996.
50. B. H. H. Juurlink. and H. A. G. Wijshoff. The E-BSP model: incorporating general locality and unbalanced communication into the BSP Model. In Proc. Euro-Par '96, pages 339-347, August 1996.
51. R. Karp, A. Sahay, E. Santos, and K.E. Schauser. Optimal broadcast and summation in the LogP model. In Proc. 5th ACM Symp. on Parallel Algorithms and Architectures, pages 142-153, June-July 1993.
52. R. M. Karp and V. Ramachandran. Parallel algorithms for shared-memory machines. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science, Volume A, pages 869-941. Elsevier, Amsterdam, 1990.
53. Z. M. Kedem, K. V. Palem, M. O. Rabin, and A. Raghunathan. Efficient program transformations for resilient parallel computation via randomization. In Proc. 24th ACM Symp. on Theory of Computing, pages 306-317, May 1992.
54. K. Kennedy. A research agenda for high performance computing software. In Developing a Computer Science Agenda for High-Performance Computing, pages 106-109. ACM Press, New York, 1994.
55. F. T. Leighton. Introduction to Parallel Algorithms and Architectures: Arrays · Trees · Hypercubes. Morgan Kaufmann. San Mateo, CA, 1992.
56. C. E. Leiserson and B. M. Maggs. Communication-efficient parallel algorithms for distributed random-access machines. Algorithmica, 3(1):53-77, 1988.
57. P. Liu, W. Aiello, and S. Bhatt. An atomic model for message-passing. In Proc. 5th ACM Symp. on Parallel Algorithms and Architectures, pages 154-163, June-July 1993.
58. P. D. MacKenzie and V. Ramachandran. Computational bounds for fundamental problems on general-purpose parallel models. In Proc. 10th ACM Symp. on Parallel Algorithms and Architectures, pages 152-163, June-July 1998.
59. B. M. Maggs, L. R. Matheson, and R. E. Tarjan. Models of parallel computation: a survey and synthesis. In Proc. 28th Hawaii International Conf. on System Sciences, pages II: 61-70, January 1995.
60. Y. Mansour, N. Nisan, and U. Vishkin. Trade-offs between communication throughput and parallel time. In Proc. 26th ACM Symp. on Theory of Computing, pages 372-381, 1994.
61. C. Martel, A. Park, and R. Subramonian. Work-optimal asynchronous algorithms for shared memory parallel computers. SIAM Journal on Computing, 21 (6):1070-1099, 1992.
62. R. P. Martin, A. M. Vahdat, D. E. Culler, and T. E. Anderson. Effects of communication latency, overhead, and bandwidth in a cluster architecture. In Proc. 24th International Symp. on Computer Architecture, pages 85-97, June 1997.
63. W. F. McColl. A BSP Realization of Strassen's Algorithm. Technical Report, Computing Laboratory, Oxford University, May 1995.
64. K. Melhorn and U. Vishkin. Randomized and deterministic simultaitons of PRAMs by parallel machines with restricted granularity of parallel memoríes. Acta Informatica, 21:339-374, 1984.
65. N. Nishimura. Asynchronous shared memory parallel computation. In Proc. 2nd ACM Symp. on Parallel Algorithms and Architectures, pages 76-84, July 1990.
66. M. H. Nodine and J. S. Vitter. Large-scale sorting in parallel memories. In Proc. 3rd ACM Symp. on Parallel Algorithms and Architectures, pages 29-39, July 1991.
67. P. Raghavan. Probabilistic construction of deterministic algorithms: approximating packing integer programs. Journal of Computer and System Sciences, 37:130-143, 1988.
68. V. Ramachandran. A general purpose shared memory model for parallel computation. In Algorithms for Parallel Processing, Vol. 105. IMA Volumes in Mathematics and Its Applications. Springer-Verlag, New York, in press.
69. J. H. Reif, editor. A Synthesis of Parallel Algorithms. Morgan Kaufmann, San Mateo, CA, 1993.
70. J. H. Reif and S. Sen. Randomized algorithms for binary search and load balancing on fixed connection networks with geometric applications. In Proc. 2nd ACM Symp. on Parallel Algorithms and Architectures, pages 327-337, July 1990.
71. B. Smith. Invited lecture, 7th ACM Symp. on Parallel Algorithms and Architectures, July 1995.
72. P. Stenström, T. Joe, and A. Gupta. Comparative performance evaluation of cache-coherent NUMA and COMA architectures. In Proc. 19th International Symp. on Computer Architecture, pages 80-91, May 1992.
73. L. G. Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8):103-111, 1990.
74. L. G. Valiant. General purpose parallel architectures. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science. Volume A, pages 943-972. Elsevier, Amsterdam, 1990.
75. U. Vishkin. A parallel-design distributed-implementation (PDDI) general purpose computer. Theoretical Computer Science, 32:157-172, 1984.
76. J. S. Vitter and E. A. M. Shriver. Optimal disk I/O with parallel block transfer. In Proc. 22nd ACM Symp. on Theory of Computing, pages 159-169, May 1990.
77. H. A. G. Wijshoff and B. H. H. Juurlink. A quantitative comparison of parallel computation models. In Proc. 8th ACM Symp. on Parallel Algorithms and Architectures, pages 13-24, June 1996.