Optimizing communication overlap for high-speed networks

Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

Volumn , Issue , 2007, Pages 35-45

  Iancu, Costin C ^a   Strohmaier, Erich ^a  

^a LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

Communication overlap; High speed networks; Latency hiding; LogGP; LogP; Performance model; Variability

Indexed keywords

COMPUTER HARDWARE; CONSTRAINT THEORY; MATHEMATICAL MODELS; OPTIMIZATION; PARALLEL PROCESSING SYSTEMS; PARAMETER ESTIMATION;

COMMUNICATION OVERLAP; IMPLEMENTATION PARAMETERS; LATENCY HIDING; PERFORMANCE MODELS;

HIGH SPEED NETWORKS;

EID: 34748877192     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1229428.1229436     Document Type: Conference Paper

References (25)

1. A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Scheiman. LogGP: Incorporating Long Messages into the LogP Model for Parallel Computation. Journal of Parallel and Distributed Computing, 44(1):71-79, 1997.
2. C. Bell, D. Bonachea, Y. Cote, J. Duell, P. Husbands, P. Hargrove, C. Iancu, M. Welcome, and K. Yelick. An Evaluation of Current High-Performance Networks. In Proceedings of 17th International. Parallel and Distributed Processing Symposium (IPDPS), 2003.
3. C. Bell, D. Bonachea, R. Nishtala, and K. Yelick. Optimizing Bandwidth Limited Problems Using One-Sided Communication and Overlap. Proceedings of the 20th International Parallel and Distributed Processing Symposium (IPDPS), 2006.
4. D. Bonachea. GASNet Specification, vl.l. Technical. Report CSD-02-1207, University of California at Berkeley, October 2002.
5. R. Brightwell, D. Doerfler, and K. Underwood. A Comparison of 4X InfiniBand and Quadrics Elan-4 Technologies. 2004 IEEE International Conference on Cluster Computing, Sept 2004.
6. R. Brightwell, R. Riesen, and K. D. Underwood. Analyzing the Impact of Overlap, Offload, and Independent Progress for Message Passing Interface Applications. International Journal of High Performance Computing Applications, May 2005.
7. C. Cascaval, E. Duesterwald, P. F. Sweeney, and R. W. Wisniewski. Multiple Page Size Modeling and Optimization. Proceedings of IEEE Parallel Architectures and Compilation Techniques (PACT), 2005.
8. D. E. Culler, R. M. Karp, D. A. Patterson, A. Sahay, K. E. Schauser, E. Santos, R. Subramonian, and T. von Eicken. LogP: Towards a Realistic Model of Parallel Computation. In Principles Practice of Parallel Programming, pages 1-12, 1993.
9. A. Danalis, K.-Y. Kim, L. Pollock, and M. Swany. Transformations to Parallel Codes for Communication-Computation Overlap. In Proceedings of the 2005 ACM/IEEE Conference on Supervomputing (SC05), page 58, 2005.
10. D.Roweth and A.Moody. Performance of All-to-All on QsNetll. Available at http://www.quadrics.net.
11. Jacquard AMD Opteron cluster. LBNL National Energy Research. Supercomputing Center.
12. N. Koziris, A. Sotiropoulos, and G. I. Goumas. A Pipelined Schedule to Minimize Completion Time for Loop Tiling with Computation and Communication Overlapping. Journal of Parallel and Distributed Computing, 63(11):1138-1151, 2003.
13. M. Krishnan and J. Nieplocha. Optimizing Performance on Linux Clusters Using Advanced Communication Protocols: Achieving Over 10 Teraflops on a 8.6 Teraflops Linpack-Rated Linux Cluster. Proceedings of the 6th International Conference on Linux clusters: The HPC Rewlution, 2005.
14. S. Kumar and L. V. Kale. Scaling All-to-All Multicast on Fat-tree Networks. In ICPADS '04: Proceedings of the Parallel and Distributed Systems, Tenth International Conference on (ICPADS'04), page 205, 2004.
15. Lemieux. http://www.psc.edU/machines/tcs/l.emieux.html.
16. J. Liu, B. Chandrasekaran, J. Jiang, S. Kini, W. Yu, D. Buntinas, P. Wyckoff, and D. Panda. Performance Comparison of MPI Implementations over InfiniBand Myrinet and Quadrics. In Proceedings of the 2003 ACM/IEEE Conference on Supercomputing (SC03), 2003.
17. C. A. Moritz and M. I. Frank. LoGPC: Modeling Network Contention in Message-Passing Programs. In Proceedings of 2002 IEEE Transactions on Parallel and Distributed Systems, volume 12-4, April 200.1.
18. L. Oliker, S. Kamil, J.Shalf, and D. Skinner. Understanding UltraScale Application Communication Requirements. IEEE International Symposium on Workload. Characterization (IISWC), 2005.
19. H. E. B. Thilo Kielmann and K. Verstoep. Fast Measurement of LogP Parameters for Message Passing Platforms. In 4th Workshop on Runtime Systems for Parallel Programming (RTSPP), 2000.
20. V. Tipparaju, M. Krishnan, J. Nieplocha, and D. P. G. Santhanaraman. Exploiting Non-blocking Remote Memory Access Communication in Scientific Benchmarks. Proceedings of the 2003 International Conference on High Performance Computng, HiPC'2003, 2003.
21. V. Tipparaju and J. Nieplocha. Optimizing All-to-All Collective Communication by Exploiting Concurrency in Modern Networks. Proceedings of the 2005 ACM/IEEE Conference on Supercomputing, 2005.
22. K. D. Underwood and R. Brightwell. The Impact of MPI Queue Usage on Message Latency. In The 2004 International Conference on Parallel Processing (ICPP-04), pages 152-160, 2004.
23. UPC Language Specification, Version 1.0. Available at http://upc.gwu.edu.
24. J. Vetter and F. Mueller. Communication Characteristics of LargeScale Scientific Applications for Contemporary Cluster Architectures. Proceedings of the 2002 International Parallel and Distributed Processing Symposium (IPDPS), 2002.
25. M.-S. Wu, R. A. Kendall, K. Wright, and Z. Zhang. Performance Modeling and Tuning Strategies of Mixed Mode Collective Communications. Proceedings of the 2005 ACM/IEEE Conference, on Supercomputing, 2005.