High performance MPI design using unreliable datagram for ultra-scale InfiniBand clusters

Proceedings of the International Conference on Supercomputing

Volumn , Issue , 2007, Pages 180-189

  Koop, Matthew J ^a   Sur, Sayantan ^a   Gao, Qi ^a   Panda, Dhabaleswar K ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

InfiniBand; Memory scalability; MPI; Unreliable datagram

Indexed keywords

DATA STORAGE EQUIPMENT; MATHEMATICAL MODELS; MESSAGE PASSING; PARALLEL PROCESSING SYSTEMS;

INFINIBAND; MEMORY SCALABILITY; MPI; UNRELIABLE DATAGRAM;

CLUSTER ANALYSIS;

EID: 34548008852     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1274971.1274997     Document Type: Conference Paper

References (33)

1. Mellanox Technologies. http://www.mellanox.com.
2. ASC. ASC Purple Benchmarks. http://www.llnl.gov/asci/purple/benchmarks/.
3. R. T. Aulwes, D. J. Daniel, N. N. Desai, R. L. Graham, L. Risinger, M. W. Sukalski, and M. A. Taylor. Network Fault Tolerance in LA-MPI. In Proceedings of EuroPVM/MPI '03, September 2003.
4. D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, D. Dagum, R. A. Fatoohi, P. O. Frederickson, T. A. Lasinski, R. S. Schreiber, H. D. Simon, V. Venkatakrishnan, and S. K. Weeratunga. The NAS parallel benchmarks, volume 5, pages 63-73, Fall 1991.
5. R. Brightwell and A. B. Maccabe. Scalability Limitations of VIA-Based Technologies in Supporting MPI. In Fourth MPI Developer's and User's Conference, 2000.
6. P. N. Brown, R. D. Falgout, and J. E. Jones. Semicoarsening multigrid on distributed memory machines. SIAM Journal on Scientific Computing, 21(5):1823-1834, 2000.
7. Compaq, Intel, and Microsoft. VI Architecture Specification V1.0, December 1997.
8. E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings, 11th European PVM/MPI Users' Group Meeting, pages 97-104, Budapest, Hungary, September 2004.
9. P. Gilfeather and A. B. Maccabe. Connection-less TCP. In 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Workshop 9, 2005.
10. R. L. Graham, S.-E. Choi, D. J. Daniel, N. N. Desai, R. G. Minnich, C. E. Rasmussen, L. D. Risinger, and M. W. Sukalksi. A Network-Failure-Tolerant Message-Passing System for Terascale Clusters. International Journal of Parallel Programming, 31(4), August 2003.
11. W. Gropp, E. Lusk, N. Doss, and A. Skjellum. A High-Performance, Portable Implementation of the MPI, Message Passing Interface Standard. Technical report, Argonne National Laboratory and Mississippi State University.
12. A. Hoisie, O. M. Lubeck, H. J. Wasserman, F. Petrini, and H. Aime. A General Predictive Performance Model for Wavefront Algorithms on Clusters of SMPs. In International Conference on Parallel Processing, pages 219-, 2000.
13. InfiniBand Trade Association. InfiniBand Architecture Specification, http://www.infinibandta.com.
14. K. Koch, R. Baker, and R. Alcouffe. Solution of the First-Order Form of the 3-D Discrete Ordinates Equation on a Massively Parallel Processor. Trans. of American Nuclear Society, pages 65-, 1992.
15. M. Koop, T. Jones, and D. K. Panda. Reducing Connection Memory Requirements of MPI for InfiniBand Clusters: A Message Coalescing Approach. In 7th IEEE Int'l Symposium on Cluster Computing and the Grid (CCGrid07), May 2007.
16. Lawrence Berkeley National Laboratory. MVICH: MPI for Virtual Interface Architecture. http://www.nersc.gov/research/FTG/mvich/index.html, August 2001.
17. J. Liu, B. Chandrasekaran, J. Wu, W. Jiang, S. Kini, W. Yu, D. Buntinas, P. Wyckoff, and D. K. Panda. Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics. In Supercomputing(SC), 2003.
18. J. Liu, A. Mamidala, and D. K. Panda. Fast and Scalable MPI-Level Broadcast using InfiniBand's Hardware Multicast Support. In Proceedings of Int'l Parallel and Distributed Processing Symposium (IPDPS 04), April 2004.
19. J. Liu, J. Wu, S. P. Kini, P. Wyckoff, and D. K. Panda. High Performance RDMA-Based MPI Implementation over InfiniBand. In 17th Annual ACM International Conference on Supercomputing (ICS '03), June 2003.
20. A. R. Mamidala, J. Liu, and D. K. Panda. Efficient Barrier and Allreduce on InfiniBand Clusters using Hardware Multicast and Adaptive Algorithms. In Proceedings of IEEE Cluster Computing, 2004.
21. A. R. Mamidala, S. Narravula, A. Vishnu, G. Santhanaraman, and D. K. Panda. On using connection-oriented vs. connection-less transport for performance and scalability of collective and one-sided operations: trade-offs and impact. In PPoPP '07: Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming, pages 46-54. ACM Press, 2007.
22. Message Passing Interface Forum. MPI: A Message-Passing Interface Standard, Mar 1994.
23. A. A. Mirin, R. H. Cohen, B. C. Curtis, W. P. Dannevik, A. M. Dimits, M. A. Duchaineau, D. E. Eliason, D. R. Schikore, S. E. Anderson, D. H. Porter, P. R. Woodward, L. J. Shieh, and S. W. White. Very high resolution simulation of compressible turbulence on the IBM-SP system. In Supercomputing '99: Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM), page 70, New York, NY, USA, 1999. ACM Press.
24. OpenFabrics Alliance. OpenFabrics. http://www.openfabrics.org/.
25. F. Petrini, D. J. Kerbyson, and S. Pakin. The Case of the Missing Supercomputer Performance: Achieving Optimal Performance on the 8,192 Processors of ASCI Q. In SC '03: Proceedings of the 2003 ACM/IEEE conference on Supercomputing, page 55. IEEE Computer Society, 2003.
26. G. Shipman, T. Woodall, R. Graham, and A. Maccabe. Infiniband Scalability in Open MPI. In International Parallel and Distributed Processing Symposium (IPDPS), 2006.
27. S. Sur, L. Chai, H.-W. Jin, and D. K. Panda. Shared Receive Queue Based Scalable MPI Design for InfiniBand Clusters. In International Parallel and Distributed Processing Symposium (IPDPS), 2006.
28. S. Sur, M. J. Koop, and D. K. Panda. High-Performance and Scalable MPI over InfiniBand with Reduced Memory Usage: An In-Depth Performance Analysis. In Super Computing, 2006.
29. S. Sur, A. Vishnu, H. W Jin, W. Huang, and D. K. Panda. Can Memory-Less Network Adapters Benefit Next-Generation InfiniBand Systems? In Hot Interconnect (HOTI 05), 2005.
30. J. Vetter and F. Mueller. Communication characteristics of large-scale scientific applications for contemporary cluster architectures. In IPDPS '02: Proceedings of the 16th International Symposium on Parallel and Distributed Processing, page 27.2, Washington, DC, USA, 2002. IEEE Computer Society.
31. J. S. Vetter and A. Yoo. An empirical performance evaluation of scalable scientific applications. In Supercomputing '02: Proceedings of the 2002 ACM/IEEE conference on Supercomputing, pages 1-18, Los Alamitos, CA, USA, 2002. IEEE Computer Society Press.
32. J. Wu, J. Liu, P. Wyckoff, and D. Panda. Impact of on-demand connection management in mpi over via. In CLUSTER '02: Proceedings of the IEEE International Conference on Cluster Computing, page 152, Washington, DC, USA, 2002. IEEE Computer Society.
33. W. Yu, Q. Gao, and D. K. Panda. Adaptive Connection Management for Scalable MPI over InfiniBand. In International Parallel and Distributed Processing Symposium (IPDPS), 2006.